JP7346066B2 - Medical information processing device and medical information processing method - Google Patents

Description

Embodiments of the present invention relate to a medical information processing device and a medical information processing method.

In recent years, with the advancement of medical technology, the types and amounts of medical data collected in daily medical care have increased, and the methods by which doctors refer to medical data to determine diagnoses and treatment plans have also become more complex. . For this reason, in the medical field, there is a need for a system that displays on a single screen various medical data necessary for doctors to consider diagnosis and treatment plans.

Special table 2017-537368 publication JP 2017-042592 Publication Special table 2018-509689 publication

The problem to be solved by the present invention is to enable an operator to easily conduct multifaceted studies using various medical data.

The medical information processing device according to the embodiment includes a display control section and a changing section. The display control unit arranges various types of medical data in a virtual space having a plurality of dimensions, and displays a plurality of pieces of medical data falling within a visual field on the virtual space on a screen of a display. The changing unit displays at least one medical data among the plurality of medical data on the screen while changing the display of other medical data on the screen in the virtual space. By changing the visual field range, the display of medical data on the screen is changed.

FIG. 1 is a diagram showing a configuration example of a medical information processing apparatus according to a first embodiment. FIG. 2 is a diagram showing an example of the display of medical data performed by the display control function according to the first embodiment. FIG. 3 is a diagram illustrating an example of a display change of medical data performed by the change function according to the first embodiment. FIG. 4 is a diagram illustrating another example of a display change of medical data performed by the change function according to the first embodiment. FIG. 5 is a diagram illustrating another example of display change of medical data performed by the change function according to the first embodiment. FIG. 6 is a diagram illustrating another example of a display change of medical data performed by the change function according to the first embodiment. FIG. 7 is a diagram illustrating another example of a display change of medical data performed by the change function according to the first embodiment. FIG. 8 is a flowchart showing a processing procedure of processing realized by each processing function of the processing circuit according to the first embodiment. FIG. 9 is a diagram showing an overview of the display of medical data performed by the medical information processing apparatus according to the second embodiment. FIG. 10 is a diagram illustrating an example of classification of panels of medical data according to the second embodiment. FIG. 11 is a diagram showing an example of coordinates assigned to each medical data panel by the display control function according to the second embodiment. FIG. 12 is a diagram illustrating an example of enlargement conversion performed by the change function according to the second embodiment. FIG. 13 is a diagram illustrating another example of enlargement conversion performed by the change function according to the second embodiment. FIG. 14 is a diagram illustrating an example of reduction conversion performed by the change function according to the second embodiment. FIG. 15 is a diagram illustrating another example of reduction conversion performed by the change function according to the second embodiment. FIG. 16 is a diagram illustrating an example of rotation conversion performed by the change function according to the second embodiment. FIG. 17 is a diagram illustrating another example of rotation conversion performed by the change function according to the second embodiment. FIG. 18 is a diagram illustrating an example of parallel translation transformation performed by the change function according to the second embodiment. FIG. 19 is a diagram illustrating another example of parallel translation transformation performed by the change function according to the second embodiment.

Hereinafter, embodiments of a medical information processing device and a medical information processing method will be described in detail with reference to the drawings.

(First embodiment)
FIG. 1 is a diagram showing a configuration example of a medical information processing apparatus according to a first embodiment.

For example, as shown in FIG. 1, a medical information processing apparatus 100 according to the present embodiment is communicably connected to an electronic medical record system 300, a radiology department system 400, a specimen testing system 500, etc. via a network 200. . For example, the medical information processing device 100 and each system are installed in a hospital or the like, and are interconnected by a network 200 such as an in-hospital LAN (Local Area Network).

The electronic medical record system 300 generates medical data regarding prescriptions and nursing records administered to the subject and stores it in a storage circuit within the system. Then, the electronic medical record system 300 transmits the medical care data stored in the storage circuit to the medical information processing apparatus 100 in response to a request from the medical information processing apparatus 100.

The radiology department system 400 generates medical data regarding vital signs and image tests performed on a subject, and stores it in a storage circuit within the system. The image tests mentioned here include tests using CT images taken by an X-ray CT (Computed Tomography) device, tests using MR images taken by an MRI (Magnetic Resonance Imaging) device, and tests using an ultrasound diagnostic device. This includes examinations using ultrasound images taken by an X-ray diagnostic device, examinations using X-ray images taken by an X-ray diagnostic device, and the like. For example, the radiology department system 400 includes a PACS (Picture Archiving and Communication System), etc., and can generate CT images, MR images, ultrasound images, and X-ray images in a format compliant with DICOM (Digital Imaging and Communications in Medicine). and other medical images are stored in a database. The radiology department system 400 then transmits the medical data stored in the storage circuit to the medical information processing apparatus 100 in response to a request from the medical information processing apparatus 100.

The specimen testing system 500 generates medical data regarding specimen tests performed on a subject and stores it in a storage circuit within the system. The specimen testing system 500 then transmits the medical data stored in the storage circuit to the medical information processing apparatus 100 in response to a request from the medical information processing apparatus 100.

The medical information processing apparatus 100 acquires various medical data from the electronic medical record system 300, the radiology department system 400, and the specimen testing system 500 via the network 200, and performs various information processing using the acquired medical data. For example, the medical information processing apparatus 100 is realized by a computer device such as a workstation, a personal computer, or a tablet terminal.

Specifically, the medical information processing device 100 includes an NW (network) interface 110, a storage circuit 120, an input interface 130, a display 140, and a processing circuit 150.

The NW interface 110 is connected to the processing circuit 150 and controls transmission and communication of various data between the medical information processing apparatus 100 and each system. Specifically, the NW interface 110 receives medical care data from each system and outputs the received medical care data to the processing circuit 150. For example, the NW interface 110 is realized by a network card, network adapter, NIC (Network Interface Controller), or the like.

The storage circuit 120 is connected to the processing circuit 150 and stores various data. Specifically, the storage circuit 120 stores medical data received from each system. For example, the memory circuit 120 is realized by a semiconductor memory element such as a RAM (Random Access Memory) or a flash memory, a hard disk, an optical disk, or the like. Note that the memory circuit 120 is an example of a memory section.

The input interface 130 is connected to the processing circuit 150 and receives various instructions and input operations for various information from the operator. Specifically, the input interface 130 converts an input operation received from an operator into an electrical signal and outputs it to the processing circuit 150. For example, the input interface 130 may include a trackball, a switch button, a mouse, a keyboard, a touchpad that performs input operations by touching the operation surface, a touchscreen that integrates a display screen and a touchpad, and a non-control device that uses an optical sensor. This is realized by a touch input circuit, a voice input circuit, etc. Note that in this specification, the input interface 130 is not limited to one that includes physical operation parts such as a mouse and a keyboard. For example, examples of the input interface 130 include an electrical signal processing circuit that receives an electrical signal corresponding to an input operation from an external input device provided separately from the device and outputs this electrical signal to a control circuit.

The display 140 is connected to the processing circuit 150 and displays various information and images. Specifically, the display 140 converts various information and various image data sent from the processing circuit 150 into electrical signals for display, and outputs the electrical signals. For example, the display 140 is realized by a liquid crystal monitor, a CRT (cathode ray tube) monitor, a touch panel, or the like.

The processing circuit 150 controls the components of the medical information processing apparatus 100 according to input operations received from the operator via the input interface 130. Specifically, the processing circuit 150 causes the storage circuit 120 to store the medical care data output from the NW interface 110. Furthermore, the processing circuit 150 reads medical data from the storage circuit 120 and displays it on the display 140.

The overall configuration of the medical information processing apparatus 100 according to the present embodiment has been described above. With such a configuration, the medical information processing apparatus 100 according to the present embodiment displays various medical data generated by the electronic medical record system 300, the radiology department system 400, and the specimen testing system 500 on one screen. It has a function. The medical information processing apparatus 100 according to the present embodiment is configured so that the operator can easily conduct multifaceted studies using various medical data.

Specifically, in this embodiment, a display device for VR (Virtual Reality), MR (Mixed Reality), or AR (Augmented Reality) is used as the display 140. For example, the display 140 is realized by a head mounted display (HMD), goggles for VR, MR, or AR, a high resolution display such as an 8K monitor, or the like.

Further, in this embodiment, an input device for VR, MR, or AR is used as the input interface 130. For example, the input interface 130 is realized by a motion sensor that detects the operator's motion, a line-of-sight sensor that detects the operator's line of sight, a controller for VR, MR, or AR that receives the operator's operation, or the like.

In this embodiment, the processing circuit 150 has an acquisition function 151, a display control function 152, and a change function 153.

The acquisition function 151 acquires various medical data from the electronic medical record system 300 , radiology department system 400 , and specimen testing system 500 via the network 200 .

Specifically, the acquisition function 151 acquires various medical data regarding the subject to be diagnosed specified by the operator from each system, and stores the acquired medical data in the storage circuit 120. Here, the acquisition function 151 may cause the storage circuit 120 to store the medical data obtained from each system in an unchanged format, or may store it in an appropriately normalized format.

The display control function 152 arranges various types of medical data in a virtual space having a plurality of dimensions, and displays a plurality of pieces of medical data that fall within the visual field on the virtual space on the screen of the display 140.

Specifically, the display control function 152 reads various medical data acquired from each system by the acquisition function 151 from the storage circuit 120, and arranges the read medical data in a predefined virtual space. Then, the display control function 152 displays medical data that falls within a predefined field of view in the virtual space on the screen of the display 140.

In the present embodiment, the display control function 152 provides a plurality of panels displaying the contents of the medical data in a virtual space having two dimensions, top and bottom and left and right, or in a virtual space having three dimensions, top and bottom, left and right, and front and back. are arranged, and medical data that falls within the visual field in the virtual space is displayed on the screen of the display 140.

FIG. 2 is a diagram showing an example of the display of medical data performed by the display control function 152 according to the first embodiment.

For example, as shown in FIG. 2, the display control function 152 arranges in the virtual space 1 a timeline panel 2 that shows the timing of occurrence of each medical event for a plurality of pieces of medical care data in chronological order. The display control function 152 also displays a panel 4 for image data corresponding to the attention range 3 specified on the timeline by the operator, a panel 5 for medication data, etc., and a panel 5 for nursing record data under the panel 2 of the timeline. The panels 6 are arranged in order from left to right. Here, the timeline, image data, medication data, and nursing record data are each examples of medical data. Then, the display control function 152 displays, on the screen of the display 140, the panels that fall within the viewing range 7 among the plurality of panels arranged in the virtual space 1.

The change function 153 displays at least one piece of medical data among the plurality of pieces of medical data displayed on the screen of the display 140 so that it does not fall off the screen, while changing the display of other medical data on the screen. By changing the field of view within the virtual space, the display of medical data on the screen is changed.

Specifically, the change function 153 receives a change instruction based on an operator's movement, line of sight, or operation, and changes the display of medical data on the screen according to the received change instruction. At this time, for example, the change function 153 may issue a change instruction such as an action in which the operator turns his or her head in a predetermined direction (left/right direction, up/down direction, etc.), or an action in which the operator moves in a given direction (forward/backward direction, left/right direction, etc.). , that the operator has moved to a certain position or angle of medical data that he/she wants to view in the virtual space, that the operator has continued to gaze at a predetermined position on the screen (the position of an icon or panel on the screen, etc.) for a certain period of time; It receives that the operator flicks his/her line of sight in a predetermined direction (up/down direction, left/right direction, etc.).

For example, the change function 153 may be configured based on an operator's motion detected by a motion sensor used as the input interface 130, an operator's line of sight detected by a line-of-sight sensor, or a change instruction received by the controller. and changes the display of medical data on the screen according to the received change instruction.

FIG. 3 is a diagram showing an example of a display change of medical data performed by the change function 153 according to the first embodiment.

Here, as shown in Figure 2, a timeline panel is arranged in the virtual space, and below it are a panel of image data and medication data as medical data corresponding to the range of interest specified on the timeline. It is assumed that the panels such as , and the panels of nursing record data are arranged in order from left to right. Further, here, it is assumed that an HMD is used as the display 140.

For example, as shown in FIG. 3(a), among the medical data placed in the virtual space, a part of the timeline panel including the attention range and an image data panel are used as panels that fall within the field of view. Assume that a panel for displaying medication data, medication data, etc. is displayed on the screen 8 of the display 140.

In this case, for example, the change function 153 accepts the action of the operator O turning his or her head from side to side as a change instruction. Then, the change function 153 moves the image data panel, medication data, etc. panel, and nursing record data panel left and right on the screen 8 in a state where they are arranged side by side in the left and right direction in accordance with the received change instruction. to change the viewing range within virtual space. At this time, the change function 153 keeps the timeline panel within the viewing range so that the attention range 3 specified on the timeline does not deviate from the screen 8.

For example, as shown in FIG. 3B, when the operator O performs an action of turning his head from left to right, the change function 153 changes the image data panel, medication data, etc. on the screen 8. and the nursing record data panel to the left. Further, at this time, the change function 153 fixes the position of the timeline panel on the screen 8 so that the attention range 3 specified on the timeline does not deviate from the screen 8.

For example, as shown in FIG. 3(c), when the operator O continues turning his head from left to right, the change function 153 changes the image data panel on the screen 8, The panel for medication data, etc. and the panel for nursing record data are further moved to the left, so that the panel for medication data, etc. and the panel for nursing record data are displayed on the screen 8. At this time, the change function 153 also fixes the position of the timeline panel on the screen 8 so that the attention range 3 specified on the timeline does not deviate from the screen 8.

FIG. 4 is a diagram showing another example of a display change of medical data performed by the change function 153 according to the first embodiment.

Here, it is assumed that a timeline panel is arranged in the virtual space, and a panel showing changes over time in specimen test data during the period indicated on the timeline is arranged below it.

For example, as shown in (a) of FIG. 4, among the medical data arranged in the virtual space, a part including the left end of the timeline panel and a panel of laboratory test data are included as panels that fall within the field of view. It is assumed that a portion including the left end of is displayed on the screen 8 of the display 140.

In this case, for example, the change function 153 determines whether the operator O continues to gaze at one of the icons 9 and 10 placed on the left and right sides of the screen 8 for a certain period of time based on the line of sight of the operator O. This will be accepted as a change order. Then, the change function 153 changes the viewing range within the virtual space so that the panel of specimen test data moves left and right on the screen 8 in accordance with the received change instruction. At this time, the change function 153 keeps the timeline panel within the viewing range so that it does not fall off the screen 8.

For example, as shown in FIG. 4B, the change function 153 moves the panel of specimen test data to the left on the screen 8 when the operator O gazes at the icon 10 on the right side for a certain period of time. . Further, at this time, the change function 153 fixes the position of the timeline panel on the screen 8 so that it does not fall off the screen 8.

For example, as shown in FIG. 4(c), if the operator O continues to gaze at the icon 10 on the right side, the change function 153 further changes the panel of specimen test data on the screen 8. Move it to the left so that the part including the right end of the panel of specimen test data is displayed. At this time as well, the change function 153 fixes the position of the timeline panel on the screen 8 so that it does not fall off the screen 8.

FIG. 5 is a diagram showing another example of a display change of medical data performed by the change function 153 according to the first embodiment.

Here, as in the example shown in Figure 4, a timeline panel is placed in the virtual space, and a panel of laboratory test data whose size in the horizontal direction is larger than the timeline panel is placed below it. Suppose that

For example, as shown in FIG. 5(a), among the medical data arranged in the virtual space, a part of the panel including the left end of the timeline panel and a panel of laboratory test data are included as panels that fall within the field of view. It is assumed that a portion including the left end of is displayed on the screen 8 of the display 140.

In this case, for example, the change function 153 receives a change instruction based on the operator O's line of sight and operation. For example, when the change function 153 detects that the operator O has continued to gaze at one of the panels displayed on the screen 8 for a certain period of time, the change function 153 changes the display of the panel that continues to be gazed at. Specify as. Further, the change function 153 changes the viewing range in the virtual space in response to the operation of the operator O such that the panel specified as the target of display change moves left and right on the screen 8. At this time, the change function 153 keeps the panels that are not specified as display change targets within the viewing range so that they do not fall off the screen 8.

For example, as shown in FIG. 5(a), when the operator O continues to gaze at the panel of laboratory test data displayed on the screen 8 for a certain period of time, the change function 153 changes the laboratory test data panel. Identify a panel for display changes.

Thereafter, for example, as shown in FIG. 5(b), when the operator performs an operation to indicate left with the controller 11, the change function 153 specifies on the screen 8 that the display is to be changed. Move the specimen test data panel to the left. Further, at this time, the change function 153 fixes the position of the timeline panel on the screen 8 so that it does not fall off the screen 8.

For example, as shown in FIG. 5C, if the operator O continues to use the controller 11 to point to the left, the change function 153 changes the panel of specimen test data on the screen 8. By moving the panel further to the left, a portion including the right end of the sample test data panel is displayed. At this time as well, the change function 153 fixes the position of the timeline panel on the screen 8 so that it does not fall off the screen 8.

In addition, in the above example, an example of changing the display of medical data was explained in which the panel of medical data specified as the target of display change was moved left and right, but this is the method for changing the display of medical data. Not limited to. For example, the contents of the panel of medical data specified as the object of display change may be changed.

FIG. 6 is a diagram showing another example of a display change of medical data performed by the change function 153 according to the first embodiment.

For example, as shown in FIG. 6(a), among the clinical data placed in the virtual space, the panels that fall within the field of view are the timeline panel and the sample test data in the period indicated on the timeline. Assume that a panel showing changes over time is arranged. Here, the panel of laboratory test data shows contents related to three types of laboratory test data, A, B, and C, and three icons indicating the types of laboratory test data shown on the panel. Assume that 12 is displayed.

In this case, for example, the change function 153 receives an action in which the operator O flicks his/her line of sight in the vertical direction as a change instruction. Then, the change function 153 switches the contents shown in the sample test data panel on the screen 8 according to the received change instruction.

For example, as shown in FIG. 6(b), the change function 153 changes the contents of the panel of specimen test data from the contents indicating all three types when the operator O flicks his/her line of sight upward once. Switch to the content showing only type A. For example, as shown in FIG. 6(c), when the operator O flicks his/her line of sight upward once more, the change function 153 changes the contents of the panel of specimen test data to only type A. Switch from the content shown to content showing only type B.

Similarly, for example, as shown in FIG. 6(d), the change function 153 changes the contents of the panel of specimen test data to only type B when the operator O flicks his/her line of sight upward once more. Switch from the content showing type C to the content showing only type C. For example, as shown in FIG. 6(a), when the operator O flicks his/her line of sight upward once more, the change function 153 changes the contents of the specimen test data panel to type C only. Switch from the content shown to the content showing all three types.

Note that, in this example, the change instruction that the change function 153 receives from the operator is not limited to one based on a flick of the line of sight. For example, the change function 153 switches the contents shown on the specimen test data panel in response to the operator O flicking his or her head up and down, blinking strongly, winking, etc. It's okay.

Alternatively, for example, the change function 153 may cause the operator O to gaze at any one of the three icons 12 indicating the type of specimen test data displayed on the panel for a certain period of time based on the line of sight of the operator O. When it is detected that the icon continues to be looked at, sample test data of the type indicated by the icon that has been looked at continuously may be displayed on the panel.

As another example of changing the display of medical data, for example, the change function 153 can change the display of medical data when the medical data to be changed is displayed in chronological order such as a timeline. The time scale may be expanded or contracted.

FIG. 7 is a diagram illustrating another example of a display change of medical data performed by the change function 153 according to the first embodiment.

Here, as an example, a case will be described in which a timeline panel is the target of display change. For example, as shown in FIG. 7(a), assume that a timeline panel indicating a period from April 2016 to August 2016 is displayed on the screen of the display 140. Further, here, it is assumed that an HMD is used as the display 140.

In this case, for example, the change function 153 accepts a movement of the operator O in the front-back direction as a change instruction. Then, the change function 153 changes the time scale of the timeline panel according to the received change instruction.

For example, as shown in FIG. 7(b), the change function 153 changes the time scale of the timeline panel from the period of April 2016 to August 2016 when operator O moves forward. The period will be changed from April 1, 2016 to April 30, 2016. That is, the change function 153 reduces the time scale of the timeline panel when the operator O moves forward.

On the other hand, when operator O moves backward, the change function 153 changes the time scale of the timeline panel from April 1, 2016 to April 30, 2016 to April 2016. The period will be changed to ~August 2016. That is, the change function 153 expands the time scale of the timeline panel when the operator O moves backward.

Each processing function that the processing circuit 150 has has been described above. Here, for example, the processing circuit 150 is realized by a processor. In this case, the processing functions of the processing circuit 150 are stored in the storage circuit 120 in the form of a computer-executable program. The processing circuit 150 reads each program from the storage circuit 120 and executes it, thereby realizing the function corresponding to each program. In other words, the processing circuit 150 in a state where each program is read has each function shown in the processing circuit 150 of FIG.

FIG. 8 is a flowchart showing a processing procedure of processing realized by each processing function of the processing circuit 150 according to the first embodiment.

For example, as shown in FIG. 8, when the acquisition function 151 receives an instruction to display medical data from the operator (step S101, Yes), various (Step S102). This processing is realized, for example, by the processing circuit 150 reading out a predetermined program corresponding to the acquisition function 151 from the storage circuit 120 and executing it.

Subsequently, the display control function 152 arranges each piece of medical care data acquired by the acquisition function 151 in a virtual space (step S103), and displays a plurality of pieces of medical care data that fall within the field of view in the virtual space on the screen of the display 140. (Step S104). This processing is realized, for example, by the processing circuit 150 reading out a predetermined program corresponding to the display control function 152 from the storage circuit 120 and executing it.

After that, when the change function 153 receives a change instruction based on the operator's movement, line of sight, or operation (step S105, Yes), a plurality of medical data displayed on the screen of the display 140 are displayed in accordance with the received change instruction. By changing the viewing range in the virtual space so that at least one of the medical data remains on the screen while changing the display of other medical data on the screen, the medical treatment on the screen can be changed. The data display is changed (step S106). This processing is realized, for example, by the processing circuit 150 reading out a predetermined program corresponding to the change function 153 from the storage circuit 120 and executing it.

Although the explanation here assumes that each processing function is realized by a single processor, it is also possible to configure a processing circuit by combining multiple independent processors, and realize the functions by having each processor execute a program. It doesn't matter if it's a thing. Further, each processing function of the processing circuit 150 may be realized by being appropriately distributed or integrated into a single or multiple processing circuits. In addition, in the example shown in FIG. 1, the single memory circuit 120 has been described as storing programs corresponding to each processing function. A configuration may also be used in which a corresponding program is read out from the circuit.

As described above, in the first embodiment, the display control function 152 arranges various types of medical data in a virtual space having a plurality of dimensions, and displays a plurality of pieces of medical data that fall within the visual field on the virtual space on the display 140. displayed on the screen. Then, the change function 153 displays at least one of the plurality of medical data displayed on the screen of the display 140 so that it does not fall off the screen, while changing the display of other medical data on the screen. The display of medical data on the screen is changed by changing the field of view within the virtual space. According to such a configuration, while the medical data that the operator is paying attention to is displayed on the screen, various other types of medical data can be switched and displayed.

Further, in the first embodiment, the change function 153 receives a change instruction based on the operator's movement, line of sight, or operation, and changes the display of medical data on the screen according to the received change instruction. According to such a configuration, the operator can display medical data more intuitively.

For this reason, according to the present embodiment, the operator can easily conduct multifaceted studies using various medical data.

Note that in the first embodiment described above, an example has been described in which the timeline panel is prevented from coming off the screen, but the embodiment is not limited to this. For example, the change function 153 may prevent panels of other medical data such as image data from coming off the screen, and may change other panels in response to change instructions from the operator.

Further, in the first embodiment described above, an example of changing the display of medical data is described in which the panel of medical data is moved left and right, but the embodiment is not limited to this. For example, if a plurality of medical data panels are arranged vertically in the virtual space, the changing function 153 may move the medical data panel vertically.

Further, in the first embodiment described above, when the display control function 152 arranges a plurality of pieces of medical data in the virtual space, for example, the display control function 152 arranges the most important medical data at the center in the horizontal direction or the vertical direction. You can also do this. Alternatively, the display control function 152 may arrange the most important medical data below the attention range set on the timeline. Alternatively, the display control function 152 may display medical data indicating a summary of each piece of medical data on top of a plurality of pieces of medical data.

Further, in the first embodiment described above, an example was explained in which the medical data is moved in the direction instructed by the operator, but the embodiment is not limited to this. For example, the change function 153 may automatically display a plurality of pieces of medical data to be changed in a direction in which the pieces of medical data are lined up.

Furthermore, in the first embodiment described above, when the display control function 152 displays the medical data on the display 140, the medical data may be displayed on top of the actual video. In that case, for example, the display control function 152 detects the position of the target region or the subject using a camera, sensor, or the like, and displays medical data near the detected image of the target region or the subject. This allows the operator to examine the diagnosis and treatment policy based on the medical data while observing the target region or subject.

Further, in the first embodiment described above, an example has been described in which the content of the panel of specimen test data is changed according to the motion of flicking the line of sight, but the embodiment is not limited to this. For example, the changing function 153 may switch the image data panel in response to an action of flicking the user's line of sight, like turning a page. Further, in such a case, the change instruction received from the operator is not limited to a flick of the line of sight, but may be, for example, an operation by the operator of scrolling with a mouse or the like.

Furthermore, in the first embodiment described above, an example was explained in which the time scale of medical data expressed in time series is expanded or decreased in accordance with the movement of the operator O in the forward and backward direction. The format is not limited to this. For example, the change function 153 may switch and display image data captured in chronological order and report data created in chronological order in accordance with the movement of the operator O in the forward and backward directions. good.

(Second embodiment)
Furthermore, in the first embodiment described above, an example was explained in which medical data is arranged in a virtual space having two dimensions, up and down and left and right, or in a virtual space having three dimensions, up and down, left and right, and front and back. However, the embodiment is not limited to this.

For example, classifications of medical data may be assigned to dimensions of the virtual space, and various types of medical data may be arranged in the virtual space according to the respective classifications. An example of such a case will be described below as a second embodiment. Note that the second embodiment will be mainly described with respect to points that are different from the first embodiment described above, and detailed descriptions of points that are common to the first embodiment will be omitted.

FIG. 9 is a diagram showing an overview of the display of medical data performed by the medical information processing apparatus 100 according to the second embodiment.

For example, as shown in FIG. 2, in the present embodiment, the display control function 152 arranges various types of medical data according to the respective classifications in a virtual space having a plurality of dimensions indicating the classifications of the medical data. Here, the classification of the medical data is the width, time, and depth of the medical data. Note that in this embodiment, such a virtual space is referred to as a "medical data space."

In the present embodiment, the change function 153 performs spatial transformation on the medical data space based on the designated conversion reference point, thereby converting at least one medical data of the plurality of medical data onto the screen. The display of medical data on the screen is changed by changing the field of view so as to change the display of other medical data on the screen while displaying the medical data so as not to deviate from the above.

Hereinafter, the processing performed by the acquisition function 151, display control function 152, and change function 153 according to this embodiment will be described in detail. Furthermore, in this embodiment, an example in which the subject is a patient will be described.

First, the acquisition function 151 acquires various medical data from the electronic medical record system 300, the radiology department system 400, and the specimen testing system 500 via the network 200. For example, the acquisition function 151 acquires information related to medical treatment, such as patient information, order information, order execution information, image examination information, and various measurement information, as medical data.

The acquisition function 151 then stores the medical care data acquired from each system in an integrated medical care DB (Data Base) constructed by the storage circuit 120. Here, the acquisition function 151 may store the medical care data acquired from each system in the integrated medical care DB in the same format, or may store the medical data in an appropriately normalized format.

For example, the integrated medical care DB stores medication data and medication summary data as data based on order information or order implementation information. Medication data is clinical data regarding medication administered to a patient. Medication summary data is medical care data related to a medication summary that summarizes medications administered to a patient at multiple points in time. In addition to medication data and medication summary data, medical data based on order information or order implementation information includes, for example, medical data related to various orders such as images, meals, rehabilitation, surgery, endovascular treatment, radiotherapy, etc. Medical data regarding the implementation of the various orders may be used.

Further, for example, the integrated medical care DB stores image data, image measurement value data, and image interpretation report data as medical data based on image examination information. The image data is medical care data regarding images taken of a patient. Image measurement value data is medical care data regarding measurement values measured based on images of a patient. The interpretation report data is medical data related to an interpretation report created based on an image taken of a patient. Note that the image referred to here refers to the medical image diagnostic device 600 (for example, various medical image diagnostic devices such as an X-ray diagnostic device, an X-ray CT device, an MRI device, an ultrasound diagnostic device, a SPECT device, a PET device, and a SPECT device). This is a medical image captured by a SPECT-CT device that is integrated with an X-ray CT device, a PET-CT device that is an integrated PET device and an X-ray CT device, or a group of these devices, etc.).

For example, the integrated medical care DB may include electrocardiogram data, electrocardiogram measurement value data, electrocardiogram data, vital data, vital summary data, nursing record data, laboratory test data, laboratory test summary data, and medical data based on various measurement information. Stores laboratory test report data. The electrocardiogram data is clinical data related to a patient's electrocardiogram measured in a test. The electrocardiogram measurement value data is medical care data regarding measurement values measured based on the patient's electrocardiogram. The electrocardiogram report data is clinical data related to an electrocardiogram report created based on a patient's electrocardiogram. Vital data is medical care data related to patient's vitals (eg, blood pressure, pulse, body temperature, etc.) measured in an examination. Vital summary data is medical care data related to a vital summary that summarizes vitals measured from a patient at multiple points in time. Nursing record data is medical data related to nursing records created based on patient's vitals. The specimen test data is medical data related to patient specimen tests (for example, tests for uric acid (UA), total protein (TP), etc.). The laboratory test summary data is clinical data related to a laboratory test summary that is a summary of laboratory tests performed on a patient at multiple points in time. The specimen test report data is medical data related to a specimen test report created based on a patient's specimen test.

Here, in this embodiment, for each of the above-mentioned medical data, a panel is defined that shows the medical data in a data display format suitable for each viewing. Here, the data display format is, for example, a table format, a time series format, an image display format, an electrocardiogram display format, a report display format, or the like. In this embodiment, the panels defined for each medical data are classified from various viewpoints depending on the purpose of clinical examination.

FIG. 10 is a diagram illustrating an example of classification of panels of medical data according to the second embodiment.

For example, as shown in the vertical direction in FIG. 10, the medical data panels are classified into panels related to intervention and panels related to response. In addition, panels related to interventions are further classified into panels related to medications, and panels related to medications are further classified by type of drug. Furthermore, the response panel is further classified into an image panel, an electrocardiogram panel, a vitals panel, and a sample test panel.

From another perspective, for example, as shown in the left-right direction in FIG. 10, the medical data panels are classified into panels related to objective data and panels related to subjective data. Furthermore, panels related to objective data are further classified into panels related to raw data and panels related to processed data, and panels related to subjective data are further classified into panels related to character data.

Furthermore, from another perspective, for example, as shown in the diagonal direction upward to the right in FIG. 10, the medical data panels are divided and classified by time-series time point.

Here, for example, the medication data panel described above is classified as a panel regarding medication and raw data, as shown in the first row from the top and the first column from the left in FIG. Separated and classified. Furthermore, as shown in the first row from the top and second column from the left in FIG. being classified.

In addition, the image data panels described above are classified as panels for images and raw data, as shown in the second row from the top and first column from the left in FIG. 10, and are further divided by time series points. being classified. Furthermore, as shown in the second row from the top and second column from the left in FIG. being classified. Furthermore, as shown in the second row from the top and the third column from the left in FIG. be done.

In addition, the electrocardiogram data panel described above is classified as a panel for electrocardiogram and raw data, as shown in the third row from the top and first column from the left in FIG. 10, and is further divided by time series time point. being classified. Furthermore, as shown in the third row from the top and second column from the left in FIG. 10, the panel for electrocardiogram measurement data described above is classified as a panel for electrocardiogram and processed data, and is divided by time-series time point. being classified. In addition, the electrocardiogram report data panel described above is classified as a panel for electrocardiogram and text data, as shown in the third row from the top and third column from the left in FIG. be done.

Furthermore, the vital data panel described above is classified as a panel related to vital data and raw data, as shown in the fourth row from the top and the first column from the left in Figure 10, and is further divided by time series time point. being classified. In addition, the panel for vital summary data mentioned above is classified as a panel for vital and processed data, as shown in the fourth row from the top and second column from the left in Figure 10, and summarizes the measured values of vitals at multiple points in time. It is classified as a panel. In addition, the nursing record data panel described above is classified as a panel for vital and character data, as shown in the fourth row from the top and third column from the left in Figure 10, and is divided and classified according to time-series points. be done.

In addition, the above-mentioned panel of specimen test data is classified as a panel related to specimen tests and raw data, as shown in the fifth row from the top and the first column from the left in Figure 10, and is further divided by time-series time point. and classified. In addition, the panel for the sample test summary data described above is classified as a panel related to sample test and processed data, as shown in the fifth row from the top and second column from the left in FIG. It is classified as a panel that summarizes values. In addition, the above-mentioned panel for laboratory test report data is classified as a panel for laboratory test and character data, as shown in the fifth row from the top and third column from the left in Figure 10, and is divided by time-series time point. It is classified as follows.

Next, the display control function 152 generates a medical data space having three dimensions, a dimension indicating the width of the medical data, a dimension indicating the time of the medical data, and a dimension indicating the depth of the medical data, according to the above-mentioned classification. Place each medical data panel on the screen.

Specifically, the display control function 152 arranges each medical data panel in the medical data space by assigning coordinates on the medical data space to each medical data panel.

FIG. 11 is a diagram showing an example of coordinates assigned to each medical data panel by the display control function 152 according to the second embodiment.

For example, as shown in FIG. 11, the display control function 152 displays the coordinates (W (width), T (time)) on the medical data space in the panel of each medical data shown in FIG. , D (depth)).

For example, coordinates are assigned to each medical data panel in the width (W) direction such that coordinate values increase in the order of medication, image, electrocardiogram, vitals, and specimen test. Further, in the depth (D) direction, coordinates are assigned to each medical data panel so that the coordinate values increase in the order of raw data, processed data, and character data. Furthermore, in the time (T) direction, coordinates are associated with each medical data panel so that the coordinate values increase in chronological order.

In this way, after arranging each medical data panel stored in the integrated medical care DB in the medical data space, the display control function 152 displays a plurality of medical data panels within the field of view on the medical data space on the screen of the display 140. indicate. Specifically, the display control function 152 generates a display screen for displaying each panel of medical care data based on the medical care data space, and displays the generated display screen on the display 140.

Next, the change function 153 displays at least one of the plurality of medical data displayed on the screen of the display 140 so that it does not fall off the screen, while changing the display of other medical data on the screen. The display of medical data on the screen is changed by changing the field of view within the medical data space.

Specifically, the change function 153 performs spatial transformation on the medical data space based on a specified conversion reference point, thereby changing the visual field range so as to change the display of other medical data on the screen. change. Here, the change function 153 receives a change instruction based on the operator's movement, line of sight, or operation, and changes the display of medical data on the screen according to the received change instruction.

First, the change function 153 obtains the display state of each medical data panel displayed on the screen of the display 140 by the display control function 152 before spatial conversion. Specifically, the change function 153 changes the display state before spatial transformation (which coordinates are displayed, which dimensions (points, lines, etc.) based on the information displayed on the screen and instructions received from the operator. , surface, etc.). Here, the display state before spatial transformation is indicated by a point, line, or plane on the medical data space. Note that, hereinafter, the display state before spatial transformation will be referred to as a "pre-conversion state."

Thereafter, the modification function 153 obtains a reference point for spatial transformation. Specifically, the change function 153 acquires a reference point for spatial transformation based on an instruction received from an operator. Here, the reference point for spatial transformation is indicated by a point, line, or plane on the medical data space. Note that hereinafter, the reference point for spatial transformation will be referred to as a "transformation reference point."

Furthermore, the change function 153 obtains the direction of spatial transformation. Specifically, the change function 153 obtains the direction of spatial transformation based on an instruction received from the operator. Note that hereinafter, the direction of spatial transformation will be referred to as a "transformation direction."

The change function 153 then performs spatial transformation. Specifically, the change function 153 changes the display state after spatial transformation (which coordinates should be displayed) based on the acquired pre-conversion state, transformation reference point and transformation direction, and the coordinates assigned to each clinical data. and which dimension (point, line, surface, etc.) to display. Here, the display state after spatial transformation is indicated by a point, line, or plane on the medical data space. Note that hereinafter, the display state after spatial transformation will be referred to as a "post-conversion state."

Thereafter, the change function 153 displays the display screen after the spatial transformation. Specifically, the change function 153, based on the post-conversion state obtained by the spatial transformation by the change function 153, the panel defined for each medical data, and each medical care data stored in the integrated medical care DB, A display screen after spatial transformation is generated, and the generated display screen is displayed on the display 140.

In this way, the change function 153 transforms the pre-conversion state indicated by a point, line, or plane on the medical data space into another state based on the conversion reference point specified in the pre-conversion state as a spatial transformation. Transform to a transformed state represented by a point, line, or plane. Here, the conversion reference point is also indicated by a point, line, or plane on the medical data space.

Then, when performing the above-described spatial transformation, the change function 153 fixes the positions on the screen of the panels other than the panel designated as the conversion reference point so that they do not fall off the screen.

Hereinafter, the spatial transformation and display change of medical data performed by the change function 153 according to the second embodiment will be described using a specific example. An example in which the change function 153 performs expansion, reduction, rotation, or translation in the medical data space as spatial transformation will be described below.

FIG. 12 is a diagram illustrating an example of enlargement conversion performed by the change function 153 according to the second embodiment.

Here, an example of converting a point into a line display by enlarging it will be described. Further, here, it is assumed that an HMD is used as the display 140.

For example, as shown on the left side of FIG. 12, with multiple medical data panels displayed on the screen, the operator selects (W, T, D) = (W Suppose that a panel of medical data to which _A , T _A , D _A ) are assigned is specified, and expansion in the depth direction (D=*) is specified as the conversion direction.

For example, when the change function 153 detects that the operator O has continued to gaze at any one of the panels displayed on the screen for a certain period of time, the change function 153 changes the state of the continued gazed panel to the pre-conversion state and the conversion standard. It is determined that it is specified as a point. Further, for example, when the change function 153 detects a movement of the operator O to move backward, the change function 153 determines that enlargement in the depth direction is specified as the conversion direction. Here, the change function 153 specifies the conversion amount of the enlargement conversion according to the movement amount of the operator O.

In this case, for example, as shown on the right side of FIG. 12, the change function 153 changes the coordinates (W _A , T _A , D A ) indicating the pre-conversion state to the coordinates (W _A , T _A ) indicating the post-conversion state. _A , *). This spatial transformation transforms the pre-conversion state indicated by a point into the post-conversion state indicated by a line extending in the depth direction by expanding the state in the depth direction based on the conversion reference point indicated by the point. It corresponds to that.

As a result, for example, as shown in FIG. 12, before the conversion, image data was displayed for one time point, but after conversion, image data, image measurement values, and image interpretation are displayed for the same time point. The report can be switched to the displayed state. When performing this spatial transformation, the change function 153 fixes the positions on the screen of the panels other than the panel designated as the conversion reference point so that they do not fall off the screen.

FIG. 13 is a diagram showing another example of enlargement conversion performed by the change function 153 according to the second embodiment.

Here, an example of converting a line into a surface display by enlarging it will be described. Further, here, it is assumed that an HMD is used as the display 140.

For example, as shown on the left side of FIG. 13, with multiple medical data panels displayed on the screen, the operator selects (W, T, D) = (W Suppose that a panel of clinical data to which _B , *, D _B ) is assigned is specified, and expansion in the width direction (W=*) is specified as the conversion direction.

For example, when the change function 153 detects that the operator O has continued to gaze at any one of the panels displayed on the screen for a certain period of time, the change function 153 changes the state of the continued gazed panel to the pre-conversion state and the conversion standard. It is determined that it is specified as a point. Further, for example, when the change function 153 detects a movement of the operator O to move backward, the change function 153 determines that enlargement in the width direction is specified as the conversion direction. Here, the change function 153 specifies the conversion amount of the enlargement conversion according to the movement amount of the operator O.

In this case, for example, as shown on the right side of FIG. 13, the change function 153 changes the coordinates (W _B , *, D _B ) indicating the pre-conversion state to the coordinates (*, *, D B ) indicating the post-conversion state. _B ) Convert to This spatial transformation is performed by expanding the pre-conversion state indicated by a line extending in the time direction in the width direction with reference to the conversion reference point indicated by the line, resulting in the transformation indicated by the surface extending in the width direction and time direction. This corresponds to converting to the post-state.

As a result, for example, as shown in FIG. 13, before the conversion, the sample test summary data was displayed for multiple time points, but after the conversion, the medication summary data and sample test summary data for the multiple time points are displayed. You can switch between displaying summary data and vital summary data. When performing this spatial transformation, the change function 153 fixes the positions on the screen of the panels other than the panel designated as the conversion reference point so that they do not fall off the screen.

FIG. 14 is a diagram illustrating an example of reduction conversion performed by the change function 153 according to the second embodiment.

Here, an example will be described in which a line is converted into a point display by reduction. Further, here, it is assumed that an HMD is used as the display 140.

For example, as shown on the left side of FIG. 14, with multiple medical data panels displayed on the screen, the operator selects (W, T, D) = (W _C , *, A panel of clinical data to which D _C ) is assigned is specified, a position where (W, T, D) = (W _C , T _C , D _C ) is specified as the conversion reference point, and time is specified as the conversion direction. Assume that reduction in the direction (T=*) is specified.

For example, when the change function 153 detects that the operator O has continued to gaze at one of the panels displayed on the screen for a certain period of time, the change function 153 designates the panel that has been gazed at as the pre-conversion state. It is further determined that the position on the panel where the gaze continues is designated as the conversion reference point. Further, for example, when the change function 153 detects a movement of the operator O to move forward, the change function 153 determines that reduction in the time direction is specified as the conversion direction. Here, the change function 153 specifies the amount of reduction conversion according to the amount of movement of the operator O.

In this case, for example, as shown on the right side of FIG. 14, the change function 153 changes the coordinates (W _C , *, D _C ) indicating the pre-conversion state to the coordinates (W _C , T _{C )} indicating the post-conversion state. , D _C ). This spatial transformation is performed by reducing the pre-conversion state indicated by a line extending in the time direction in the time direction with reference to the conversion reference point indicated by one point included in the line, so that the state after the transformation indicated by the dot This corresponds to converting into a state.

As a result, for example, as shown in FIG. 14, before the conversion, the sample test summary data was displayed for multiple time points, but after the conversion, the sample test summary data for one of the multiple time points is displayed. You can switch to a state where the measurement values of the test are displayed. When performing this spatial transformation, the change function 153 fixes the positions on the screen of the panels other than the panel designated as the conversion reference point so that they do not fall off the screen.

FIG. 15 is a diagram showing another example of reduction conversion performed by the change function 153 according to the second embodiment.

Here, an example will be described in which a surface is converted into a line display by reduction. Further, here, it is assumed that an HMD is used as the display 140.

For example, as shown on the left side of FIG. 15, with multiple medical data panels displayed on the screen, the operator selects (W, T, D) = (*, *, D) as the pre-conversion state. _A panel _of clinical data to which _D =*) is specified, and further, it is assumed that reduction in that direction is specified.

For example, when the change function 153 detects that the operator O has continued to gaze at one of the panels displayed on the screen for a certain period of time, the change function 153 designates the panel that has been gazed at as the pre-conversion state. It is further determined that the position on the panel where the gaze continues is designated as the conversion reference point. Further, for example, when the change function 153 detects a movement of the operator O to move forward, the change function 153 determines that reduction in the time direction is specified as the conversion direction. Here, the change function 153 specifies the amount of reduction conversion according to the amount of movement of the operator O.

In this case, for example, as shown on the right side of FIG. 15, the change function 153 changes the coordinates (*, *, D _D ) indicating the pre-conversion state to the coordinates (*, T _D , D _D ). This spatial transformation is performed by reducing the pre-conversion state represented by a plane extending in the width direction and time direction in the time direction based on a conversion reference point represented by a line extending in the width direction included in the plane. , corresponds to converting to the post-conversion state shown by the line extending in the width direction.

As a result, for example, as shown in FIG. 15, before the conversion, medication summary data, laboratory test summary data, and vitals summary data were displayed for multiple time points; For one time point included in time point , the state is switched to a state in which information indicating the drug, the measured value of the sample test, and the measured value of vital signs are displayed. When performing this spatial transformation, the change function 153 fixes the positions on the screen of the panels other than the panel designated as the conversion reference point so that they do not fall off the screen.

FIG. 16 is a diagram illustrating an example of rotation conversion performed by the change function 153 according to the second embodiment.

Here, an example of rotationally transforming a line will be described. Further, here, it is assumed that an HMD is used as the display 140.

For example, as shown on the left side of FIG. 16, with multiple medical data panels displayed on the screen, the operator selects (W, T, D) = (W _E1 , *, A panel of clinical data to which D _E1 ) is assigned is specified, a position where (W, T, D) = (W _E1 , T _E1 , D _E1 ) is specified as the conversion reference point, and depth is specified as the conversion direction. Assume that rotation in the direction (D=*) is specified.

For example, when the change function 153 detects that the operator O has continued to gaze at one of the panels displayed on the screen for a certain period of time, the change function 153 designates the panel that has been gazed at as the pre-conversion state. It is further determined that the position on the panel where the gaze continues is designated as the conversion reference point. Further, for example, when the operator O receives an operation to rotate the object 13 in the medical data space displayed on the screen of the display 140, the change function 153 specifies rotation in the depth direction as the conversion direction. It is determined that the Alternatively, for example, when the change function 153 detects that the medical data that the operator O wants to view in the medical data space 14 has moved to a certain angle, the change function 153 determines that rotation in the depth direction is specified as the conversion direction. do. Here, the change function 153 specifies the amount of rotation conversion according to the amount of operation or movement of the operator O.

In this case, for example, as shown on the right side of FIG. 16, the change function 153 changes the coordinates (W _E1 , *, D _E1 ) indicating the pre-conversion state to the coordinates (W _E1 , T _{E1 )} indicating the post-conversion state. , *). This spatial transformation extends in the depth direction by rotating the pre-conversion state indicated by a line extending in the time direction in the depth direction with reference to the transformation reference point indicated by one point included in the line. This corresponds to converting to the post-conversion state shown by the line.

As a result, for example, as shown in FIG. 16, before the conversion, the image data was displayed at multiple time points, but after the conversion, the image data is displayed at one time point included in the multiple time points. , the image measurement value and the interpretation report are displayed. When performing this spatial transformation, the change function 153 fixes the positions on the screen of the panels other than the panel designated as the conversion reference point so that they do not fall off the screen.

FIG. 17 is a diagram showing another example of rotation conversion performed by the change function 153 according to the second embodiment.

Here, an example of rotationally transforming a surface will be explained. Further, here, it is assumed that an HMD is used as the display 140.

For example, as shown on the left side of FIG. 17, with multiple medical data panels displayed on the screen, the operator selects (W, T, D) = (*, *, D) as the pre-conversion state. A panel of clinical data to which _F1 ) is assigned is specified, a position where (W, T, D) = (W _F1 , *, D _F1 ) is specified as a conversion reference point, and a depth direction ( Suppose that rotation to D=*) is specified.

For example, when the change function 153 detects that the operator O has continued to gaze at one of the panels displayed on the screen for a certain period of time, the change function 153 designates the panel that has been gazed at as the pre-conversion state. It is further determined that the position on the panel where the gaze continues is designated as the conversion reference point. Further, for example, when the operator O receives an operation to rotate the object 13 in the medical data space displayed on the screen of the display 140, the change function 153 specifies rotation in the depth direction as the conversion direction. It is determined that the Alternatively, for example, when the change function 153 detects that the medical data that the operator O wants to view in the medical data space 14 has moved to a certain angle, the change function 153 determines that rotation in the depth direction is specified as the conversion direction. do. Here, the change function 153 specifies the amount of rotation conversion according to the amount of operation or movement of the operator O.

In this case, for example, as shown on the right side of FIG. 17, the change function 153 changes the coordinates (*, *, D _F1 ) indicating the pre-conversion state to the coordinates (W _F1 , *, * ). This spatial transformation involves rotating the pre-conversion state represented by a surface extending in the width direction and time direction in the depth direction with reference to a transformation reference point represented by a line included in the surface and extending in the time direction. This corresponds to converting to the post-conversion state represented by a plane extending in the time direction and the depth direction.

As a result, for example, as shown in FIG. 17, before the conversion, the image measurement values, laboratory test summary data, and vitals summary data were displayed for multiple time points, but after the conversion, the relevant The state is switched to display the measured values of the image, the image data, and the interpretation report for a plurality of time points, respectively. When performing this spatial transformation, the change function 153 fixes the positions on the screen of the panels other than the panel designated as the conversion reference point so that they do not fall off the screen.

FIG. 18 is a diagram illustrating an example of parallel translation transformation performed by the change function 153 according to the second embodiment.

Here, an example of translating a line will be described. Further, here, it is assumed that an HMD is used as the display 140.

For example, as shown on the left side of FIG. 18, with multiple medical data panels displayed on the screen, the operator selects (W, T, D) = (* , T _G , D _G ) is specified, and the forward direction (-T) in the time direction is specified as the conversion direction.

For example, when the change function 153 detects that the operator O has continued to gaze at any one of the panels displayed on the screen for a certain period of time, the change function 153 changes the state of the continued gazed panel to the pre-conversion state and the conversion standard. It is determined that it is specified as a point. Further, for example, when the change function 153 detects that the operator O has moved to the position of the medical data that he or she wants to view in the medical data space 14, the change function 153 determines that the forward direction in the time direction has been specified as the conversion direction. Here, the change function 153 specifies the amount of translation transformation according to the amount of movement of the operator O.

In this case, for example, as shown on the right side of FIG. 18, the change function 153 changes the coordinates (*, T _G , D _G ) indicating the pre-conversion state to the coordinates (*, T _G −) indicating the post-conversion state. T, _DG ). This spatial transformation is performed by translating the pre-conversion state indicated by a line extending in the width direction in parallel in the time direction with reference to the conversion reference point indicated by the line, resulting in the transformation indicated by another line extending in the width direction. This corresponds to converting to the post-state.

As a result, for example, as shown in FIG. 18, before the conversion, the information indicating the drug, the measured value of the sample test, and the measured value of the vital signs were displayed for one point in time; , for another point in time, the display switches to a state in which information indicating the drug, the measured value of the sample test, and the measured value of vital signs are displayed. When performing this spatial transformation, the change function 153 fixes the positions on the screen of the panels other than the panel designated as the conversion reference point so that they do not fall off the screen.

FIG. 19 is a diagram showing another example of parallel translation transformation performed by the change function 153 according to the second embodiment.

Here, an example of translating a surface will be described. Further, here, it is assumed that an HMD is used as the display 140.

For example, as shown on the left side of FIG. 19, the operator specifies a panel of medical data to which (W, T, D) = (*, *, D _H ) is assigned as the pre-conversion state and conversion reference point. It is assumed that the backward direction (+D) in the depth direction is specified as the conversion direction.

For example, when the change function 153 detects that the operator O has continued to gaze at any one of the panels displayed on the screen for a certain period of time, the change function 153 changes the state of the continued gazed panel to the pre-conversion state and the conversion standard. It is determined that it is specified as a point. Further, for example, when the change function 153 detects that the operator O has moved to the position of the medical data that he or she wants to view in the medical data space 14, the change function 153 determines that the backward direction in the depth direction is specified as the conversion direction. . Here, the change function 153 specifies the amount of translation transformation according to the amount of movement of the operator O.

In this case, for example, as shown on the right side of FIG. 19, the change function 153 changes the coordinates (*, *, D _H ) indicating the pre-conversion state to the coordinates (*, *, D _H ) indicating the post-conversion state. +D). This spatial transformation is performed by translating the pre-conversion state represented by a surface extending in the width direction and time direction in the depth direction with reference to the conversion reference point represented by the surface. This corresponds to converting to the post-conversion state shown in other aspects.

As a result, for example, as shown in FIG. 19, before the conversion, the image measurement values and the electrocardiogram measurement values were displayed for multiple time points, but after the conversion, the image measurement values and the electrocardiogram measurement values were displayed for the multiple time points. The image data and the electrocardiogram image data are displayed. When performing this spatial transformation, the change function 153 fixes the positions on the screen of the panels other than the panel designated as the conversion reference point so that they do not fall off the screen.

Note that the spatial transformation performed by the change function 153 is not limited to the two-dimensional spatial transformation as described above. For example, the modification function 153 may perform spatial transformations of three or more dimensions, such as expansion from a plane to a space, rotation or translation of a space, reduction from a space to a plane, and the like.

As described above, in the second embodiment, the display control function 152 arranges various types of medical data according to the respective classifications in a virtual space having a plurality of dimensions indicating the classifications of the medical data. With such a configuration, for example, it becomes possible to switch the display of medical data for each classification such as width, time, depth, etc. of the medical data, and the operator can display multifaceted medical data using various types of medical data. You will be able to conduct reviews efficiently.

In addition, in the second embodiment described above, an example was explained in which the width, time, and depth of medical data are used as dimensions in the medical data space, but the classification of medical data assigned to the medical data space is based on these dimensions. Other classifications may be used without limitation. For example, patients who are the subject of medical data may be used as the classification.

Furthermore, in the second embodiment described above, there is an operation in which the operator rotates an object in the medical data space displayed on the screen of the display 140, and medical data that the operator wants to see in the medical data space 14. Although an example has been described in which the display of medical data is changed in accordance with an operation of moving to an angle, the embodiment is not limited to this.

For example, the display control function 152 arranges medical data in a semi-transparent manner in the virtual space displayed on the screen of the display 140, and the change function 153 arranges the medical data displayed in a semi-transparent manner so that the operator can extend his/her hand. When the operator performs an action to take a hand, the medical data taken by the operator may be clearly displayed in accordance with the action.

Alternatively, for example, the display control function 152 may arrange a transparent board in the virtual space displayed on the screen of the display 140, and display the medical data on the board. In that case, for example, when the operator moves forward, the change function 153 causes the next medical data to be displayed on the transparent board accordingly. Alternatively, when the operator moves forward, the change function 153 zooms the medical data displayed on the transparent board accordingly, and when the operator moves beyond the transparent board, the next data may be displayed. Alternatively, the changing function 153 may change the display of medical data in response to an operation of pushing or pulling out a transparent plate.

Note that in the embodiment described above, an example has been described in which the display control unit, reception unit, and change unit in this specification are realized by the display control function 152 and change function 153 of the processing circuit 150, respectively. is not limited to this. For example, in addition to the display control function 152 and change function 153 described in the embodiment, the display control unit, reception unit, and change unit in this specification may be realized by only hardware or by a combination of hardware and software. It is also possible to realize the same function by

Furthermore, the term "processor" used in the above explanation refers to, for example, a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), an Application Specific Integrated Circuit (ASIC), or a programmable logic device. (For example, it refers to circuits such as Simple Programmable Logic Device (SPLD), Complex Programmable Logic Device (CPLD), and Field Programmable Gate Array (FPGA)). The processor implements functions by reading and executing programs stored in the storage circuit 120. Note that instead of storing the program in the storage circuit 120, the program may be directly incorporated into the circuit of the processor. In this case, the processor implements its functions by reading and executing a program built into the circuit. Further, the processor of this embodiment is not limited to being configured as a single circuit, but may be configured as a single processor by combining a plurality of independent circuits to realize its functions.

Here, the program executed by the processor is provided by being pre-installed in a ROM (Read Only Memory), a storage circuit, or the like. This program is a file in a format that can be installed or executable on these devices, such as CD (Compact Disk)-ROM, FD (Flexible Disk), CD-R (Recordable), DVD (Digital Versatile Disk), etc. It may be provided recorded on a computer readable storage medium. Further, this program may be stored on a computer connected to a network such as the Internet, and may be provided or distributed by being downloaded via the network. For example, this program is composed of modules including each of the above-mentioned functional units. In actual hardware, a CPU reads a program from a storage medium such as a ROM and executes it, so that each module is loaded onto the main storage device and generated on the main storage device.

According to at least one embodiment described above, the operator can easily conduct multifaceted studies using various medical data.

Although several embodiments of the invention have been described, these embodiments are presented by way of example and are not intended to limit the scope of the invention. These embodiments can be implemented in various other forms, and various omissions, substitutions, and changes can be made without departing from the gist of the invention. These embodiments and their modifications are included within the scope and gist of the invention as well as within the scope of the invention described in the claims and its equivalents.

100 Medical information processing device 150 Processing circuit 152 Display control function 153 Change function

Claims (9)

A timeline showing the occurrence timing of each medical event for multiple medical data in chronological order is arranged in a two-dimensional or three-dimensional virtual space, and corresponds to the attention range specified by the operator on the timeline. A plurality of pieces of medical data including at least one of graphs, image data, numerical data, and report data are placed side by side under the timeline , and a plurality of pieces of medical data that fall within the field of view in the virtual space are arranged side by side under the timeline. a display control unit that displays medical data on a display screen;
While fixing and displaying the timeline in the first display area on the screen so that the range of attention specified by the operator does not deviate from the screen, a plurality of medical data corresponding to the range of attention are displayed. The display of medical data on the screen is changed by changing the viewing range in the virtual space so that the data is displayed while being switched to a second display area below the first display area on the screen. A medical information processing device comprising: a changing section for changing; The change unit receives a change instruction based on an operator's movement, line of sight, or operation, and changes the display of the medical data on the screen according to the received change instruction.
The medical information processing device according to claim 1. The change unit is configured to provide, as the change instruction, an action in which the operator turns his or her head in a predetermined direction, an action in which the operator moves in a predetermined direction, a position where the medical data that the operator wishes to view in the virtual space is located, or receiving that the operator has moved to a certain angle, that the operator has continued to gaze at a predetermined position on the screen for a certain period of time, and that the operator has flicked his line of sight in a predetermined direction;
The medical information processing device according to claim 2. The virtual space has a dimension indicating a classification of the medical data,
The display control unit arranges the plurality of medical data in the virtual space according to their respective classifications.
The medical information processing device according to any one of claims 1 to 3. The dimension of the medical data indicates the width, time, or depth of the medical data;
The medical information processing device according to claim 4. The changing unit changes the visual field range so as to switch and display a plurality of medical data corresponding to the attention range by performing spatial transformation on the virtual space based on a designated transformation reference point. change,
The medical information processing device according to any one of claims 1 to 5. The changing unit performs enlargement, reduction, rotation, or translation in the virtual space as the spatial transformation.
The medical information processing device according to claim 6. The display control unit displays the plurality of medical data on a display realized by a display device for VR (Virtual Reality), MR (Mixed Reality), or AR (Augmented Reality).
The medical information processing device according to any one of claims 1 to 7. A timeline showing the occurrence timing of each medical event for multiple medical data in chronological order is arranged in a two-dimensional or three-dimensional virtual space, and corresponds to the attention range specified by the operator on the timeline. A plurality of pieces of medical data including at least one of graphs, image data, numerical data, and report data are placed side by side under the timeline , and a plurality of pieces of medical data that fall within the field of view in the virtual space are arranged side by side under the timeline. medical data on the display screen,
While fixing and displaying the timeline in the first display area on the screen so that the range of attention specified by the operator does not deviate from the screen, a plurality of medical data corresponding to the range of attention are displayed. The display of medical data on the screen is changed by changing the viewing range in the virtual space so that the data is displayed while being switched to a second display area below the first display area on the screen. Medical information processing methods, including changing them.