JP7579698B2 - Medical information management server, surgical training device, image transmission method and program - Google Patents

Description

This disclosure relates to a medical information management server that manages and distributes simulation data, a surgical training device, a surgical training system, an image transmission method, a display method, and a program.

A technology has been known for surgical training devices that reproduces the appearance of educational video accompanying the movement of an endoscope when displayed during practice (see, for example, Patent Document 1). With this technology, the movement history of the educational endoscope during the process of capturing educational video using the educational endoscope is reproduced in a training endoscope inserted into a training device that provides an internal surgical practice space, and the educational video and the video captured by the training endoscope are synthesized and displayed.

JP 2016-148765 A

In order to become a skilled surgeon, medical students and interns must attend many actual surgeries and cases and gain a wide variety of surgeries and experience. However, in the above-mentioned Patent Document 1, the movement history of one educational endoscope is merely reproduced by a training endoscope, so it is not possible to accommodate a wide variety of surgeries and cases, and even for the same case, medical students and interns cannot trace the techniques of a surgeon or professor or attempt a technique using a different approach.

The present disclosure has been made in consideration of the above, and aims to provide a medical information management server, a surgical training device, a surgical training system, an image transmission method, a display method, and the like, which allow a patient to gain experience equivalent to that of an actual surgery.

In order to solve the above-mentioned problems and achieve the objectives, the medical information management server according to the present disclosure includes a recording unit that records at least a plurality of surgical procedures and a plurality of simulation data in association with each other, and a control unit that, when one of the plurality of surgical procedures is selected on the client side, transmits the simulation data corresponding to the selected surgical procedure to the client side.

The surgical training device according to the present disclosure further comprises a display unit capable of displaying a simulation video based on simulation data transmitted from a medical information management server that records multiple simulation data associated with multiple surgical procedures, one or more medical instruments, a detection unit that detects the operation content of the operator on the medical instruments, and a control unit that causes the display unit to display a medical instrument image related to the medical instruments on the simulation video, and the control unit controls the display mode of the medical instrument image and the simulation video based on the operation content detected by the detection unit.

The surgical training system according to the present disclosure includes a medical information management server and a surgical training device capable of communicating with the medical information management server via a network. The medical information management server includes a recording unit that records at least a plurality of surgical procedures and a plurality of simulation data in association with each other, and a first control unit that transmits, when any one of the plurality of surgical procedures is selected on the client side, simulation data corresponding to the selected surgical procedure to the client side. The surgical training device includes a display unit that can display a simulation video based on the simulation data transmitted from the medical information management server, one or more medical instruments that can be used to treat a model that imitates a subject, a detection unit that detects the operation content of the medical instruments by an operator, and a second control unit that causes the display unit to display a medical instrument image related to the medical instrument on the simulation video. The second control unit controls the display mode of the medical instrument image and the simulation video based on the operation content detected by the detection unit.

The image transmission method according to the present disclosure is executed by a medical information management server that includes a recording unit that records at least a plurality of surgical procedures and a plurality of pieces of simulation data in association with each other, and when one of the plurality of surgical procedures is selected on the client side, the simulation data corresponding to the selected surgical procedure is transmitted to the client side.

The display method according to the present disclosure is executed by a surgical training device that includes a display unit capable of displaying a simulation video based on simulation data transmitted from a medical information management server that records multiple simulation data associated with multiple surgical procedures, and one or more medical instruments capable of treating a model that mimics a subject, and detects the operation content of the medical instrument by an operator, causes the display unit to display a medical instrument image related to the medical instrument on the simulation video, and controls the display mode of the medical instrument image and the simulation video based on the operation content.

The program disclosed herein is executed by a medical information management server that includes a recording unit that records at least a plurality of surgical procedures and a plurality of pieces of simulation data in association with each other, and when one of the plurality of surgical procedures is selected on the client side, the program causes the server to transmit the simulation data corresponding to the selected surgical procedure to the client side.

The program disclosed herein is a program executed by a surgical training device that includes a display unit capable of displaying a simulation video based on simulation data transmitted from a medical information management server that records multiple simulation data associated with multiple surgical procedures, and one or more medical instruments capable of treating a model that mimics a subject, and detects the operation content of the medical instrument by an operator, displays a medical instrument image related to the medical instrument on the display unit on the simulation video, and controls the display mode of the medical instrument image and the simulation video based on the operation content.

This disclosure has the effect of allowing students to gain experience equivalent to that of actual surgery.

FIG. 1 is a diagram showing a schematic configuration of a surgical training system according to an embodiment. FIG. 2 is a block diagram showing the functional configuration of the main parts of a surgical training device according to one embodiment. FIG. 3 is a block diagram showing the functional configuration of a medical information management server according to an embodiment. FIG. 4 is a diagram showing an example of a setting screen displayed when performing surgical training on a display device according to an embodiment. FIG. 5 is a diagram showing an example of a setting screen displayed when performing surgical training on a display device according to an embodiment. FIG. 6 is a diagram showing an example of a setting screen displayed when performing surgical training on the display device according to the embodiment. FIG. 7 is a diagram showing an example of a setting screen displayed when performing surgical training on the display device according to the embodiment. FIG. 8 is a diagram showing an example of a setting screen displayed when performing surgical training on the display device according to one embodiment. FIG. 9 is a diagram showing an example of a setting screen displayed when performing surgical training on a display device according to an embodiment. FIG. 10 is a diagram showing an example of a setting screen displayed when performing surgical training on a display device according to an embodiment. FIG. 11 is a diagram showing an example of a setting screen displayed when performing surgical training on a display device according to an embodiment. FIG. 12 is a flowchart showing an overview of the process executed by the medical information management server according to one embodiment. FIG. 13 is a flowchart showing an outline of the simulation data transmission process of FIG. FIG. 14 is a flowchart showing an outline of the simulation data generation process of FIG. FIG. 15 is a flowchart showing an overview of the processing executed by the surgery training device according to one embodiment.

The embodiments for implementing the present disclosure will be described in detail with reference to the drawings. Note that the present disclosure is not limited to the following embodiments. In the following description, the figures referred to merely show the shape, size, and positional relationship in a schematic manner to the extent that the contents of the present disclosure can be understood. In other words, the present disclosure is not limited to only the shape, size, and positional relationship exemplified in each figure. Furthermore, in the description of the drawings, the same parts are denoted by the same reference numerals. Furthermore, as an example of a surgical training system according to the present disclosure, a surgical training system using an endoscopic system using a laparoscope will be described, but it can also be applied to a surgical training system using a surgical microscope system.

[Overview of the surgical training system]
Fig. 1 is a diagram showing a schematic configuration of a surgical training system according to an embodiment. The surgical training system 1 shown in Fig. 1 is a system in which a plurality of holes are formed in the abdomen of a model _O1 simulating a subject such as a human or animal using a trocar, and the abdomen of the model _O1 is inflated with carbon dioxide or the like, and an operator U1 (trainee) performs training in laparoscopic surgery while observing and treating the model _O1 by inserting one or more medical instruments, such as a medical imaging device (a videoscope such as a rigid mirror), forceps, and an energy device such as an electric scalpel, into the model O1. In Fig. 1, two holes are formed in the model _O1 , but the number of holes can be appropriately changed depending on the training content, for example, the case to be trained, the surgical procedure, and the surgical technique.

The surgical training system 1 shown in FIG. 1 includes a trocar 10, a trocar 20, a medical instrument 30, a medical instrument 40, a display device 50, a control device 60, a medical information management server 70, and a number of hospital management servers 80. In one embodiment, the trocar 10, the trocar 20, the medical instrument 30, the medical instrument 40, the display device 50, and the control device 60 function as a surgical training device 2.

The trocars 10 and 20 are cylindrical and are inserted into the model O _1. The trocars 10 and 20 have a medical instrument 30 or a medical instrument 40, which will be described later, inserted therein.

The medical instrument 30 is inserted into the model _O1 via the trocar 10. The medical instrument 30 performs a simulated treatment on the subject of observation in response to the operation of an operator such as a doctor or a medical intern. The medical instrument 30 also detects the operation of the operator and outputs the detection result to the control device 60 via a transmission cable. The medical instrument 30 is, for example, an injection, forceps, a knife, a high-frequency snare, a treatment tool, or an energy device such as a high-frequency scalpel or an electric scalpel.

The medical instrument 40 is inserted into the model _O1 via the trocar 20. The medical instrument 40 performs a simulated treatment on the subject of observation in response to the operation of an operator such as a doctor or a medical intern. The medical instrument 40 also detects the operation of the operator and outputs the detection result to the control device 60 via a transmission cable. The medical instrument 40 is, for example, an injection, forceps, a knife, a high-frequency snare, a treatment tool, or an energy device such as a high-frequency scalpel or an electric scalpel.

Under the control of the control device 60, the display device 50 displays a two-dimensional image or a three-dimensional image based on image data input from the control device 60. The display device 50 is realized using, for example, a liquid crystal display, an organic electroluminescent display (EL display), an audio speaker, etc.

The control device 60 controls each part of the surgery training device 2. The control device 60 performs various image processing on the simulation data input from the medical information management server 70 via the network 90, and outputs the processed data, thereby displaying a simulation video based on the simulation data on the display device 50. The control device 60 is realized using a processor including a memory and hardware such as a central processing unit (CPU), a graphics processing unit (GPU), and a field-programmable gate array (FPGA). Here, the simulation data is data for performing a surgery simulation in a three-dimensional space using an organ, and includes data related to the organ or medical instrument. The simulation data for each organ and medical instrument includes, for example, the following parameters, and is defined by polygon data in a three-dimensional space.
(1) Position (position of organ (coordinate information))
(2) Shape (shape of organs, shape of each medical device)
Furthermore, the simulation data for each organ and medical instrument may include the following parameters (3) to (7).
(3) Mass density (4) Stiffness (5) Friction (6) Viscosity (7) Touch The simulation data may further include information on time series changes regarding the movement and deformation of the medical instrument or organ over time.

The medical information management server 70 transmits the simulation data requested by the control device 60 via the network 90. The medical information management server 70 also records two-dimensional image data or three-dimensional image data including at least data related to organs acquired for each of a plurality of patients by a CT (Computed Tomography) device or an MRI (Magnetic Resonance Imaging) device, etc. The medical information management server 70 also records simulation image data based on intraoperative video data captured by a medical imaging device during surgery on the subject, and at least one of two-dimensional image data and three-dimensional image data including at least data related to organs captured by a CT device or an MRI device. The medical information management server 70 is realized using a processor including hardware such as a CPU, GPU, and FPGA, and a hard disk drive (HDD), a solid state drive (SSD), etc.

The hospital management servers 80 transmit educational material data for the simulation data generated in the medical information management server 70 to the medical information management server 70 via the network 90. Here, the educational material data is a video file that corresponds to two-dimensional or three-dimensional image data of each of the multiple subjects (patients) acquired by a CT device or an MRI device, intraoperative video data including at least organs (e.g., stomach, intestines, etc.) imaged during surgery of each of the multiple subjects (patients) by a medical imaging device (e.g., a video microscope such as an endoscope or a rigid endoscope), subject information (patient information) regarding each of the multiple subjects, multiple surgeons (surgeons and hospital names) who operated on each of the multiple subjects, procedure information regarding the surgical procedure of each of the multiple subjects, and medical instrument information regarding each of the multiple medical instruments used in the surgical procedure. The hospital management servers 80 are realized using a processor equipped with hardware such as a CPU, a GPU, and an FPGA, and a HDD, an SSD, etc.

[Functional configuration of the main parts of the surgical training device]
Next, a description will be given of the functional configuration of the main parts of the surgical training device 2. FIG.

[Configuration of medical device]
First, a description will be given of the configuration of the medical instrument 30. The medical instrument 30 includes a first detection unit 31, a second detection unit 32, a recording unit 33, and a communication unit 34. Note that the medical instrument 40 has the same configuration as the medical instrument 30, and therefore detailed description thereof will be omitted.

The first detection unit 31 detects the insertion and removal of the medical instrument 30 held by the surgeon into and from the trocar 10, for example, the amount of pushing or pulling, and outputs this detection result to the communication unit 34. The first detection unit 31 is realized, for example, using a gyro sensor and an acceleration sensor.

The second detection unit 32 detects the pressure value (pressure value) or speed associated with the opening and closing operation of the medical instrument 30 operated by the surgeon, and outputs this detection result to the communication unit 34. The second detection unit 32 is realized using a pressure sensor, a gyro sensor, an acceleration sensor, etc.

The recording unit 33 is realized using a volatile memory or a non-volatile memory, and records medical instrument information related to the medical instrument 30. Here, the medical instrument information includes the model name, shape, product number, type information of the medical instrument 30 (e.g., type of forceps, electric scalpel, etc.), and manufacturing date. The recording unit 33 outputs the medical instrument information to the communication unit 34.

The communication unit 34 is capable of communicating with the first communication unit 61, and transmits the detection results of the first detection unit 31, the detection results of the second detection unit 32, and the medical instrument information recorded by the recording unit 33 to the control device 60. The communication unit 34 is realized using a communication module or the like.

[Configuration of the control device]
Next, a description will be given of the configuration of the control device 60. The control device 60 includes a first communication unit 61, an image processing unit 62, an input unit 63, a recording unit 64, a second communication unit 65, and a control unit 66.

Under the control of the control unit 66, the first communication unit 61 transmits various control data to the medical instruments 30, 40, and outputs various data transmitted from the medical instruments 30, 40 and the medical information management server 70 to the image processing unit 62 or the control unit 66. The first communication unit 61 is realized, for example, using an S/P conversion circuit and a communication module. The first communication unit 61 may also transmit various data to the medical instruments 30, 40 by wireless communication or optical communication.

Under the control of the control unit 66, the image processing unit 62 displays a simulation video based on the simulation data (polygon video data) input from the medical information management server 70 via the first communication unit 61. Specifically, the image processing unit 62 generates a simulation image of the observation object inside the subject (a simulation video in which the polygon data video data is rendered into graphics) and outputs this simulation video to the display device 50 to display the simulation video. The image processing unit 62 is realized using a processor having a memory and hardware such as a GPU (Graphics Processing Unit) and FPGA.

The input unit 63 receives input of operations from the user and outputs a signal to the control unit 66 according to the received operation. The input unit 63 is realized using a mouse, a keyboard, a touch panel, switches, buttons, etc.

The recording unit 64 records various data related to the surgical training device 2. The recording unit 64 has a program recording unit 641 that records various programs executed by the surgical training device 2. The recording unit 64 is realized using RAM (Random Access Memory), ROM (Read Only Memory), HDD (Hard Disk Drive), SSD (Solid State Drive), etc.

Under the control of the control unit 66, the second communication unit 65 acquires simulation data from the medical information management server 70 via the network 90, and outputs the acquired simulation data to the control unit 66 and the image processing unit 62. The second communication unit 65 is realized using a communication module or the like.

The control unit 66 controls each unit constituting the control device 60. The control unit 66 causes the display device 50 to display a medical instrument image related to the medical instrument on a simulation video based on simulation data acquired from the medical information management server 70 via the second communication unit 65. Specifically, the control unit 66 controls the display mode of the medical instrument image and the simulation video displayed by the display device 50 based on the operation content of the medical instrument 30 detected by each of the first detection unit 31 and the second detection unit 32. In addition, when a stop signal for stopping the simulation video displayed by the display device 50 is input from the input unit 63, the control unit 66 changes the viewpoint of the simulation video based on the operation content detected by each of the first detection unit 31 and the second detection unit 32. Specifically, the control unit 66 causes the display device 50 to display a simulation video of a viewpoint rotated 90 degrees clockwise from the lower layer of the simulation video or the visual field area of the current viewpoint. The control unit 66 is realized by using a processor having hardware such as a memory and a CPU. In one embodiment, the control unit 66 functions as a second control unit.

[Configuration of medical information management server]
Next, a description will be given of the configuration of the medical information management server 70. Fig. 3 is a block diagram showing the functional configuration of the medical information management server 70. The medical information management server 70 shown in Fig. 3 includes a communication unit 71, a recording unit 72, a generation unit 73, an inference unit 74, a simulation data recording unit 75, an image data recording unit 76, a user data recording unit 77, a medical instrument data recording unit 78, and a server control unit 79.

Under the control of the server control unit 79, the communication unit 71 transmits simulation data requested by the control device 60 (client side) to the control device 60 via the network 90. The communication unit 71 is realized using a communication module or the like.

The recording unit 72 records various data related to the medical information management server 70. The recording unit 72 has a program recording unit 721 that records various programs executed by the medical information management server 70. The recording unit 72 is realized using RAM, ROM, etc.

The generating unit 73 generates simulation data based on intraoperative video data captured during surgery of the subject, which is transmitted from each of the multiple hospital management servers 80 via the network 90, and at least one of two-dimensional image data and three-dimensional data captured in advance by a CT device or an MRI device, under the control of the server control unit 79, and records the simulation data in the simulation data recording unit 75. The generating unit 73 also generates simulation data by aligning the positions of organs included in the intraoperative video data and at least one of the two-dimensional image data and three-dimensional data, based on the inference results of the inference unit 74, which will be described later. Furthermore, the generating unit 73 generates simulation data that reflects the orientation and crushing state of organs included in the intraoperative video data and at least one of the two-dimensional image data and three-dimensional data, based on the inference results of the inference unit 74, which will be described later. The generating unit 73 is realized using a processor having a memory and hardware such as a GPU.

Under the control of the server control unit 79, the inference unit 74 estimates the organs and positions of the subject shown in the intraoperative video data based on the intraoperative video data transmitted from each of the multiple hospital management servers 80 via the network 90 and at least one of the two-dimensional image data and three-dimensional data, and outputs the estimation result to the generation unit 73. The inference unit 74 is realized using a trained model that uses as learning parameters (teaching data) the intraoperative video data, the organs and positions shown in the intraoperative video data, distance information indicating the distance from the tip of the medical imaging device (videoscope) to the organs, the feature quantities of the intraoperative video data, and multiple simulation data (e.g., polygon data based on at least one of CT image data and MRI image data), and the like as learning parameters, machine learning the learning parameters by deep learning or the like, and outputs the estimation result of estimating the organs and positions of the subject to be observed shown in the intraoperative video data as output parameters. Here, examples of the machine learning include deep neural network (DNN), convolutional neural network (CNN), and recurrent neural network (RNN). When the intraoperative video data transmitted from each of the hospital management servers 80 via the network 90 is multispectral image data irradiated with multispectral light by a light source device, the inference unit 74 estimates the biological tissue types (e.g., organs such as the gallbladder and pancreas, blood vessels, nerves, lymph, ureters, and fat) and their positions (areas) of each part of the subject to be observed that appears in the multispectral image data using the difference in reflected light frequency (for each spectrum) included in the multispectral image data, and outputs the estimation result to the generation unit 73. Furthermore, the inference unit 74 estimates the orientation (direction) and degree of crushing of organs, blood vessels, nerves, etc. shown in the intraoperative video data when the optical axis of the optical system of the medical imaging device (the insertion direction of the trocar) is used as a reference, based on the position and orientation of the medical imaging device, the insertion position of the subject, and the amount of operation (distance) inserted into the subject, which are associated with the intraoperative video data transmitted from each of the multiple hospital management servers 80 via the network 90, and outputs the estimation result to the generation unit 73. The inference unit 74 is realized using a processor having a memory and hardware such as a GPU.

The simulation data recording unit 75 records multiple surgical procedures and multiple pieces of simulation data in association with each other. Specifically, as shown in Table T1 in FIG. 4, the simulation data recording unit 75 records the surgical procedures, the simulation data, surgeon information identifying the surgeon who performed the simulation data, medical instrument information regarding the medical instrument used in the simulation data, and level information indicating the level of the simulation data in association with each other. The simulation data recording unit 75 is realized using a HDD, SSD, etc.

The image data recording unit 76 records two-dimensional image data captured by a CT device or three-dimensional image data captured by an MRI examination, which is transmitted from each of the multiple hospital management servers 80 via the network 90, in association with hospital identification information indicating the name of the hospital and subject information (patient information) relating to the subject. Here, subject information refers to the subject's name, disease name, medical history, biological information, attending physician, and surgeon, etc. The image data recording unit 76 is realized using an HDD, SSD, etc.

The user data recording unit 77 records identification information for each of the multiple operators U1 who have trained using the surgical training device 2 in the medical information management server 70 via the network 90. Specifically, as shown in Table T2 in FIG. 5, the user data recording unit 77 records the user ID, the level (difficulty) of the surgical training of the operator U1, and the training content as identification information for identifying the operator U1, in association with each other. Furthermore, the user data recording unit 77 may record the number of training sessions, the hospital or academic society to which the user ID belongs, the address, and the bank account in association with the user ID. The user data recording unit 77 is realized using a HDD, SSD, or the like.

The medical instrument data recording unit 78 records medical instrument information related to the medical instrument. Here, medical instrument information includes the model name, model number, animation data, CAD data (polygon data), etc. of the medical instrument.

The server control unit 79 controls each unit constituting the medical information management server 70. The server control unit 79 is realized using a processor having a memory and hardware such as a CPU. When one of a plurality of surgical procedures is selected by the control device 60 (client side), the server control unit 79 transmits simulation data corresponding to the selected surgical procedure to the control device 60 (client) side. The server control unit 79 has a determination unit 791, a billing unit 792, and a point assignment unit 793. In one embodiment, the server control unit 79 functions as the first control unit.

The determination unit 791 determines the identification information of the operator U1 who accessed the medical information management server 70 via the control device 60 and the network 90, and the identification information recorded by the user data recording unit 77, and outputs the determination result to the billing unit 792 and the point assignment unit 793.

The billing unit 792 performs billing based on the result of the judgment by the judgment unit 791 by billing the registered address or affiliated hospital registered by the user corresponding to the user ID recorded by the user data recording unit 77 for a fee based on the level or number of times the simulation data was downloaded to the control device 60.

Based on the result of the judgment by the judgment unit 791, the score assignment unit 793 assigns points to the user ID recorded by the user data recording unit 77 according to the level associated with the simulation video or the number of times training was performed using the simulation data.

[Training method overview]
Next, there will be described an outline of a surgical training method using the surgical training system 1. Figures 6 to 11 are diagrams showing examples of setting screens displayed on the display device 50 when performing surgical training.

First, the operator U1 uses the input unit 63 of the control device 60 to access the browser of the medical information management server 70 and inputs identification information (user ID, password, etc.) for identifying the operator U1. In this case, the medical information management server 70 transmits training screen data to the control device 60 (client side) that has accessed the browser. As a result, as shown in FIG. 6, the display device 50 displays a training screen W1 corresponding to the training screen data under the control of the control device 60. The training screen W1 includes at least the level of the surgical technique, the surgical target of the surgical technique (e.g., rectum), the mission (e.g., tumor removal and suturing), and a thumbnail image SM1 corresponding to the simulation data. As shown in FIG. 6, the operator U1 uses the input unit 63 to operate a pointer or the like and set the desired level and the desired surgical target in each predetermined cell. In this case, the control device 60 transmits the setting contents set by the operator U1 to the medical information management server 70.

Also, as shown in FIG. 7, when a request signal is input from the control device 60 requesting the setting of realism settings for surgical procedures and various modes during training, the medical information management server 70 transmits realism setting and panic mode setting screen data to the control device 60. As a result, as shown in FIG. 7, the display device 50 displays the realism setting and panic mode setting screen W2 under the control of the control device 60. The realism setting and panic mode setting screen W2 includes at least bleeding speed, blood pressure drop speed, O2 drop, malformed organs and blood vessels, hardness and fragility, vital signs, and the presence or absence of panic during surgery.

Furthermore, as shown in FIG. 8, when a request signal for selecting and setting a medical tool is input from the control device 60, the medical information management server 70 transmits the tool selection and setting setting screen data to the control device 60. As a result, as shown in FIG. 8, the display device 50 displays the tool selection and setting setting screen W3 under the control of the control device 60. The tool selection and setting setting screen W3 includes multiple medical tools M1 to M5 (multiple treatment tools) used in the surgical procedure selected by the operator U1. For example, the tool selection and setting setting screen W3 includes forceps, an electric scalpel, a medical imaging device (rigid scope), etc. The operator U1 selects the medical tools to be used in the surgical procedure to be performed in training. Furthermore, the operator U1 can increase the number of medical tools (selection items) that can be selected by charging the medical information management server 70. In this case, the medical information management server 70 charges a fee according to the number of selected medical tools based on the identification information of the operator U1.

Also, as shown in FIG. 9, when a request signal for requesting a surgical procedure or case base is input from the control device 60, the medical information management server 70 transmits case selection screen data displaying a list of thumbnail images of multiple simulation data corresponding to each of multiple surgical procedures or multiple cases to the control device 60. As a result, as shown in FIG. 9, the display device 50 displays a case selection screen W4 under the control of the control device 60. The case selection screen W4 includes thumbnail images P1 to P5 corresponding to each of the multiple simulation data. The operator U1 selects the desired thumbnail image. In this case, the medical information management server 70 transmits to the control device 60 surgeon data related to the surgeon (operator) who performed the simulation data of the thumbnail image corresponding to the selection signal input from the control device 60. As a result, as shown in FIG. 10, the display device 50 displays a surgeon screen W5 related to the surgeon, such as a supervising professor, under the control of the control device 60. The surgeon screen W5 includes a portrait of the surgeon (operator), the case operated by the surgeon, the surgical procedure, the level and message by the surgical training system 1, etc.

Also, as shown in FIG. 11, when a request signal requesting simulation data of a new approach is input from the control device 60, the medical information management server 70 transmits simulation data screen data of the new approach to the control device 60. As a result, as shown in FIG. 10, the display device 50 displays a simulation data screen W6 of the new approach under the control of the control device 60. The simulation data screen W6 includes a thumbnail image SM2 corresponding to the simulation video, etc.

In this way, the operator U1 can perform training by using the input unit 63 of the control device 60 to download simulation data of the desired surgical case according to the level from the medical information management server 70 to the control device 60.

[Medical information management server processing]
Next, a description will be given of the processing executed by the medical information management server 70. FIG 12 is a flowchart showing an outline of the processing executed by the medical information management server 70.

First, the server control unit 79 determines whether or not a request has been made from the client side (step S1). Specifically, the server control unit 79 determines whether or not a request signal requesting various types of simulation data has been input from the control device 60 (client side) via the network 90. If the server control unit 79 determines that a request has been made from the client side (step S1: Yes), the medical information management server 70 proceeds to step S2, which will be described later. On the other hand, if the server control unit 79 determines that no request has been made from the client side (step S1: No), the medical information management server 70 proceeds to step S4, which will be described later.

In step S2, the medical information management server 70 executes a simulation data transmission process to transmit simulation data to the client side. Details of the simulation video transmission process will be described later.

Then, the server control unit 79 judges whether or not the access from the client side has ended (step S3). If the server control unit 79 judges that the access from the client side has ended (step S3: Yes), the medical information management server 70 ends this process. On the other hand, if the server control unit 79 judges that the access from the client side has not ended (step S3: No), the medical information management server 70 returns to the above-mentioned step S1.

In step S4, the server control unit 79 determines whether or not intraoperative video data, two-dimensional image data, and three-dimensional image data have been received from one of the multiple hospital management servers 80 via the network 90. If the server control unit 79 determines that intraoperative video data, two-dimensional image data, and three-dimensional image data have been received from one of the multiple hospital management servers 80 (step S4: Yes), the medical information management server 70 proceeds to step S5, which will be described later. On the other hand, if the server control unit 79 determines that intraoperative video data, two-dimensional image data, and three-dimensional image data have not been received from one of the multiple hospital management servers 80 (step S4: No), the medical information management server 70 proceeds to step S3.

In step S5, the medical information management server executes a simulation data generation process to generate simulation data based on the intraoperative video data and either the two-dimensional image data or the three-dimensional image data. Details of the simulation data generation process will be described later. After step S5, the medical information management server 70 proceeds to step S3.

[Outline of simulation data transmission process]
Next, an outline of the simulation data transmission process described in step S2 of Fig. 12 will be described below. Fig. 13 is a flow chart showing an outline of the simulation data transmission process.

As shown in FIG. 13, first, the server control unit 79 acquires identification information for identifying the operator U1 transmitted from the control device 60 (client side) accessed via the network 90 and the communication unit 71 (step S101). In this case, the determination unit 791 determines whether the identification information recorded by the user data recording unit 77 is the identification information of the operator U1 transmitted from the control device 60.

Then, the determination unit 791 determines whether or not a setting signal for setting a mission in surgical training has been input from the control device 60 (step S102). If the determination unit 791 determines that a setting signal for setting a mission in surgical training has been input from the control device 60 (step S102: Yes), the medical information management server 70 proceeds to step S103, which will be described later. On the other hand, if the determination unit 791 determines that a setting signal for setting a mission in surgical training has not been input from the control device 60 (step S102: No), the medical information management server 70 proceeds to step S105, which will be described later.

In step S103, the server control unit 79 causes the control device 60 to transmit simulation data corresponding to the mission set in the surgical training to the communication unit 71 (step S103). Specifically, the server control unit 79 acquires simulation data corresponding to the mission set in the surgical training from the control device 60 from among the multiple simulation data recorded by the simulation data recording unit 75, and causes the acquired simulation data to be transmitted to the communication unit 71. This allows the operator U1 to perform training according to the set mission while watching the simulation video displayed by the display device 50.

Then, the server control unit 79 charges the operator U1 and assigns points to the identification information based on the identification information and the level of the simulation data downloaded to the control device 60 (step S104). Specifically, the charging unit 792 charges the operator U1 by charging a fee based on the level or number of times of the simulation data downloaded to the control device 60 to the registered address or affiliated hospital registered by the user who matches the user ID recorded by the user data recording unit 77 determined by the determination unit 791. Furthermore, the score assigning unit 793 assigns points according to the level associated with the simulation video or the number of times training was performed with the simulation data to the user ID recorded by the user data recording unit 77 determined by the determination unit 791. After step S104, the medical information management server 70 returns to the main routine of FIG. 12. Note that the charging method by the charging unit 792 is one example, and in addition to the level of the simulation data, it may also be charged for downloading medical instrument information of a new medical instrument, downloading simulation data of a new surgical procedure, and downloading simulation data by a specified surgeon.

In step S105, the judgment unit 791 judges whether or not a request signal for selecting a desired surgical procedure case from the multiple surgical procedure cases has been input from the control device 60. If the judgment unit 791 judges that a request signal for selecting a desired surgical procedure case from the multiple surgical procedure cases has been input from the control device 60 (step S105: Yes), the medical information management server 70 proceeds to step S106, which will be described later. On the other hand, if the judgment unit 791 judges that a request signal for selecting a desired surgical procedure case from the multiple surgical procedure cases has not been input from the control device 60 (step S105: No), the medical information management server 70 proceeds to step S109, which will be described later.

In step S106, the server control unit 79 causes the communication unit 71 to transmit surgical procedure case information that summarizes multiple surgical procedure cases corresponding to each of the multiple simulation data recorded by the simulation data recording unit 75.

Next, the server control unit 79 acquires information about the medical instruments used by the operator U1 in the surgical training from the control device 60 via the network 90 and the communication unit 71 (step S107).

Then, the server control unit 79 transmits simulation data to the communication unit 71 based on the surgical procedure case and medical instrument information selected by the operator U1 from among the multiple simulation data recorded by the simulation data recording unit 75 (step S108). This allows the operator U1 to perform surgical training for the desired surgical procedure case using the medical instruments 30 and 40 while watching the simulation video displayed on the display device 50. After step S108, the medical information management server 70 proceeds to step S104.

In step S109, the determination unit 791 determines whether a request signal for selecting a desired surgeon from multiple surgeons has been input from the control device 60. If the determination unit 791 determines that a request signal for selecting a desired surgeon from multiple surgeons has been input from the control device 60 (step S109: Yes), the medical information management server 70 proceeds to step S110, which will be described later. On the other hand, if the determination unit 791 determines that a request signal for selecting a desired surgeon from multiple surgeons has not been input from the control device 60 (step S109: No), the medical information management server 70 proceeds to step S104.

In step S110, the server control unit 79 causes the communication unit 71 to transmit surgeon information summarizing multiple surgeons corresponding to each of the multiple simulation data recorded by the simulation data recording unit 75.

The server control unit 79 then acquires the surgeon information selected by the operator U1 from the control device 60 via the network 90 and the communication unit 71 (step S111), and causes the communication unit 71 to transmit simulation data based on the surgeon information selected by the operator U1 from among the multiple simulation data recorded by the simulation data recording unit 75 (step S112). This allows the operator U1 to trace and learn the surgical technique performed by the surgeon while watching the simulation video displayed on the display device 50. After step S112, the medical information management server 70 proceeds to step S104.

[Outline of simulation data generation process]
Next, an outline of the simulation data generation process explained in step S5 of Fig. 12 will be described below. Fig. 14 is a flow chart showing an outline of the simulation data generation process.

14, first, the inference unit 74 estimates the organs and positions of the subject shown in the intraoperative video data based on the intraoperative video data transmitted from each of the multiple hospital management servers 80 via the network 90 and at least one of the two-dimensional image data and the three-dimensional data including at least data related to the organs (step S201). Specifically, when the intraoperative video data transmitted from each of the multiple hospital management servers 80 via the network 90 is multispectral image data irradiated with multispectral light by a light source device, the inference unit 74 estimates the types of biological tissues (e.g., organs such as the gallbladder and pancreas, blood vessels, nerves, lymph, ureters, and fat) and their positions (areas) of each part of the subject to be observed shown in the multispectral image data using the difference in the reflected light frequency (for each spectrum) contained in the multispectral image data, and outputs the estimation result to the generation unit 73. In addition, the inference unit 74 may estimate the orientation (direction) and degree of crushing of organs, blood vessels, nerves, etc. shown in the intraoperative video data when the optical axis of the optical system of the medical imaging device (the insertion direction of the trocar) is used as a reference, based on the position and orientation of the medical imaging device, the insertion position of the subject, and the amount of operation (distance) inserted into the subject, which are associated with the intraoperative video data transmitted from each of the multiple hospital management servers 80 via the network 90, and output the estimation result to the generation unit 73.

Next, the generating unit 73 aligns the positions of the organs included in the intraoperative video data and at least one of the two-dimensional image data and three-dimensional data based on the inference result of the inference unit 74 to generate simulation data (step S202). Specifically, the generating unit 73 determines the orientation, position, and identification of the organs included in the intraoperative video data and at least one of the two-dimensional image data and three-dimensional data based on the inference result of the inference unit 74, and then aligns the organs to generate simulation data that reflects the degree of crushing of the organs expected from the current viewpoint, etc.

Then, the server control unit 79 acquires the surgeon information from the metadata (header information) included in the intraoperative video file that stores the intraoperative video data (step S203), and acquires the medical instrument information from the metadata (header information) included in the intraoperative video file that stores the intraoperative video data (step S204).

Then, the server control unit 79 generates a simulation file that associates the simulation data generated by the generation unit 73 with the surgeon information and the medical instrument information, and records the simulation file in the simulation data recording unit 75 (step S205). After step S205, the medical information management server 70 returns to the main routine of FIG. 12.

Processing of surgical training equipment
Next, a description will be given of the processing executed by the surgery training device 2. FIG.

First, the control unit 66 causes the identification information of the operator U1 input via the input unit 63 to be transmitted to the second communication unit 65 (step S301).

Next, if the operator U1 selects the mission mode via the input unit 63 (step S302: Yes), the control unit 66 causes the display device 50 to display mission information that can be set in the surgical training (step S303).

Then, the control unit 66 causes the mission information set by the operator U1 using the input unit 63 to be transmitted to the second communication unit 65 (step S304).

Next, the control unit 66 causes the display device 50 to display a simulation video based on the simulation data corresponding to the mission set for the operator U1 and transmitted from the medical information management server 70 (step S305).

Then, the control unit 66 determines whether or not a stop signal to stop the playback of the simulation video has been input from the input unit 63 (step S306). If the control unit 66 determines that a stop signal to stop the playback of the simulation video has been input from the input unit 63 (step S306: Yes), the surgical training device 2 proceeds to step S307, which will be described later. On the other hand, if the control unit 66 determines that a stop signal to stop the playback of the simulation video has not been input from the input unit 63 (step S306: No), the surgical training device 2 proceeds to step S309, which will be described later.

In step S307, the control unit 66 controls the viewpoint of the simulation video based on the operation of the medical instruments 30 and 40. Specifically, the control unit 66 controls the image processing unit 62 based on the operation of the medical instruments 30 and 40 to generate an image of a display layer lower than the image of the current display layer as the viewpoint of the simulation video, or to generate an image of a display layer that can be observed in a state where the viewpoint is changed 90 degrees clockwise from the current viewpoint, thereby controlling the viewpoint of the simulation video displayed by the display device 50. Of course, the control unit 66 may also perform, for example, zooming in or zooming out in addition to changing the viewpoint of the simulation video by controlling the image processing unit 62 based on the operation of the medical instruments 30 and 40.

Next, if the playback of the simulation video has ended (step S308: Yes), the surgical training device 2 ends this process. On the other hand, if the playback of the simulation video has not ended (step S308: No), the surgical training device 2 proceeds to step S306 described above.

In step S309, the control unit 66 controls the display mode of the simulation video based on the simulation data, based on the operation content of the medical instruments 30, 40. Specifically, the control unit 66 moves the medical instruments 30, 40 shown in the simulation video in accordance with the operation content of the operator U1, based on the operation content of the medical instruments 30, 40, and reflects the treatment by the medical instruments 30, 40 in the simulation video. After step S309, the surgical training device 2 proceeds to step S308.

Next, a case will be described where the operator U1 has not selected the mission mode via the input unit 63 in step S302 (step S302: No). In this case, the surgical training device 2 proceeds to step S310.

Next, if the operator U1 selects the surgical procedure case mode via the input unit 63 (step S310: Yes), the surgical training device 2 proceeds to step S311, which will be described later. On the other hand, if the operator U1 does not select the surgical procedure case mode via the input unit 63 (step S310: No), the surgical training device 2 proceeds to step S315, which will be described later.

In step S311, the control unit 66 causes the display device 50 to display surgical procedure case information that can be selected during surgical training.

Next, the control unit 66 causes the surgical procedure case information selected by the operator U1 using the input unit 63 to be transmitted to the second communication unit 65 (step S312).

Then, the control unit 66 causes the second communication unit 65 to transmit medical instrument information regarding the medical instruments 30 and 40 (step S313).

Then, the control unit 66 causes the display device 50 to display a simulation video based on the simulation data corresponding to the surgical procedure case selected by the operator U1 and transmitted from the medical information management server 70 (step S314). After step S314, the surgical training device 2 proceeds to step S306.

In step S315, if the operator U1 selects the surgeon mode via the input unit 63 (step S315: Yes), the surgical training device 2 proceeds to step S316, which will be described later.

In step S316, the control unit 66 causes the display device 50 to display surgeon information that allows multiple surgeons to be selected in the surgery mode.

Next, the surgeon information selected by the operator U1 is transmitted to the second communication unit 65 via the input unit 63 (step S317).

Then, the control unit 66 causes the display device 50 to display a simulation video based on the simulation data related to the surgeon information selected by the operator U1 and transmitted from the medical information management server 70 (step S318). After step S318, the surgical training device 2 proceeds to step S306.

According to the embodiment described above, when one of a number of surgical procedures is selected by the control device 60 (client side), the server control unit 79 transmits simulation data corresponding to the selected surgical procedure to the control device 60, so that the operator U1 can gain experience equivalent to an actual surgery, thereby improving the learning curve of the operator U1.

Furthermore, according to one embodiment, training in surgical techniques can be carried out in one's spare time, regardless of location.

In addition, according to one embodiment, the generating unit 73 generates simulation data based on intraoperative video data captured during surgery on the subject by a medical imaging device transmitted from each of the hospital management servers 80, and at least one of two-dimensional image data and three-dimensional data including data on at least the organs of the subject captured in advance by a CT device or an MRI device, so that simulation data for multiple surgical procedures and multiple cases can be easily collected.

Furthermore, according to one embodiment, the generating unit 73 generates simulation data by aligning the organs contained in the intraoperative video data and at least one of the two-dimensional image data and three-dimensional data based on the estimation results of the inference unit 74, so that realistic simulation data can be generated.

In addition, according to one embodiment, the simulation data recording unit 75 records multiple pieces of simulation data in association with multiple pieces of surgeon information related to multiple surgeons who performed each of the multiple simulation data, so that the operator U1 can select the simulation data performed by the desired surgeon and perform training.

In addition, according to one embodiment, the simulation data recording unit 75 records multiple pieces of simulation data in association with multiple pieces of medical instrument information related to multiple medical instruments used in each of the multiple simulation data, so that the operator U1 can select simulation data using the desired medical instrument and perform training.

Furthermore, according to one embodiment, when one or more of the multiple pieces of medical instrument information stored in the control device 60 (client side) are selected, the server control unit 79 transmits the selected medical instrument information to the control device 60 side, so that the operator U1 can train with the simulation data using the desired medical instrument.

Furthermore, according to one embodiment, when the billing unit 792 of the server control unit 79 transmits simulation data to the control device 60 (client side), the operator U1 who operates the client side is billed based on the identification information of the operator U1, so that a profit can be obtained.

In addition, according to one embodiment, the server control unit 79 transmits surgical technique information regarding surgical techniques selectable by the control device 60 to the control device 60 based on the level associated with the identification information of the operator U1 transmitted from the control device 60, so that simulation data appropriate for the level of the operator U1 can be selected.

In addition, according to one embodiment, the server control unit 79 assigns points to the number of times the operator U1 virtually trains using simulation data based on the identification information of the operator U1, so that the level of the technique performed by the operator U1 can be easily understood.

In addition, according to one embodiment, the simulation data recording unit 75 further associates and records each level of the multiple simulation data, so that the operator U1 can select simulation data that corresponds to his or her own level and perform training.

In addition, according to one embodiment, the control unit 66 controls the display mode of the medical instrument image and the simulation video displayed by the display device 50 based on the operation content of the operator U1 detected by each of the first detection unit 31 and the second detection unit 32, allowing intuitive operation.

In addition, according to one embodiment, when the control unit 66 receives a stop signal from the input unit 63 to stop the simulation video displayed by the display device 50, and detects the operation content detected by each of the first detection unit 31 and the second detection unit 32, the viewpoint of the simulation video is changed based on the operation content, so that the operator U1 can observe the simulation video from a viewpoint desired.

In addition, according to one embodiment, the control unit 66 causes the display device 50 to display multiple pieces of surgical procedure information related to each of the multiple surgical procedures transmitted from the medical information management server 70, and when any one or more pieces of surgical procedure information is selected via the input unit 63, the control unit 66 requests the medical information management server 70 to transmit simulation data corresponding to the selected surgical procedure information, so that the simulation data corresponding to the case desired by the operator U1 can be obtained from the medical information management server 70.

Furthermore, according to one embodiment, the control unit 66 transmits identification information of the operator U1 to the medical information management server 70, so that simulation data corresponding to the operator U1 can be acquired.

In addition, according to one embodiment, the control unit 66 transmits medical instrument information regarding each of the medical instruments 30, 40 used by the operator U1 to the medical information management server 70, so that simulation data suitable for the medical instruments 30, 40 used by the operator U1 for training can be obtained.

In one embodiment, one operator U1 performs the training, but the present invention is not limited to this. Multiple operators U1 may perform the training while watching the same simulation video displayed by the display device 50, or multiple operators U1 may perform the training using surgical training devices installed in different locations.

Other Embodiments
Various inventions can be formed by appropriately combining the multiple components disclosed in the above-mentioned surgical training system according to one embodiment of the present disclosure. For example, some components may be deleted from all the components described in the above-mentioned surgical training system according to one embodiment of the present disclosure. Furthermore, the components described in the above-mentioned medical observation system according to one embodiment of the present disclosure may be appropriately combined.

In addition, in the surgical training system according to one embodiment of the present disclosure, the "unit" described above can be read as "means" or "circuit." For example, the control unit can be read as control means or control circuit.

The program executed by the surgical training system according to one embodiment of the present disclosure is provided as file data in an installable or executable format recorded on a computer-readable recording medium such as a CD-ROM, a flexible disk (FD), a CD-R, a DVD (Digital Versatile Disk), a USB medium, or a flash memory.

In addition, the program executed by the surgical training system according to one embodiment of the present disclosure may be configured to be stored on a computer connected to a network such as the Internet and provided by downloading via the network.

Note that in the explanation of the flowcharts in this specification, the order of processing between steps is clearly indicated using expressions such as "first," "then," and "continue." However, the order of processing required to implement the present invention is not uniquely determined by these expressions. In other words, the order of processing in the flowcharts described in this specification can be changed as long as there are no contradictions.

Although some of the embodiments of the present application have been described in detail above with reference to the drawings, these are merely examples, and the present disclosure can be implemented in other forms with various modifications and improvements based on the knowledge of those skilled in the art, including the forms described in the section on this disclosure.

The present technology can also be configured as follows.
(Appendix 1)
a recording unit that records at least a plurality of surgical procedures and a plurality of simulation data including at least data related to biological tissue in association with each other;
a control unit that, when any one of the plurality of surgical procedures is selected on a client side, transmits simulation data corresponding to the selected surgical procedure to the client side;
Equipped with
Medical information management server.
(Appendix 2)
A medical information management server according to (Supplementary Note 1),
a generating unit configured to generate the simulation data based on intraoperative video data captured during an operation on a subject and at least one of two-dimensional image data and three-dimensional data captured in advance by a CT device or an MRI device on the subject;
Medical information management server.
(Appendix 3)
A medical information management server according to (Supplementary Note 2),
An inference unit that estimates organs and positions of the subject shown in the intraoperative video data based on the intraoperative video data and at least one of the two-dimensional image data and the three-dimensional data,
The generation unit is
generating the simulation data by aligning organs included in the intraoperative video data and at least one of the two-dimensional image data and the three-dimensional data based on the estimation result of the inference unit;
Medical information management server.
(Appendix 4)
A medical information management server according to (Supplementary Note 3),
The intraoperative video data is
a multispectral light source is used to irradiate the subject with multispectral light, and a return light is received from the subject, thereby obtaining spectral image data including at least a reflection frequency for each biological tissue;
The inference unit is
estimating at least the type and location of the biological tissue using a reflection frequency for each of the biological tissues included in the spectral image data;
Medical information management server.
(Appendix 5)
A medical information management server according to any one of (Supplementary Note 1) to (Supplementary Note 4),
The recording unit is
a plurality of pieces of surgeon information relating to a plurality of surgeons who performed each of the plurality of simulation data are further associated with the plurality of simulation data and recorded;
The control unit is
When any one of the plurality of pieces of surgeon information is selected on the client side, the plurality of pieces of simulation data corresponding to the selected piece of surgeon information are transmitted to the client side.
Medical information management server.
(Appendix 6)
A medical information management server according to any one of (Supplementary Note 1) to (Supplementary Note 5),
The recording unit is
a plurality of pieces of medical instrument information related to a plurality of medical instruments used in each of the plurality of pieces of simulation data are further associated with the plurality of pieces of simulation data and recorded;
The control unit is
When transmitting the simulation data to the client side, the medical device information associated with the simulation data is also transmitted to the client side.
Medical information management server.
(Appendix 7)
A medical information management server according to (Supplementary Note 6),
The control unit is
When one or more of the plurality of pieces of medical instrument information are selected on the client side, the selected pieces of medical instrument information are transmitted to the client side.
Medical information management server.
(Appendix 8)
A medical information management server according to any one of (Supplementary Note 1) to (Supplementary Note 7),
The control unit is
when the simulation data is transmitted to the client side, a fee is charged to the operator operating the client side based on identification information of the operator.
Medical information management server.
(Appendix 9)
A medical information management server according to (Supplementary Note 8),
The control unit is
transmitting surgical procedure information regarding the surgical procedure selectable on the client side to the client side based on the level associated with the identification information;
Medical information management server.
(Appendix 10)
A medical information management server according to (Supplementary Note 9),
The control unit is
giving a score to the number of times the operator has virtually performed training using the simulation data based on the identification information;
Medical information management server.
(Appendix 11)
A medical information management server according to any one of (Supplementary Note 8) to (Supplementary Note 10),
The recording unit is
further correlating and recording each level of the plurality of simulation data;
Medical information management server.
(Appendix 12)
a display unit capable of displaying a simulation video based on a plurality of simulation data including at least data related to a biological tissue and a plurality of surgical procedures transmitted from a medical information management server that records the plurality of simulation data in association with each other;
one or more medical devices;
A detection unit that detects an operation performed by an operator on the medical instrument;
a control unit that causes a medical instrument image related to the medical instrument to be displayed on the display unit on the simulation video;
Equipped with
The control unit controls a display mode of the medical instrument image and the simulation video based on the operation content detected by the detection unit.
Surgical training equipment.
(Appendix 13)
13. The surgical training apparatus according to claim 12,
The control unit is
when a stop signal for stopping the simulation moving image displayed by the display unit is input, and when the detection unit detects the operation content, the viewpoint of the simulation moving image is changed based on the operation content.
Surgical training equipment.
(Appendix 14)
14. The surgical training apparatus according to claim 13,
The control unit is
displaying a plurality of pieces of surgical procedure information related to each of the plurality of surgical procedures on the display unit, and when any one or more of the plurality of pieces of surgical procedure information is selected, requesting the medical information management server to transmit the simulation data corresponding to the selected surgical procedure information.
Surgical training equipment.
(Appendix 15)
15. The surgical training apparatus according to claim 14,
an input unit for receiving input of identification information of the operator,
The control unit is
transmitting the identification information to the medical information management server;
Surgical training equipment.
(Appendix 16)
A surgical training apparatus according to any one of (Supplementary Note 12) to (Supplementary Note 15),
The control unit is
transmitting medical instrument information relating to the medical instrument to the medical information management server;
Surgical training equipment.
(Appendix 17)
A medical information management server;
a surgery training device capable of communicating with the medical information management server via a network;
Equipped with
The medical information management server includes:
a recording unit that records at least a plurality of surgical procedures and a plurality of simulation data including at least data related to biological tissue in association with each other;
a first control unit that, when any one of the plurality of surgical procedures is selected on a client side, transmits simulation data corresponding to the selected surgical procedure to the client side;
Equipped with
The surgical training device comprises:
a display unit capable of displaying a simulation video based on the simulation data transmitted from the medical information management server;
one or more medical instruments capable of performing treatment on a model that mimics a subject;
A detection unit that detects an operation performed by an operator on the medical instrument;
a second control unit that causes a medical instrument image related to the medical instrument to be displayed on the display unit on the simulation moving image;
Equipped with
The second control unit controls a display mode of the medical instrument image and the simulation video based on the operation content detected by the detection unit.
Surgical training system.
(Appendix 18)
An image transmission method executed by a medical information management server including a recording unit that records at least a plurality of surgical procedures and a plurality of simulation data including at least data related to biological tissue in association with each other, the method comprising:
When any one of the plurality of surgical procedures is selected on the client side, simulation data corresponding to the selected surgical procedure is transmitted to the client side.
How to send images.
(Appendix 19)
A display method is executed by a surgical training device including: a display unit capable of displaying a simulation video based on simulation data transmitted from a medical information management server that records at least a plurality of surgical procedures and a plurality of simulation data including at least data related to biological tissues in association with each other; and one or more medical instruments capable of treating a model simulating a subject, the display unit comprising:
Detecting an operation performed by an operator on the medical instrument;
causing the display unit to display a medical instrument image relating to the medical instrument on the simulation video, and controlling a display mode of the medical instrument image and the simulation video based on the operation content.
Display method.
(Appendix 20)
A program executed by a medical information management server having a recording unit that records at least a plurality of surgical procedures and a plurality of simulation data including at least data related to biological tissue in association with each other,
When any one of the plurality of surgical procedures is selected on a client side, simulation data corresponding to the selected surgical procedure is transmitted to the client side.
program.
(Appendix 21)
A program to be executed by a surgical training device, the program comprising: a display unit capable of displaying a simulation video based on simulation data transmitted from a medical information management server that records at least a plurality of surgical procedures and a plurality of simulation data including at least data related to biological tissues in association with each other; and one or more medical instruments capable of treating a model simulating a subject, the program comprising:
detecting an operation performed by an operator on the medical instrument;
causing the display unit to display a medical instrument image relating to the medical instrument on the simulation video, and causing the display unit to control a display mode of the medical instrument image and the simulation video based on the operation content.
program.

LIST OF SYMBOLS 1 Surgery training system 2 Surgery training device 10, 20 Trocar 30, 40 Medical instrument 31 First detection unit 32 Second detection unit 33, 64, 72 Recording unit 34, 71 Communication unit 50 Display device 60 Control device 61 Communication unit 1 62 Image processing unit 63 Input unit 65 Second communication unit 66 Control unit 70 Medical information management server 71 Communication unit 73 Generation unit 74 Inference unit 75 Simulation data recording unit 76 Image data recording unit 77 User data recording unit 78 Medical instrument data recording unit 79 Server control unit 80 Hospital management server 83 Input unit 90 Network 641 Program recording unit 721 Program recording unit 791 Determination unit 792 Charging unit 793 Point assignment unit O ₁ model U1 Operator

Claims (11)

a recording unit that records at least a plurality of surgical procedures and a plurality of simulation data including at least data related to biological tissue in association with each other;
a control unit that, when any one of the plurality of surgical procedures is selected on a client side, transmits simulation data corresponding to the selected surgical procedure to the client side;
a generating unit that generates the simulation data based on intraoperative video data captured during an operation on a subject and at least one of two-dimensional image data and three-dimensional data captured in advance by a CT device or an MRI device on the subject;
an inference unit that estimates organs and positions of the subject shown in the intraoperative video data based on the intraoperative video data and at least one of the two-dimensional image data and the three-dimensional data;
Equipped with
The generation unit is
generating the simulation data by aligning organs included in the intraoperative video data and at least one of the two-dimensional image data and the three-dimensional data based on the estimation result of the inference unit;
Medical information management server. 2. The medical information management server according to claim 1 ,
The intraoperative video data is
a multispectral light source is used to irradiate the subject with multispectral light, and a return light is received from the subject, thereby obtaining spectral image data including at least a reflection frequency for each biological tissue;
The inference unit is
estimating at least the type and location of the biological tissue using a reflection frequency for each of the biological tissues included in the spectral image data;
Medical information management server. 2. The medical information management server according to claim 1,
The recording unit is
a plurality of pieces of simulation data are recorded in association with a plurality of pieces of surgeon information relating to a plurality of surgeons who performed the plurality of pieces of simulation data;
The control unit is
When any one of the plurality of pieces of surgeon information is selected on the client side, the plurality of pieces of simulation data corresponding to the selected piece of surgeon information are transmitted to the client side.
Medical information management server. 2. The medical information management server according to claim 1,
The recording unit is
a plurality of pieces of medical instrument information related to a plurality of medical instruments used in each of the plurality of pieces of simulation data are further associated with the plurality of pieces of simulation data and recorded;
The control unit is
When transmitting the simulation data to the client side, the medical device information associated with the simulation data is also transmitted to the client side.
Medical information management server. 5. The medical information management server according to claim 4 ,
The control unit is
When one or more of the plurality of pieces of medical instrument information are selected on the client side, the selected pieces of medical instrument information are transmitted to the client side.
Medical information management server. 2. The medical information management server according to claim 1,
The control unit is
when the simulation data is transmitted to the client side, a fee is charged to the operator operating the client side based on identification information of the operator.
Medical information management server. 7. The medical information management server according to claim 6 ,
The control unit is
transmitting surgical procedure information regarding the surgical procedure selectable on the client side to the client side based on the level associated with the identification information;
Medical information management server. The medical information management server according to claim 7 ,
The control unit is
giving a score to the number of times the operator has virtually performed training using the simulation data based on the identification information;
Medical information management server. 7. The medical information management server according to claim 6 ,
The recording unit is
further correlating and recording each level of the plurality of simulation data;
Medical information management server. An image transmission method executed by a medical information management server including a recording unit that records at least a plurality of surgical procedures and a plurality of simulation data including at least data related to biological tissue in association with each other, the method comprising:
a control step of transmitting , when any one of the plurality of surgical procedures is selected on a client side, simulation data corresponding to the selected surgical procedure to the client side;
a generating step of generating the simulation data based on intraoperative video data captured during an operation on a subject and at least one of two-dimensional image data and three-dimensional data captured in advance by a CT device or an MRI device on the subject;
an estimation step of estimating organs and positions of the subject shown in the intraoperative video data based on the intraoperative video data and at least one of the two-dimensional image data and the three-dimensional data;
Including,
The generating step includes:
generating the simulation data by aligning organs included in the intraoperative video data and at least one of the two-dimensional image data and the three-dimensional data based on the estimation result of the estimation step;
How to send images. A program executed by a medical information management server having a recording unit that records at least a plurality of surgical procedures and a plurality of simulation data including at least data related to biological tissue in association with each other,
a control step of transmitting , when any one of the plurality of surgical procedures is selected on a client side, simulation data corresponding to the selected surgical procedure to the client side;
a generating step of generating the simulation data based on intraoperative video data captured during an operation on a subject and at least one of two-dimensional image data and three-dimensional data captured in advance by a CT device or an MRI device on the subject;
an estimation step of estimating organs and positions of the subject shown in the intraoperative video data based on the intraoperative video data and at least one of the two-dimensional image data and the three-dimensional data;
Run the command,
The generating step includes:
Based on the estimation result of the estimation step, the positioning of organs included in the intraoperative video data and at least one of the two-dimensional image data and the three-dimensional data is performed to generate the simulation data.
program.

Images