JP7398232B2 - Medical image diagnostic equipment and event occurrence prediction method - Google Patents

Description

Embodiments of the present invention relate to a medical image diagnostic apparatus and an event occurrence prediction method.

Medical image diagnostic apparatuses may operate in cooperation with peripheral devices such as power supplies. For example, an on-vehicle X-ray diagnostic device is installed in a medical examination vehicle along with a power supply device and peripheral devices such as a barium shaker and a bucky device. In this case, the X-ray diagnostic apparatus is extremely convenient because the functions of the peripheral devices can be used even when it is inside the examination vehicle.

However, some peripheral devices do not have their own failure prediction function. In particular, in the case of small-scale peripheral equipment, it is extremely difficult to predict failures of peripheral equipment, even if the X-ray diagnostic apparatus itself has a failure prediction function. For this reason, the user is required to confirm whether there is a failure by visually inspecting the peripheral devices or performing a trial run at the time of initial inspection, which is cumbersome.

Further, in this type of confirmation, it is only possible to know whether a failure has occurred at the time of confirmation, and it is difficult to predict the occurrence of a failure event. For this reason, even peripheral devices that have no failure at the time of confirmation may fail after the X-ray diagnostic apparatus starts operating. In this case, the inspection must be stopped to restore the peripheral device, resulting in downtime.

In addition, when multiple peripheral devices are installed together with the device in a relatively narrow space such as in a medical examination vehicle, the components and cables of the devices may interfere or collide, or the components moved by the user may interfere with other components or other devices. There is a risk of interference or collision with the user. However, it is difficult to predict the occurrence of these near-miss events, and danger avoidance is often left to the attention of each user.

Special Publication No. 2015-526228

Christopher M. Bishop, Pattern recognition and machine learning (Part 1), (USA), 1st edition, Springer, 2006, P. 225-290

The problem to be solved by the present invention is to predict the occurrence of an event related to a peripheral device using at least one of a log of the device itself, a log of the peripheral device, and a log of the environment.

The medical image diagnostic apparatus according to the embodiment includes a prediction unit. The prediction unit generates a trained model that outputs a prediction result of an event related to a peripheral device based on a log of the device itself, a log of a peripheral device that operates in cooperation with the device, and a log of the environment. By inputting at least one of the log of the own device, the log of the peripheral device, and the log of the environment, the result of predicting the occurrence of an event related to the peripheral device is output.

FIG. 1 is a block diagram illustrating an example of a configuration of an on-vehicle medical examination system including a medical image diagnostic apparatus according to an embodiment. FIG. 3 is a diagram for explaining logs collected and temporarily stored by a log collection function. FIG. 7 is a diagram for explaining that when an event is a failure, a buffer log is saved using detection of an event occurrence as a trigger. FIG. 7 is a diagram for explaining a modified example of a buffer log and a saved log. FIG. 3 is an explanatory diagram illustrating an example of a data flow during learning of a prediction function for predicting the occurrence of a failure in a peripheral device. FIG. 3 is an explanatory diagram showing an example of a data flow during operation of a prediction function for predicting the occurrence of a failure in a peripheral device. FIG. 3 is a diagram for explaining that when the event is a near-miss, a buffer log is saved using detection of near-miss occurrence as a trigger;

Hereinafter, embodiments of a medical image diagnostic apparatus and an event occurrence prediction method will be described in detail with reference to the drawings.

As the medical image diagnostic device according to one embodiment, various devices such as an X-ray diagnostic device, an X-ray CT (Computed Tomography) device, and a Magnetic Resonance Imaging (MRI) device can be used. In the following description, an example will be described in which an on-vehicle medical image diagnosis apparatus that operates in cooperation with peripheral devices such as a power supply device is used as the medical image diagnosis apparatus according to the present invention.

FIG. 1 is a block diagram showing an example of the configuration of a vehicle-mounted medical examination system 1 including a medical image diagnostic apparatus 10 according to an embodiment.

The vehicle-mounted medical examination system 1 includes a vehicle-mounted medical image diagnostic apparatus 10, a peripheral device group 40, and a sensor group 45.

The medical image diagnostic apparatus 10 includes an imaging device 20 that performs medical imaging of a subject and a console 30. The imaging device 20 is configured to generate medical image data regarding the subject.

The console 30 generates and displays medical images based on medical image data. Console 30 has a display 31, an input interface 32, a storage circuit 33, a communication circuit 34, and a processing circuit 35.

The peripheral device group 40 may include various devices that operate in cooperation with the medical image diagnostic apparatus 10. In FIG. 1, a peripheral device group 40 includes a power supply device 41 as peripheral devices, a barium shaker 42, and a device for moving the grid at a constant speed to reduce the influence of the grid to prevent the incidence of scattered X-rays on the image. An example is shown in which a bookie 43 and a workstation 44 are included.

The sensor group 45 includes sensors depending on the type of log to be collected, such as an acceleration sensor, a temperature sensor, and a humidity sensor, and is used to collect logs of the peripheral devices 41-44 and environment logs.

The display 31 of the console 30 is configured with a general display output device such as a liquid crystal display or an OLED (Organic Light Emitting Diode) display, and displays various information such as medical images generated by the processing circuit 35 under the control of the processing circuit 35. indicate.

The input interface 32 may be a common one such as a trackball, switch, button, mouse, keyboard, touchpad that performs input operations by touching the operation surface, a non-contact input interface using an optical sensor, a voice input interface, etc. It is realized by an input device and outputs an operation input signal corresponding to the user's operation to the processing circuit 35.

The storage circuit 33 has a configuration including a recording medium readable by a processor, such as a magnetic or optical recording medium or a semiconductor memory. Part or all of the program and data in the storage medium of the storage circuit 33 may be downloaded by communication via an electronic network, or may be provided to the storage circuit 33 via a portable storage medium such as an optical disk. good.

The communication circuit 34 implements various information communication protocols depending on the connection form with the peripheral devices 41 to 44 forming the peripheral device group 40 and the connection form with each sensor forming the sensor group 45. The communication circuit 34 connects its own device and other electrical devices so as to be able to transmit and receive data according to these various protocols. The medical image diagnostic apparatus 10 may be connected to other devices via a network. In this case, the network refers to information and communication networks in general that utilize telecommunications technology, such as cloud networks, and includes wireless/wired LANs such as hospital backbone LANs (Local Area Networks), Internet networks, telephone communication lines, Includes fiber optic communications networks, cable communications networks and satellite communications networks.

The processing circuit 35 realizes a function of controlling the medical image diagnostic apparatus 10 in an integrated manner. Further, the processing circuit 35 reads out and executes the program stored in the storage circuit 33, thereby processing at least the log of the medical image diagnostic apparatus 10, the log of each peripheral device 41-44 of the peripheral device group 40, and the log of the environment. The occurrence of an event regarding each peripheral device 41-44 is predicted using one of the two.

The processor of the processing circuit 35 implements a log collection function 351, an event occurrence detection function 352, a log storage function 353, and a prediction function 354, as shown in FIG. Each of these functions is stored in the storage circuit 33 in the form of a program. Note that some of the functions 351 to 354 of the processing circuit 35 may be realized by an external processor connected to the console 30 via a network so as to be capable of transmitting and receiving data.

FIG. 2 is a diagram for explaining logs collected and temporarily stored by the log collection function 351.

The log collection function 351 collects logs that can be collected by the medical image diagnostic apparatus 10. Specifically, the log collection function 351 collects the log of the medical image diagnostic apparatus 10 itself and at least one of the peripheral devices 41 to 44 and the environment log for a buffer period set for each type of log. are temporarily stored in a storage medium such as the storage circuit 33 or another storage medium connected to the medical image diagnostic apparatus 10. In the following explanation, the temporarily stored log will be referred to as a buffer log. Note that the buffer period may be the same regardless of the type of log. The log collection function 351 is an example of a collection unit.

Although FIG. 2 shows an example where the log is time-series raw data, the log is not limited to time-series data. For example, if an event log is used as the log, information on the number of events may be held as the log. Furthermore, even if the original raw data is a time-series signal value, the time when the signal value exceeds a threshold value and discrete data of the signal value at that time may be retained as a log.

Logs that can be collected by the medical image diagnostic apparatus 10 include logs of the medical image diagnostic apparatus 10 itself, logs of peripheral devices 41-44, and environment logs. The log of the medical image diagnostic apparatus 10 itself may include information on the examination flow.

The logs of the peripheral devices 41-44 may be collected if each of the peripheral devices 41-44 has a log storage function and a log output function. On the other hand, if each of the peripheral devices 41-44 does not have a log output function, the logs of the peripheral devices 41-44 may be collected visually by the user, or if an analog signal output is obtained, the signal output Logs may be collected based on Alternatively, a sensor may be provided corresponding to the data to be collected for the log of the peripheral device, and the log of the peripheral device may be collected based on the output of the sensor.

The environment log may include a power environment log. Further, the environment log may include a surrounding environment log. The surrounding environment log may include vibrations, sounds, temperature inside the car, temperature outside the car, humidity, etc. A log of the surrounding environment can be acquired using, for example, the sensors that constitute the sensor group 45.

Further, when the medical image diagnostic apparatus 10 and the peripheral equipment group 40 are both installed in a medical examination car as in the example shown in FIG. 1, the environment log may include a log of the driving environment of the medical examination car. The driving environment log may include driving conditions, road surface conditions, and the like. The driving situation and road surface condition can be acquired using, for example, an acceleration sensor that constitutes the sensor group 45. Furthermore, although the driving environment log is not time-series data, it may include information that can contribute to the frequency of event occurrence, such as mileage and driver information.

FIG. 3 is a diagram for explaining that when an event is a failure, a buffer log is saved using the detection of an event occurrence as a trigger.

The event occurrence detection function 352 detects the occurrence of an event related to any of the peripheral devices 41-44, and provides information to that effect to the log storage function 353. Events include failures, near-misses, and anomaly detections by so-called anomaly detection techniques. Further, the event may further include a communication failure.

The event occurrence detection function 352 detects the occurrence of an event based on input by the user, or directly from the peripheral device 41 to 44 if the peripheral device is equipped with a function to output information about an event that has occurred on itself. Upon receiving the information, it is detected that an event related to any of the peripheral devices 41-44 has occurred.

For example, when the event is a failure and the failure occurrence is detected based on input by the user, the event occurrence detection function 352 generates and displays the failure occurrence reception screen 51 as shown in the upper part of FIG. It is preferable to display it on 31. If any of the peripheral devices 41-44 has a function to notify the occurrence of a failure, the event occurrence detection function 352 may detect the occurrence of a failure by receiving the notification from any of the peripheral devices 41-44. .

When the log storage function 353 receives information that an event related to any of the peripheral devices 41 to 44 has occurred, the log storage function 353 saves the medical image temporarily stored in the storage medium such as the storage circuit 33 at the current point in time when this information is received. At least one of the log of the diagnostic device 10, the log of the peripheral devices 41-44, and the log of the environment is stored in a storage medium such as the storage circuit 33 so that it can be used even after the buffer period has elapsed. In the following description, the log saved by the log saving function 353 will be referred to as a saved log 611. The log storage function 353 is an example of a storage section.

The saved log 611 saved by the log saving function 353 is stored as learning data for machine learning, and the information on the occurrence event corresponding to the saved log 611 is stored in association with each other as training data for machine learning.

Although FIG. 3 shows an example in which information on whether or not a failure has occurred is received, information may be collected by further dividing failures.

FIG. 4 is a diagram for explaining a modified example of the buffer log and storage log 611.

The end of the period for temporarily storing the buffer log does not have to be the present, but may be a predetermined period in the future from the present (see FIG. 4). Similarly, the saved log 611 may be a log from the time when information indicating that an event has occurred to a point in the future for a predetermined period of time. Furthermore, although FIG. 4 shows an example in which the predetermined period in the future is constant regardless of the log type, it may be made different for each log type.

The prediction function 354 is a trained model that outputs a prediction result of the occurrence of an event related to the peripheral devices 41-44 based on the log of the medical image diagnostic apparatus 10, the log of the peripheral devices 41-44, and the environment log. By inputting at least one of the log of the medical image diagnostic apparatus 10, the log of the peripheral devices 41-44, and the log of the environment, the result of predicting the occurrence of an event related to the peripheral devices 41-44 is output. As machine learning for constructing a trained model, deep learning using a multilayer neural network such as a CNN (convolutional neural network) or a convolutional deep belief network (CDBN) can be used.

Further, when the event occurrence prediction result is a prediction result that an event will occur, the prediction function 354 may notify the user of information that the event occurrence is predicted and information of the corresponding peripheral device. The notification method may be, for example, an image via the display 31 or a voice via a speaker (not shown). The prediction function 354 is an example of a prediction unit.

As mentioned above, events include failures, near misses, and anomaly detection by so-called anomaly detection techniques.

FIG. 5 is an explanatory diagram showing an example of a data flow during learning of the prediction function 354 for predicting the occurrence of a failure in a peripheral device. The prediction function 354 sequentially updates the parameter data 72 by receiving a large number of training data sets and performing deep learning. The training data set includes saved logs 611, 612, 613, etc. as learning data constituting a learning data group 61, and a teacher data group 62 corresponding to each learning data. It is composed of occurrence event information 621, 622, 623, . . . as corresponding teacher data.

In FIG. 5, "1" in the "flag" of the occurrence event information 621 means that a failure has occurred, and "0" means that no failure has occurred.

The prediction function 354 performs so-called learning, in which the prediction function 354 updates the parameter data 72 so that the result of processing the learning data by the neural network 71 approaches the teacher data every time a training data set is given. Generally, when the rate of change in the parameter data 72 converges within a threshold value, learning is determined to be complete. Hereinafter, the parameter data 72 after learning will be particularly referred to as learned parameter data 72t.

FIG. 6 is an explanatory diagram showing an example of a data flow during operation of the prediction function 354 for predicting the occurrence of a failure in a peripheral device. During operation, the prediction function 354 receives input of the buffer log 81 (see FIGS. 2 and 4) at a preset period or time, or at a timing instructed by the user, and uses the learned model 70 to A failure prediction result 82 for peripheral devices 41-44 is output. “1” in the “flag” of the failure occurrence prediction result 82 means that the prediction function 354 predicts that a failure will occur in the peripheral device corresponding to the flag, and “0” means that it is predicted that no failure will occur. It means what you did. Further, the prediction function 354 may output the reliability of failure occurrence prediction (also referred to as a prediction score, confidence score, or accuracy) in addition to or in place of the "flag" of the failure occurrence prediction result 82. Whether the flag is 1 or 0 is determined depending on whether the reliability of failure prediction exceeds a predetermined threshold.

Note that the neural network 71 and the learned parameter data 72t constitute a learned model 70. Various methods are known for this type of learning method and method for constructing a trained model, such as the method disclosed in Non-Patent Document 1. The neural network 71 is stored in the storage circuit 33 in the form of a program. The learned parameter data 72t may be stored in the storage circuit 33 or in a storage medium connected to the processing circuit 35 via a network. When the trained model 70 (neural network 71 and trained parameter data 72t) is stored in the storage circuit 33, the prediction function 354 realized by the processor of the processing circuit 35 reads the trained model 70 from the storage circuit 33 and By executing this, a failure prediction result 82 for the peripheral devices 41-44 is output.

Note that the trained model 70 may be constructed using an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array).

Further, in the above example, an example of the trained model 70 that can predict the occurrence of events related to all of the peripheral devices 41 to 44 constituting the peripheral device group 40 was shown, but it is possible to prepare one trained model for each peripheral device. Alternatively, one may be provided for any set of peripheral devices.

The trained model 70 in the case where the event is a failure has been explained with reference to FIGS. good.

FIG. 7 is a diagram for explaining that when the event is a near-miss, a buffer log is saved using the near-miss occurrence detection as a trigger. Examples of near misses include cases where a member is about to get caught up in another member, a member is about to collide with another member, and the subject is about to fall.

When the event is a near miss, the occurrence of the near miss may be detected by image analysis of visible light camera images or audio analysis such as collision sounds, as long as the event is visually distinguishable, such as a collision. Further, if the event is a near-miss and the near-miss occurrence is detected based on input by the user, the event occurrence detection function 352 generates a near-miss occurrence reception screen 91 as shown in an example in the upper part of FIG. It is preferable to display it on the display 31.

If the event is a near-miss, the medical image diagnostic apparatus 10 collects the cumulative values such as the layout of each component of the in-vehicle medical examination system 1, the period since introduction of each component, and the number of times it has been used as a saved log 611. It's okay. The layout includes, for example, the position of the top plate of the photographing device 20, the position of the high voltage generator, and the like.

Even when the event is a near-miss, when the log storage function 353 receives information that a near-miss has occurred regarding any of the peripheral devices 41 to 44, the log storage function 353 saves the buffer log as a storage log 92 in a storage medium such as the storage circuit 33. let The saved log 92 saved by the log saving function 353 is stored as learning data for machine learning, and the information on near-miss occurrences corresponding to the saved log 92 is stored in association with each other as training data for machine learning. Then, a learning model for outputting near-miss occurrence prediction results is constructed and operated using the same procedure as the example shown in FIGS. 5 and 6.

In addition, if the event is a near-miss, the logs that may be related to the occurrence of the near-miss include the log of a photographing device such as a video camera that photographs a predetermined area including peripheral devices, and the log of audio generated in a predetermined area including peripheral devices. may also be included (see middle row of FIG. 7). Possible sources of sound include the sound of collisions between members, the user's screams, and the like.

In addition, when the event is an abnormality detection, for example, when an input of "no problem" is accepted on the failure occurrence reception screen 51, or when the device starts normal shutdown without user input indicating a failure, the buffer log It is preferable to save this as a saved log 611 and to associate and accumulate information indicating that the data is normal as teaching data corresponding to this saved log 611. By constructing a trained model using the training data set accumulated in this way, it is possible to detect that the normality of the peripheral device is the same as the normality, and therefore it is possible to detect an abnormality when it deviates from normality. You can get a trained model that can detect and notify the user.

According to the medical image diagnostic apparatus 10 according to the present embodiment, at least one of the logs of the peripheral device group 40 and the environment logs are collected and managed centrally, and the peripheral device 41- It is possible to predict the occurrence of 44 events. Therefore, even if the peripheral device does not have a failure prediction function, the medical image diagnostic apparatus 10 can predict the failure occurrence of the peripheral device.

Therefore, downtime due to sudden failure of peripheral equipment during normal operation can be prevented. Furthermore, the importance of visually inspecting peripheral devices during start-up inspections and checking for failures through test runs can be significantly reduced. In particular, if normality can be detected, the prediction function 354 may notify the user of the prediction result that it is normal. In this case, the user can feel at ease based on the notification that the system is normal and can omit the confirmation work at the time of the start-up inspection.

Further, the buffer period of the buffer log may be any period necessary for predicting the occurrence of an event using a trained model, and there is no need to semi-permanently store it for a long period of time such as several years.

Furthermore, the medical image diagnostic apparatus 10 can predict near misses. The predicted near-miss occurrence is output and notified by a trained model built based on logs accumulated by users who frequently use the system. Therefore, even users who are using the device for the first time and who do not fully understand the positional relationship with peripheral devices can use the device safely by being notified of the predicted near-miss occurrences. Can be done.

According to at least one embodiment described above, it is possible to predict the occurrence of an event related to a peripheral device using at least one of the log of the device itself, the log of the peripheral device, and the log of the environment.

In the above embodiments, the term "processor" refers to, for example, a dedicated or general-purpose CPU (Central Processing Unit), GPU (Graphics Processing Unit), or Application Specific Integrated Circuit (ASIC). It shall refer to circuits such as programmable logic devices (eg, Simple Programmable Logic Devices (SPLDs), Complex Programmable Logic Devices (CPLDs), and FPGAs). A processor implements various functions by reading and executing programs stored in a storage medium.

Furthermore, in the above embodiment, an example is shown in which a single processor of the processing circuit realizes each function, but a processing circuit may be configured by combining multiple independent processors, and each processor realizes each function. Good too. Furthermore, when multiple processors are provided, a storage medium for storing programs may be provided individually for each processor, or a single storage medium may collectively store programs corresponding to the functions of all processors. Good too.

Although several embodiments of the present invention have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These embodiments can be implemented in various other forms, and various omissions, substitutions, and changes can be made without departing from the gist of the invention. These embodiments and their modifications are included within the scope and gist of the invention as well as within the scope of the invention described in the claims and its equivalents.

1 In-vehicle medical examination system 10 Medical image diagnosis device 20 Imaging device 30 Console 31 Display 32 Input interface 33 Memory circuit 35 Processing circuit 40 Peripheral device group 41-44 Peripheral device 51 Failure reception screen 61 Learning data group 62 Teacher data group 81 Buffer Log 82 Failure prediction result 91 Near miss occurrence reception screen 92 Save log 351 Log collection function 352 Event detection function 353 Log storage function 354 Prediction function

Claims (10)

For a trained model that outputs a prediction result of the occurrence of an event related to the peripheral device based on a log of the own device, a log of a peripheral device that operates in cooperation with the own device, and a log of the environment. a prediction unit that outputs a prediction result of occurrence of the event related to the peripheral device by inputting at least one of the log of the own device, the log of the peripheral device, and the log of the environment;
A medical imaging diagnostic device equipped with The prediction unit is
The log of the own device and the log of the peripheral device are used for a trained model that outputs a failure prediction result of the peripheral device based on at least one of the log of the own device, the log of the peripheral device, and the log of the environment. outputting a failure prediction result of the peripheral device by inputting at least one of a log and a log of the environment;
The medical image diagnostic apparatus according to claim 1. The log for the above environment is
including at least one of a log of the surrounding environment, a log of the power supply environment, and a log of the driving environment of the examination vehicle when both the own device and the peripheral device are provided in the examination vehicle;
The medical image diagnostic apparatus according to claim 2. The prediction unit is
Notifying a user of information that a failure is predicted to occur in the peripheral device in accordance with a prediction result of failure occurrence in the peripheral device;
The medical image diagnostic apparatus according to claim 2 or 3. The prediction unit is
The log of the own device and the surroundings are used for a trained model that outputs a prediction result of occurrence of a near-miss involving the peripheral device based on at least one of the log of the own device, the log of the peripheral device, and the log of the environment. outputting a near-miss occurrence prediction result involving the peripheral device by inputting at least one of a device log and the environment log;
The medical image diagnostic apparatus according to claim 1. The log for the above environment is
A log of the surrounding environment, a log of the driving environment of the medical examination vehicle if both the own device and the peripheral device are installed in the medical examination vehicle, a log of the photographing device that photographs a predetermined area including the peripheral device, and the peripheral device. a log of audio occurring in a predetermined area;
The medical image diagnostic apparatus according to claim 5. a collection unit that temporarily stores each of the log of the own device and at least one of the log of the peripheral device and the log of the environment in a storage unit for a period set for each type of log;
When receiving information that the event related to the peripheral device has occurred, at least the log of the own device, the log of the peripheral device, and the log of the environment that are temporarily stored in the storage unit at the time when this information is received. a storage unit that stores the one in a storage unit so that it can be used even after the expiration of the period;
Furthermore,
The received information that the event regarding the peripheral device has occurred is used as training data, and at least one of the log of the own device, the log of the peripheral device, and the log of the environment stored by the storage unit is used as learning data. are used for learning the trained model, respectively.
A medical image diagnostic apparatus according to any one of claims 1 to 6. The trained model is
A training data set consisting of learning data and teacher data, in which received information indicating that the event regarding the peripheral device has occurred is used as the teacher data, and the log of the own device stored by the storage unit and the peripheral device are used as the training data set. The model is trained using a training data set in which at least one of a device log and an environment log is used as the learning data.
The medical image diagnostic apparatus according to claim 7. At least one of the log of the own device, the log of the peripheral device, and the log of the environment used by the trained model,
at least one of the log of the own device, the log of the peripheral device, and the log of the environment, which are temporarily stored in the storage unit by the collection unit at the current time;
The medical image diagnostic apparatus according to claim 7 or 8. Using a trained model that outputs a prediction result of an event related to the peripheral device based on a log of the device itself, a log of a peripheral device operating in cooperation with the device, and an environment log. An event occurrence prediction method used in a medical image diagnostic apparatus equipped with a prediction unit , the method comprising:
acquiring at least one of a log of the own device, a log of the peripheral device operating in cooperation with the own device, and a log of the environment;
The prediction unit applies the self -device to the learned model that outputs a prediction result of the occurrence of an event related to the peripheral device based on at least one of a log of the self-device, a log of the peripheral device, and a log of the environment. outputting a result of predicting the occurrence of the event regarding the peripheral device by inputting the log of the peripheral device and at least one of the log of the peripheral device and the log of the environment;
A method for predicting the occurrence of an event.

Images