JP7783766B2 - Medical image processing system and method of operation - Google Patents

Description

The present invention relates to a medical image processing system and its operating method.

In medical devices such as endoscopes, it is common for systems to automatically detect equipment failures and other anomalies and save them as error logs. Error logs record the type of anomaly, the date and time of occurrence, as well as the device settings, program processing status, and communication status between modules at the time the anomaly occurred. Error logs make it possible to determine the fact that an anomaly such as a failure occurred and the circumstances under which it occurred. However, when detecting failures using error logs, it is difficult to identify the specific nature of the failure or its cause. Therefore, it is necessary to understand the specific nature of the failure using data other than error logs.

Specifically, in Patent Document 1, multiple status signals that indicate the operating state of the equipment are inspected. When a status signal is detected to be outside of an acceptable range, the time of detection, the time elapsed since startup, and the subsequent state are recorded. The type of abnormality is also indicated by a light, sound, etc. In Patent Document 2, an abnormality is detected from data indicating the operating state of the equipment, and the operating state is recorded along with the type of abnormality detected. A determination is made as to whether the abnormality has been confirmed as a malfunction, and if an abnormality that has not been confirmed as a malfunction is detected, the recorded data is erased.

Japanese Patent Application Publication No. 5-5633 Japanese Patent Application Laid-Open No. 2003-57076

Patent Document 1 detects abnormalities using equipment status signals, while Patent Document 2 determines whether the abnormality is one that occurs temporarily during extremely high loads, etc., and is likely to self-recover over time, or one that is not. Patent Documents 1 and 2 acquire signals from the equipment's operating status, detect, record, and determine the type of abnormality, and determine the type of abnormality. While it may be possible to determine that the abnormality is not due to a malfunction, such as a temporary high load caused by external noise, it is difficult to determine that it is a malfunction or identify its cause.

Furthermore, with devices that are limited in size, such as endoscopes, it is not possible to install dedicated sensors for detecting faults, or such installations are costly. Furthermore, analyzing all of the data obtained from inspection results requires time and effort from the user, making it inefficient. In addition, the data storage capacity of endoscope systems is not always sufficient, making constant storage inefficient. Therefore, there is a need to efficiently confirm the nature of the fault and identify the cause when detecting faults from inspection results obtained from standard inspection equipment and recording them on external devices.

The present invention aims to provide a medical image processing system and its operating method that can detect abnormalities in equipment due to failures without using dedicated sensors to detect failures, and can efficiently analyze the cause of the failure by recording the detection results for each failure in equipment external to the endoscope system.

The medical image processing system of the present invention comprises an external device having a processor and an endoscope system. The processor acquires real-time medical video from the endoscope system during an endoscopic examination, receives a trigger signal from the endoscope system containing information about a failure that has been created based on a failure scene in the medical video, temporarily stores the medical video, and extracts failure videos and failure still images containing the failure scene from the temporarily stored medical video based on the timing of receiving the trigger signal, and stores the failure videos and failure still images in the main memory area of the external device.

It is preferable to acquire medical video with an embedded trigger signal from the endoscope system and extract failure video and failure still images from the medical video based on the trigger signal.

It is preferable to create a fault still image upon fault detection and acquire the fault still image from the endoscope system.

It is preferable to obtain a second medical video from the endoscope system that freezes the video of the malfunction for a certain period of time, temporarily store the second medical video, and extract a still image of the malfunction from the temporarily stored second medical video based on the timing of receiving the trigger signal.

It is preferable that the endoscope system receives a second trigger signal created by a freeze operation by the user, and extracts a video and still image of the failure based on the second trigger signal.

It is preferable to process the failure still images and failure videos extracted in response to the second trigger signal so that they can be visually distinguished from the failure still images and failure videos extracted in response to the trigger signal.

Before the endoscopy begins, it is preferable to set a first extraction range that extracts the timing at which the trigger signal is received and the scenes before and after that timing from the medical video.

In the first extraction range, a first limit period is set at the timing of receiving the trigger signal, which continues at least until the recording range of the fault video is confirmed, and if a trigger signal indicating the same type of fault is received during the first limit period, it is preferable not to start new extraction but to only acquire fault information.

Before the endoscopy begins, it is preferable to set a second extraction range that contains information on the occurrence of the same type of fault and extracts a range of continuous trigger signals from the medical video.

Before the endoscopic examination begins, it is preferable to set a third extraction range in place of the second extraction range, which extracts the second extraction range and scenes before and after the second extraction range from the medical video.

For the second or third extraction range, it is preferable to set a second limit period at the timing of receiving a trigger signal that continues at least until the recording range of the failure video is confirmed, and if a trigger signal indicating the same type of failure is received during the second limit period, not to start new extraction but to accept the acquisition of failure information and the extension of the extraction range.

It is preferable to determine the recording range of the failure video within the second or third extraction range after a certain period of time has elapsed.

It is recommended to set a limit on the number of times extraction can be performed.

When acquiring failure videos and failure still images, it is preferable to process the personal information contained in the image information contained in the medical video so that it cannot be deleted or viewed.

The operating method of the medical image processing system of the present invention includes an external device having a processor and an endoscope system, and includes the steps of acquiring real-time medical video from the endoscope system during an endoscopic examination, acquiring a trigger signal from the endoscope system containing information about a failure that has been created based on a failure scene in the medical video, temporarily saving the medical video, extracting a failure video and a failure still image containing the failure scene from the temporarily saved medical video based on the timing of receiving the trigger signal, and saving the failure video and failure still image in a main memory area.

This invention makes it possible to detect equipment abnormalities caused by failures without using dedicated sensors to detect failures, and to efficiently analyze the cause of the failure by recording the detection results for each failure in an external device.

FIG. 1 is an explanatory diagram showing the configuration of a medical image processing system. FIG. 2 is a functional block diagram of a processor device. FIG. 2 is a block diagram showing the functions of the medical image processing apparatus. 10A and 10B are explanatory diagrams of a medical video on which a fault detection process has been performed. 10A, 10B, and 10C are explanatory diagrams of frame images detected in the fault detection process. FIG. 2 is an explanatory diagram of data transmitted from a processor device to a medical image processing device. FIG. 10 is an image diagram showing a playback screen of a medical video. FIG. 10 is an explanatory diagram illustrating extraction of a failure video in a first extraction range in response to a trigger signal. FIG. 10 is an explanatory diagram illustrating extraction of a failure video in a second extraction range in response to a trigger signal. FIG. 10 is an explanatory diagram illustrating extraction of a failure video in a third extraction range in response to a trigger signal. FIG. 10 is an image diagram showing a video of a failure displayed on a screen. FIG. 10 is an image diagram showing a fault video and a fault still image displayed on the screen. FIG. 10 is an explanatory diagram illustrating a case where extracted failure videos are stored without being selected. FIG. 10 is an explanatory diagram illustrating a case where extracted failure videos are selected and saved. FIG. 10 is an image diagram showing a thumbnail display of a video of a failure. 1 is a flowchart showing a series of processing steps according to the present invention. FIG. 11 is an explanatory diagram illustrating temporary storage of a failure video in a nonvolatile memory in the second embodiment. FIG. 11 is an explanatory diagram of data transmitted from the processor device to the medical image processing device in the third embodiment. FIG. 10 is an explanatory diagram of data transmitted from the processor device to the medical image processing device in the fourth embodiment.

[First embodiment]
FIG. 1 is a diagram showing the configuration of a medical image processing system 10 including an endoscope system 11 according to an embodiment of the present invention. The medical image processing system 10 includes an endoscope system 11, a medical image processing device 17, a display 18, and a user interface (UI) 19. The endoscope system 11 includes an endoscope 12, a light source device 13, a processor device 14, a display 15, and a user interface (UI) 16. The processor device 14 is electrically connected to the light source device 13, the display 15, the user interface 16, and the medical image processing device 17. The medical image processing device 17 is electrically connected to the display 18 and the user interface 19. The processor device 14 and the medical image processing device 17 have programs related to processes such as image processing stored in a program memory (not shown). The medical image processing device 17 is an external device not included in the endoscope system 11 that captures medical videos through endoscopic examinations.

The endoscope 12 has a long, slender shape and includes an insertion section 12a that is inserted into the patient's body to acquire images, and an operation section 12b that accepts user operations such as bending the insertion section 12a and zooming when acquiring images, as well as a freeze operation to acquire still images. The freeze operation is achieved by pressing a freeze button 12d on the operation section 12b. The tip of the insertion section 12a is also equipped with a tip section 12c that has a photographing function and is used to irradiate illumination light and project treatment tools.

The light source device 13 is optically connected to the endoscope 12 and supplies illumination light to the endoscope 12 during endoscopic examination. The display 15 displays images acquired by the processor device 14. The user interface 16 is an input device for inputting data to the processor device 14, and may use a keyboard, mouse, foot pedal, gesture recognizer, voice recognizer, etc. Input may also be made using input means provided on the medical device, such as a switch on the endoscope 12, rather than the user interface 16. Note that images include videos, still images, and frame images that make up videos.

Unless otherwise specified, white light should be used as illumination when capturing images during an endoscopic examination, and a video signal of 60 frames per second (60 fps) should be acquired and the capture time recorded. Furthermore, if the video signal is 60 fps, it is preferable to count the time in units of 1/100th of a second.

The medical image processing device 17 is a device capable of sending and receiving data to and from the processor device 14, and receives medical video and failure information data. The medical image processing device 17 temporarily stores the received medical video, and extracts and stores failure videos using the failure information. It is preferable to use an internal SSD (Solid State Drive) for storage. Instead of an SSD, recording media such as USB (Universal Serial Bus) memory or HDD (Hard Disc Drive) may also be used.

As shown in FIG. 2, in the processor device 14, a central control unit (not shown) composed of an image control processor runs programs in program memory, thereby realizing the functions of an image acquisition unit 21, an input receiving unit 22, an output control unit 23, and a fault identification unit 25. Furthermore, with the realization of the function of the fault identification unit 25, the functions of a fault determination unit 26, a trigger signal creation unit 27, and a still image creation unit 28 are also realized.

The processor device 14 acquires medical videos and still images captured in real time by the endoscope 12, performs fault detection processing, and displays the images on the display 15. It also transmits a trigger signal containing fault information, such as the type of fault detected by the fault detection processing, to the medical image processing device 17.

The image acquisition unit 21 receives data such as medical videos captured by the endoscope 12. The acquired medical videos are sent to the output control unit 23 and the fault identification unit 25. The input receiving unit 22 is connected to the user interface 16. The output control unit 23 controls the display of the medical videos on the display 15 and the transmission of the medical videos to the medical image processing device 17.

As shown in FIG. 3, in the medical image processing device 17, a central control unit (not shown) consisting of an image control processor runs programs in the program memory, thereby realizing the functions of a data acquisition unit 31, input receiving unit 32, output control unit 33, main memory area 34, and failure video extraction unit 35. Furthermore, with the realization of the function of the failure video extraction unit 35, the functions of an extraction range setting unit 36, mode control unit 37, temporary storage unit 38, extraction unit 39, and selection unit 41 are also realized. Furthermore, the extraction unit 39 includes the functions of an image information management unit 40, and the selection unit 41 includes the functions of a temporary storage unit for selection 42 and importance calculation unit 43.

The following describes the fault detection process and trigger signal generation performed by the processor device 14. The fault determination unit 26 performs fault detection processing on the medical video captured by the endoscope 12, detecting whether or not there is a fault in the endoscope 12 and determining the type of fault. The trigger signal generation unit 27 generates a trigger signal that contains information about frame images with faults determined by the fault determination unit 26 and fault information, and has a trigger function that causes the medical image processing device 17 to start extraction processing.

The fault determination unit 26 performs fault detection processing in real time during the endoscopic examination. When the endoscopic examination begins, the image acquisition unit 21 receives one or a small number of frame images and inputs them to the fault identification unit 25. The frame images input to the fault identification unit 25 are sequentially subjected to fault detection processing by the fault determination unit 26, and fault scenes are detected.

As shown in Figure 4, the fault detection process for a temporally continuous medical video 50 captured inside a living body will be described. The fault determination unit 26 classifies the type of each frame that makes up the medical video 50 using the fault detection process. Figure 4 (A) shows a portion of the temporally continuous frame image group that makes up the medical video 50, and the fault detection process sorts them into one of three types: normal frame image 51 with no faults, fault frame image 52 with an abnormality due to a fault, and fault-free abnormal frame image 53 with an abnormality but not a fault. After sorting, information about the fault frame image 52 is sent to the medical image processing device 17 as a trigger signal. Note that the medical video 50 acquired by the processor device 14 is not a video file, but a group of frame images being captured in real time.

Figure 4(B) shows the same group of frame images as Figure 4(A), with each rectangle representing a single frame image. The fault detection process identifies a fault scene consisting of multiple fault frame images 52 representing the same fault in the medical video 50. That is, consecutive fault frame images 52 with the same type of fault and the same image characteristics are identified as a single fault scene. In Figure 4(B), three consecutive frames are considered a fault scene, but an actual fault scene typically consists of dozens of frames, making it a huge number. A single, non-consecutive fault frame image 52 can also be identified as a fault scene. It is also possible to determine that a single fault frame image 52 is a single fault scene if it is not completely consecutive, but is detected with a frequency of at least a certain percentage, for example, 80% or more. The fault video extraction process, described below, is performed for each fault scene. The first fault frame image 52 in a fault scene is the trigger signal reception timing, and serves as the starting point for the extraction process.

The fault detection process involves detecting faulty frame images 52 from the frame images in the acquired medical video 50, and then performing fault judgment on the faulty frame images 52 to determine the type of fault, such as "disconnection," "scope contamination," or "lens damage." The judgment result is sent to the trigger signal creation unit 27.

As shown in Figure 5, the frame images classified by the fault detection process each have different characteristics. As shown in Figure 5(A), normal frame image 51 is a frame image in which no abnormality was detected. As shown in Figure 5(B), fault frame image 52 is a frame image in which a fault of the endoscope 12 was detected, such as a state in which a "break" causes a break C, which is a vertical or horizontal line running through the image, or a state in which a portion of the image is filled in regardless of the observation target due to "scope dirt," and a non-lesion abnormal region T is included. Furthermore, as shown in Figure 5(C), fault-free abnormal frame image 53 is a frame image that is abnormal but fault-free, such as a state in which part or all of the image is not visible due to "external noise," or a state in which a lesion region R is visible in the image as a severe "lesion." Fault-free abnormal frame image 53 is treated in the same way as normal frame image 51 during the extraction process.

The fault detection process is performed, for example, using the functions of the trained model required for the fault detection process. That is, the fault determination unit 26 has a computer algorithm consisting of a neural network that performs machine learning, and performs specific inferences to detect the presence or absence of a fault for each frame image of the input medical video 50 according to the learning content, and to determine the type of fault if a fault is present. In addition to the frame image classification results and type of fault, it is preferable to obtain information such as the degree of match with the previously learned learning content as the inference results.

The trigger signal creation unit 27 creates failure information and a trigger signal with a trigger function for each detected failure frame image 52. The failure information includes information such as the type of failure in the failure frame image 52, the scale of the failure, and the failure scene formed by consecutive failure frame images 52, and the trigger function is a trigger that starts the process of extracting a failure video from the medical video 50 in the medical image processing device 17. Because the failure detection process and trigger signal creation are performed in real time, the trigger signal is created in conjunction with the detection of the failure frame image 52. The created trigger signal is sent to the medical image processing device 17 via the output control unit 23.

The still image creation unit 28 creates a failure still image, which is a still image of the failure frame image 52 detected by the failure detection process. The failure still image is preferably created simultaneously with the trigger signal. The created failure still image is transmitted to the medical image processing device 17 via the output control unit 23. Note that only the first failure still image may be created from failure frame images 52 that are consecutive and of the same type of failure.

As shown in FIG. 6 , the medical video 50 is transmitted to the display 15, and the medical video 50 and trigger signal are transmitted to the medical image processing device 17 via the output control unit 23 of the processor device 14. Data transmission may be wired or wireless. In the wired case, the medical video 50 may be transmitted using a DVI (Digital Visual Interface) as the video transmission line, and the trigger signal may be transmitted using RS232c (Recommended Standard 232 version C) as the trigger transmission line, or the medical video 50 and trigger signal may be transmitted to the medical image processing device 17 using a USB cable. The data transmitted to the medical image processing device 17 is received by the data acquisition unit 31. Note that the medical video 50 transmitted to the display 15 and the medical image processing device 17 is the same, so it may be transmitted to the medical image processing device 17 via the display 15.

Since the medical video 50 is captured in real time, data is constantly transmitted, but the trigger signal is only created and transmitted when a failure is detected. Because the medical video 50 transmitted to the medical image processing device 17 does not contain failure information, the medical image processing device 17 compares it with the trigger signal to identify the failure frame image 52.

As shown in FIG. 7, the display 15 that constitutes the endoscope system 11 plays back the medical video 50 and displays an image display area 60 where the captured content can be confirmed, and an image information display field 61 where image information of the medical video 50 being played back is displayed. The image display area 60 performs real-time playback during an endoscopic examination, and the image information displayed in the image information display field 61 includes, for example, a patient ID 62, patient name 63, endoscope type, and operation history. In FIG. 7, the patient ID 62 is "□□□□", the patient name is "△△△△", and the endoscope type is "upper gastrointestinal endoscope". Furthermore, the operation history is preferably displayed in reverse chronological order, displaying the history of operations performed on the operation unit 12b of the endoscope 12. For example, it is preferable to display the most recent operation as the "bend" operation, the second most recent operation as the "zoom" operation, and the third most recent operation as the "freeze" operation.

The medical image processing device 17, an external device to the endoscope system 11, extracts and saves failure videos based on the medical videos 50 and trigger signals received in real time from the processor device 14. The medical image processing device 17 receives real-time medical videos 50 from the endoscope system 11 during an endoscopic examination, temporarily stores the acquired medical videos 50 in the temporary storage unit 38 of the medical image processing device, and receives a trigger signal created by the endoscope system containing a trigger function that starts extraction and failure information created based on the failure scene. Based on the received trigger signal, the extraction unit 39 extracts failure videos containing failure scenes from the temporarily saved medical videos 50. The extracted failure videos are saved in the main memory area 34. The medical image processing device 17 also acquires failure still images created by the still image creation unit 28.

Because the trigger signal is received asynchronously with the medical video 50, there is an error in the timing between the trigger signal and the acquisition by the medical image processing device 17 of the failure frame image 52 corresponding to the trigger signal. Because the trigger signal is created based on information detected from the medical video 50, it is likely to be delayed compared to the corresponding frame of the medical video 50. Therefore, it is preferable to perform extraction including this error.

The extraction range setting unit 36 sets the extraction range for extracting the failure video before the endoscopic examination begins. In response to receiving a trigger signal, the extraction unit 39 executes the extraction process according to one of the set first to third extraction ranges. The start timing of extraction in the trigger function is determined by information on the reception timing of the trigger signal in the medical image processing device 17, and it is preferable to extract the failure video, including scenes before and after the trigger signal, from the medical video 50 in chronological order based on the reception timing of the trigger signal.

The extraction process is performed based on the extraction range set by the extraction range setting unit 36. The first extraction range extracts a group of consecutive frame images within a range of n seconds before and after the trigger signal reception timing as scenes before and after the trigger signal. The second extraction range extracts a range in which trigger signals of the same type of fault occur consecutively. The third extraction range extracts consecutive frame images within a certain range of n seconds before and after the second extraction range. It is preferable to set the first extraction range when standardizing or shortening the recording time of fault videos, the second extraction range when comparing a large number of fault videos, and the third extraction range when comparing fault scenes with normal scenes.

The fixed period of n seconds before and after the first extraction range and the third extraction range is preferably, for example, 3 seconds (n=3) or 5 seconds (n=5), which allows for comparison with normal frame images 51 before and after the occurrence of a fault, without placing a strain on storage. It is also preferable to determine the extraction range of normal frame images 51 based on the type of fault, the duration of the fault, and the average area of the abnormal area caused by the fault in the frame image, as determined by the fault detection process. For example, for fault scenes with a long duration or where the type of fault is a "disconnection," the value of n seconds, i.e., the range of fault videos to be extracted, is increased.

For the second and third extraction ranges, the upper limit of the recording range for each failure video is set in advance. In cases where the failure is not temporary and will not self-recover, or where self-recovery takes time, the recording range for the failure video is determined after a certain period of time has passed. For example, if a failure scene is detected for 5 seconds, that point in time will be the end of the failure video to be extracted. It does not have to be 5 seconds, but could be 10 seconds or 30 seconds.

The mode control unit 37 controls the extraction and storage of failure videos. The mode control unit 37 is set to normal mode or storage mode, and in storage mode, it further classifies the mode into one of first to fourth storage modes depending on the conditions and method used to extract and store failure videos from the medical video 50. The storage mode is preferably set in advance before the endoscopic examination. In normal mode, the acquired medical video 50 is displayed in real time on the display 15 or display 18 and stored in the main memory area 34 of the medical image processing device 17.

In the first storage mode, the extracted failure videos are not selected and are all saved in the main memory area 34. In the second to fourth storage modes, the extracted failure videos are sent to the selection unit 41, temporarily saved in the temporary memory area via the temporary storage unit for selection 42, and the failure videos to be saved in the main memory area 34 are selected. In the second storage mode, selection is based on the type of failure and the order of failure detection, in the third storage mode, selection is based on the type of failure and the severity of the failure, and in the fourth storage mode, selection is based on user selection. Details of extraction and selection in each storage mode will be described later.

The temporary storage unit 38 stores the real-time images of the medical video 50 transmitted from the processor device 14 in a temporary storage area. The temporary storage area is formed in volatile or non-volatile memory, and if it is non-volatile memory, it may be physically the same as the main memory area 34. Note that the medical video 50, which records the entire medical examination, may only be stored temporarily and does not need to be stored in the main memory area 34.

The extraction unit 39 receives the trigger signal received from the data acquisition unit 31 and acquires from the temporary storage unit 38 a group of frame images to be extracted as a failure video based on the trigger signal. It matches the frame images corresponding to the failure information contained in the trigger signal, i.e., identifies the failure frame images 52 in the temporarily stored medical video 50. It extracts the failure video from the medical video 50 in accordance with the trigger signal and the extraction range settings. It is preferable that the failure video retains trigger signal information.

Since the trigger signal is transmitted each time a failure frame image 52 is detected, a large number of trigger signals are received for each failure scene. Creating a failure video for each failure frame image 52 in the same failure scene, i.e., acquiring failure videos of the same failure scene, does not need to be duplicated; if it is acquired, the amount of extraction processing will be enormous, putting pressure on storage capacity. Therefore, when starting to extract failure videos, it is preferable to limit the start of extraction for the same failure scene.

During the extraction process, a certain period of time is set as a limit for the trigger function, which does not start a new extraction process after receiving a trigger signal. If a trigger signal is received during the limit period, the trigger function is not activated, and the failure information of the failure frame image 52 is received and compared with the medical video 50. Extraction of the failure video using the trigger function is activated again after the limit period ends. The limit period begins immediately after receiving the trigger signal and ends at the same time as the end of the extraction range or a certain number of seconds after the end.

As shown in Figure 8, in the extraction process for the first extraction range, a first limit period is set as the limit period for the trigger function. During the first limit period, only failure information is acquired from the received trigger signal. The first limit period ends n seconds after the timing of receiving the trigger signal, which is the end of the first extraction range, or after the end of the extraction range.

As shown in Figures 9 and 10, in the extraction process for the second and third extraction ranges, a second limit period is set as the limit period for the trigger function. During the second limit period, new extraction is not initiated for the same type of fault, but fault information is obtained from the trigger signal, and the extraction range is extended when consecutive trigger signals of the same type are received. The second limit period ends simultaneously with or after the extraction range is determined. The longer consecutive trigger signals of the same type are received, the longer the second limit period becomes.

The image information management unit 40 manages the image information contained in the medical video 50 and the fault information obtained from the trigger signal. The fault information is information relating to the state of the fault, such as the non-lesion abnormal region T in the fault frame image 52, and is linked to the extracted fault video 54. The fault information includes, for example, the timing of receiving the trigger signal, the type of fault, the area of the non-lesion abnormal region T caused by the fault, and the duration of the fault scene.

In addition to extracting the failure video, the extraction process also edits image information other than the failure information. The failure video 54 is used to analyze the failure of the endoscope 12, but the analysis is performed by a service technician from the medical device manufacturer, not the user of the endoscope 12. For this reason, it is necessary to ensure that the patient information contained in the image information of the medical video 50 cannot be obtained from the failure video 54. During the extraction process, the image information management unit 40 processes patient information such as the patient ID 62 and patient name 63 recorded in the medical video 50 so that it cannot be deleted or viewed. Processing includes masking, such as filling in text, and pixelating text.

Even if the user who performed the endoscopic examination views or analyzes the failure video 54, if the data is created to be handled outside the hospital network, patient information is not retained. Furthermore, if the failure frame image 52 contains a lesion area R, disease information such as the name of the detected lesion is also treated as patient information. For this reason, the lesion information is also deleted or processed so that it cannot be viewed during the extraction process.

When a large number of failure scenes are detected from medical videos 50, the extraction and sorting processes take time, and it takes time to acquire and confirm failure videos 54. For this reason, it is preferable to set an upper limit on the number of times failure videos 54 can be extracted. If an upper limit is set, extraction stops when a set number of failures, such as five or ten, is detected. For example, if the upper limit is set to five, extraction will not be performed even if a failure frame image 52 is detected after a failure video 54 has been extracted. After the upper limit is reached, specific methods include setting the trigger function limit period to unlimited so that trigger signals themselves are not accepted, or switching from storage mode to normal mode in the mode control unit to end the failure detection process. Additionally, an upper limit may be set for each type of failure. Alternatively, the upper limit may be set to one time and a determination may be made as to whether or not the endoscope 12 has a failure.

When the storage mode is the first storage mode, selection is not performed, and the failure videos 54 extracted by the extraction unit 39 are stored in the main memory area 34. When the storage mode is the second to fourth storage modes, the failure videos 54 to be stored are selected, and the failure videos 54 are sent from the extraction unit 39 to the selection unit 41, where they are temporarily stored in the temporary storage unit for selection 42.

The selection unit 41 selects the failure videos 54 according to the selection criteria defined for each of the second to fourth storage modes. This selection narrows down the failure scenes that are likely to be useful in analyzing the cause of the failure. Selection is performed by comparing the failure scenes and failure information in the failure videos 54, or by determining whether the failure scenes meet the criteria defined for each storage mode.

When performing the selection process, the temporary storage unit for selection 42 temporarily stores each failure video 54 before the selection process is performed. It is preferable that selection be performed after the extraction process is completed and all failure videos 54 have been temporarily stored. Temporary storage is achieved by storing the videos in a temporary storage area (not shown) that is volatile memory or non-volatile memory within the medical image processing device 17 via the temporary storage unit for selection 42. If the temporary storage area is non-volatile memory, the videos may be temporarily stored in the same physical area as the main storage area 34.

The importance calculation unit 43 calculates the importance of each failure video 54. The importance can be expressed, for example, as a percentage (%) or a graded rating (high, medium, low). The more important a failure video 54 is, the more likely it is to be useful for failure analysis, etc. The failure information used to calculate the importance is obtained by the image information management unit 40, which stores failure information for trigger signals.

The importance is calculated using at least one of the following: the order of trigger signal reception timing, the type of fault, the frequency of fault occurrence, the severity of the fault, the confidence of fault detection, the visibility of the fault, its impact, and the type of endoscope 12. The severity of the fault can be calculated using the area of the non-lesion abnormal region T in the fault frame image 52, as well as the color brightness, saturation, hue, and location of occurrence. The visibility of the fault can be calculated using the difference in brightness between the non-lesion abnormal region T and the background subject. Types of endoscopes 12 include treatment endoscopes, diagnostic endoscopes, upper gastrointestinal endoscopes, lower gastrointestinal endoscopes (colonoscopes), transnasal endoscopes, etc.

The output control unit 33 controls the display of the failure video 54 on the display 18. When displaying the failure video 54, the output control unit 33 accepts confirmation of the failure scene and reference to failure information via the user interface 16, as well as user editing of image information and video composition including failure information. It is also possible to output and display the temporarily saved medical video 50 on the display 15 or display 18.

As shown in FIG. 11 , the display 18 displays an image display area 60 that displays the failure video 54 and corresponding failure still images, an image information display area 61 that displays image information including failure information, and a video playback toolbar 64 that displays the playback status of the failure video 54. In the image display area 60, for example, the failure video 54 can be played back in the order in which it was extracted, allowing the details of the failure to be confirmed. The failure information displayed in the image information display area 61 includes, for example, the type of failure, the duration of the failure scene, the average area of the non-lesion abnormal region T, and the order in which the trigger signal was received. During the extraction process, personal information contained in the image information of the medical video 50 is processed so that it cannot be viewed. For example, in FIG. 7, the patient ID 62 displayed as "□□□□" and the patient name 63 displayed as "△△△△" are irreversibly processed so that the text cannot be read.

As shown in FIG. 12, in addition to the failure video 54, the display 18 may also display a failure still image 55 corresponding to the failure scene in the failure video 54 in the image display area 60. Because the failure still image 55 shows the failure scene in the failure video 54, it is possible to confirm the state of the failure, for example, the non-lesion abnormal area T.

Unless otherwise specified, the extraction process for each storage mode described below will be performed within the first extraction range. For example, consecutive frame images for three seconds before and after the timing of the fault discovery will be extracted to obtain the fault video 54.

The storage mode is preferably set in advance by the user based on the content of the endoscopic examination, etc. There are two storage modes: a first storage mode in which no selection process is performed and all failure videos 54 extracted for each failure scene are stored in the main memory area 34, and second to fourth modes in which the extracted failure videos 54 are sent to the temporary storage unit for selection 42 for temporary storage, and the failure videos 54 to be stored in the main memory area 34 are selected.

In the first storage mode, the medical video 50 temporarily stored in the temporary storage unit 38 is extracted as a failure video 54 based on the first failure frame image 52 in the failure scene, i.e., the timing of receiving the trigger signal, and all of the extracted failure videos 54 are stored in the main memory area 34.

As shown in Figure 13, in the first storage mode in which no sorting is performed, an example will be described in which failure frame image 52a, failure frame image 52b, and failure frame image 52c, which are different failure scenes, are detected from medical video 50. Based on the trigger signal, failure video 54a containing the failure scene of failure frame image 52a, failure video 54b containing the failure scene of failure frame image 52b, and failure video 54c containing the failure scene of failure frame image 52c are extracted based on the extraction range setting and saved in main memory area 34. In the case of the first extraction range, failure videos 54 that are not affected by the length of the failure scene are each extracted.

The extraction and selection of failure videos 54 in the second to fourth storage modes will be explained below. All extracted failure videos 54 are temporarily stored in the temporary selection storage unit 42. The failure videos 54 are selected based on the failure information, and the selected failure videos 54 are stored in the main memory area 34.

As shown in Figure 14, an example will be described in which failure frame images 52a, 52b, and 52c, which are different failure scenes but have the same type of failure, are detected from a medical video 50 by the failure detection process in the second to fourth storage modes for sorting. Based on the trigger signal and extraction range settings, failure video 54a containing the failure scene of failure frame image 52a, failure video 54b containing the failure scene of failure frame image 52b, and failure video 54c containing the failure scene of failure frame image 52c are extracted and temporarily stored in the temporary storage unit for sorting 42. After all failure videos 54 to be sorted have been temporarily stored, the sorting unit 41 compares each failure scene and sorts them according to the set sorting criteria. If failure video 54c is selected, failure videos 54a and 54b are deleted, and failure video 54c is stored in the main memory area 34.

In the second and third storage modes, the failure videos 54 are sorted by failure type. If multiple failures are detected during the failure detection process, at least one failure video 54 for each type is saved in the main memory area 34. Even if multiple failures are detected in the medical video 50, including a type of failure that occurs frequently and has a large area of non-lesion abnormal region T and a type of failure that occurs infrequently and has a small area of non-lesion abnormal region T, it is possible to review the saved failure videos 54 without missing any scenes of each type of failure.

In the second storage mode, the failure videos 54 to be saved in the main memory area 34 are selected based on the type of failure and the chronological order of the trigger signal reception timing in the medical video 50. For example, for each failure type, the first or last N failure scenes contained in each failure video 54, or failure videos 54 corresponding to one failure scene at regular intervals, are saved. The first failure scene is one in which little time has elapsed since the start of the endoscopic examination, making it easier to detect failures other than those caused by the endoscopic examination, such as scope contamination. In the last failure scene, if multiple failures occur, it is easier to detect multiple types of failures from a single failure scene. Furthermore, by selecting at regular intervals, it is possible to check whether there are any failures that occur throughout the entire endoscopic examination.

In the third storage mode, the importance of each failure video 54 is calculated for each type of failure, and the failure videos 54 to be stored in the main memory area 34 are selected based on the importance. The importance calculation unit 43 calculates the importance of each failure scene and links it to the failure information. The importance indicates the possibility of a serious failure or its usefulness in analyzing the cause of the failure. For example, calculation results with an importance of 70% or more, or "high," are saved as failure videos 54, and calculation results with an importance of less than 70% or "low" are deleted. The method of calculating the importance will be described later. Furthermore, instead of selecting failure videos 54 whose importance is evaluated to be above a certain value, a certain number or percentage of failure videos 54 may be selected in order of increasing importance from the failure videos 54 temporarily stored in the temporary storage unit for selection 42.

In the fourth storage mode, the user selects whether or not to save the failure video 54 individually in the main memory area 34. Specifically, the temporarily saved failure video 54 and image information containing at least the failure information are displayed on the display 18, and the user checks the played failure video 54 and image information and selects the failure video 54 to save individually in the main memory area 34. User operations in the selection process are performed via the user interface 19, and include methods such as executing play commands and save commands provided in the command area 65 using a keyboard, mouse, or foot pedal. User operations may also be used to edit the image information.

As shown in FIG. 15, instead of playing the failure videos 54 one by one in order, a list of the failure videos 54 may be displayed in the image display area 60. It is preferable that the image display area 60 displays a list of each failure video 54, and the image information display field 61 displays image information for the selected failure video 54. A decision as to whether to save or delete the video may be made based on the image information. When playing a selected failure video 54, the video may be played at the same size as displayed in the list, or may be temporarily enlarged and played as shown in FIG. 11. It is also preferable that for failure videos 54 that are not currently being played in the image display area 60, the corresponding failure still images 55 are displayed as thumbnails. Note that multiple failure videos 54 may be played simultaneously.

The selection process is not limited to the contents of the second to fourth storage modes described above, and may combine the selection conditions for each storage mode. Furthermore, after the failure video 54 is saved in the main memory area 34, it may be displayed on the display 18 for confirmation.

The sorting process may be performed whenever a failure video 54 is extracted, rather than after the extraction process has finished when all failure videos 54 have been collected. The most recently temporarily saved failure video 54 is compared in an either/or format with the newly extracted failure video 54, and the unnecessary one is deleted. For example, in the third storage mode, the importance of the temporarily saved mth extracted failure video 54 and the newly extracted m+1th failure video 54 are compared, and the one with the lower importance is deleted. This allows the sorting process to be performed even when time or temporary storage space is limited.

The function of the selection unit 41 may be located within the processor device 14. In that case, a trigger signal is sent for selected faults, so the medical image processing device 17 may always operate in the first storage mode. Furthermore, the selection by the processor device 14 may also be performed by the selection unit 41 of the medical image processing device 17 in combination. When using multiple selections, it is preferable to perform the selection under different conditions.

The sequence of operations for controlling the extraction and storage of a failure video 54 in this embodiment will be described with reference to the flowchart shown in Figure 16. The endoscope system 11 acquires a real-time medical video 50 captured inside a living body using the endoscope 12 (step ST110). The processor device 14 in the endoscope system 11 performs failure detection processing to detect failure frame images 52 from the medical video 50 and determine whether a failure has occurred. A failure still image 55 may be created from the failure frame image 52 (step ST120). Simultaneously with the failure detection processing, the medical video 50 is transmitted to the medical image processing device 17 (step ST130). If no failure scene is detected from the medical video 50 by the failure detection processing (N in step ST140), it is determined that there is no failure in the endoscope 12 used in the endoscopic examination.

If a fault is detected in the medical video 50 through the fault detection process (Y in step ST140), a trigger signal containing information on the timing of the fault occurrence is created (step ST150). The trigger signal is immediately sent to the medical image processing device after creation (step ST160). The medical image processing device 17 compares the trigger signal reception timing contained in the received trigger signal with the medical video 50 stored in real time, and identifies the trigger signal reception timing (step ST170). A fault video 54 containing at least a fault scene is extracted from the medical video 50 according to the set extraction range (step ST180). If a fault still image 55 was not acquired from the fault frame image 52 during fault detection, a fault still image 55 depicting the fault scene contained in the fault video is also acquired (step ST190).

All extracted failure videos 54 may be saved as is, or the failure videos 54 to be saved may be selected. Whether or not selection is performed is determined by the save mode. If the save mode is the first save mode, i.e., if selection of failure videos 54 is not performed (N in step ST200), all extracted failure videos 54 are saved in the main memory area 34, and the process ends (step ST230).

The selection process compares the failure videos 54 with each other, so it is preferable to perform the process after the extraction process is completed, i.e., after the endoscopic examination is completed. When selecting failure scenes in the second to fourth storage modes (Y in step ST200), the failure videos 54 are temporarily stored in the temporary storage unit for selection 42 (step ST210). The temporarily stored failure videos 54 are then judged for each storage mode to determine whether they meet certain criteria, and the failure videos 54 are compared with each other, and the failure videos 54 to be stored are selected (step ST220). The selected failure videos 54 are stored in the main memory area 34, and the process ends (step ST230).

Second Embodiment
In the first embodiment, the temporary storage area for temporarily storing the medical video 50 captured in real time during an endoscopic examination and the failure video 54 is the same whether it is a volatile memory or a nonvolatile memory. In this embodiment, a description will be given of a mode in which the temporary storage is performed in a nonvolatile memory. Note that a description of the content common to the above embodiment will be omitted.

The functions of the temporary storage unit 38 and the temporary storage unit for selection 42 may be realized by non-volatile memory that maintains stored data even when the medical image processing device 17 is not powered. It is possible that the power to the endoscope system 11 or medical image processing device 17 may be turned off during an endoscopic examination, or that the power to the medical image processing device 17 may be turned off after the endoscopic examination is completed before the failure video 54 has been completely saved in the main memory area 34. Even in such cases, the data temporarily stored in the non-volatile memory can be prevented from being lost when the power is turned off.

When sorting is performed, the extracted failure videos 54 are temporarily stored in the temporary sorting storage unit 42, which is a non-volatile memory of the medical image processing device 17, in sequence. If the power to the endoscope system 11 or medical image processing device 17 is turned off during an endoscopic examination, the endoscopic examination is terminated at that point, and when the power is turned on again, sorting of the failure videos 54 begins. After the endoscopic examination is completed, the sorting process is performed, and the selected failure videos 54 are stored in the main memory area 34.

As shown in Figure 17, this section describes the case where the power is turned off during an examination when temporary storage is performed in a temporary storage area, which is non-volatile memory. Examination A is an endoscopic examination that ended without the power being turned off, and Examination B is an endoscopic examination in which the endoscopic system 11 or medical image processing device 17 was turned off during real-time detection. In Examination A, in which the user operated the endoscope examination, the failure video 54 is saved as in the first embodiment. In Examination B, in which the endoscope examination was forcibly terminated due to the power being turned off rather than being terminated by the user, when the power is turned back on, the failure video 54 is extracted from the medical video 50 temporarily saved in non-volatile memory at the time the power was turned off. If the second to fourth save modes are selected, the failure video 54 is saved in the main memory area 34. Temporarily saving in non-volatile memory allows the failure video 54 to be extracted and selected from the medical video 50 that was captured and temporarily saved regardless of the end status of the endoscopic examination, such as when the power was turned off during the examination.

The saving process sends the selected failure videos 54 from the non-volatile memory to the main memory area 34 after selection is complete. However, the medical image processing device 17 may be powered off before selection is complete or during the saving process, or the main memory area 34 may be disconnected if it is detachable from the medical image processing device 17. If the saving process is interrupted, it may be impossible to distinguish between failure videos 54 that have been saved in the main memory area 34 and failure videos 54 that have not been saved. For this reason, an identifier such as a flag is set when extracting the failure videos 54 to enable automatic discrimination.

When temporarily saving failure videos 54, a different identifier is set for each failure video 54 after extraction. The identifier is preferably a flag or the like and is set as image information, but is not particularly limited as long as it allows for distinguishing between failure videos 54. When resuming the interrupted saving process, the identifier of the failure video 54 saved in the main memory area 34 is checked, and it is determined whether or not there is a failure video 54 with the same identifier in the temporary storage area, which is non-volatile memory. Failure videos 54 with the same identifier as the confirmed identifier are not saved, and are deleted from non-volatile memory.

If there is a failure video 54 in the temporary storage area for which the same identifier has not been confirmed, it is determined that the failure video 54 is not saved in the main memory area 34, and a save process is performed. Note that the identifier may be used not only when the save process is interrupted, but also during normal save processing to confirm that the save has been performed reliably.

[Third embodiment]
In the third embodiment, information contained in a trigger signal is embedded in a medical video 50 during fault detection processing, and a fault video 54 and a fault still image 55 are extracted. Descriptions of content common to the above embodiments will be omitted. The fault identification unit 25 of the processor device 14 realizes the function of a trigger signal providing unit (not shown) that edits the medical video 50 in response to the creation of a trigger signal.

As shown in FIG. 18, the processor device 14 edits the medical video 50 captured by the endoscope 12 in real time, assigns a trigger signal to frame images in which a fault is detected, and transmits them to the medical image processing device 17. It is preferable to perform fault detection processing and embed the trigger function and fault information of the trigger signal for the detected fault frame image 52 into the medical video 50 as an identifier such as a fault flag.

The medical image processing device 17 extracts the failure video 54 and failure still image 55 based on the trigger function and failure information embedded in the temporarily stored medical video 50 acquired from the processor device of the endoscope system 11. By extracting from medical videos 50 with failure flags attached, it is possible to eliminate errors between the timing of failure detection and the extracted frame images. It is preferable that the medical video 50 output from the output control unit 23 has undergone failure detection processing. In this case, the frame images captured during the endoscopic examination are displayed on the display 15 with a delay corresponding to the time required for the failure detection processing. If display delays are a problem, the medical video 50 before the failure detection processing may be transmitted from the output control unit 23 to the display 15.

[Fourth embodiment]
In the fourth embodiment, a second medical video is created in the fault detection process by freezing a frame in the medical video 50 in which a fault is detected for a certain period of time, and the created second medical video is transmitted to the medical image processing device 17. Description of content common to the above embodiments will be omitted. The fault identification unit 25 of the processor device 14 realizes the function of a second medical video creation unit (not shown) that creates a second medical video in which the medical video 50 is edited in response to fault detection.

As shown in FIG. 19, when the endoscopic examination begins, the processor device 14 copies the medical video 50 captured by the endoscope 12 in real time, and transmits the edited second medical video together with the medical video 50 to the medical image processing device 17. The second medical video records the same content as the medical video 50 when no fault is detected, and when a fault is detected, records footage that has been frozen for a certain period of time from the time of fault detection. In addition to the medical video 50 and the second medical video, a trigger signal is also transmitted from the processor device 14 to the medical image processing device 17. The second medical video transmitted from the endoscopic system 11 to the medical image processing device 17 together with the medical video 50 is temporarily stored in the temporary storage unit 38.

The extraction unit 39 extracts a failure video 54 from the medical video 50 and a failure still image 55 from the second medical video upon receiving a trigger signal from the endoscope system 11. The timing of the temporary storage of the medical video 50 and the second medical video in the temporary storage unit 38 and the reception of the trigger signal are asynchronous, resulting in errors. Since the same failure frame image 52 continues in the failure scene of the second medical video upon reception of the trigger signal, it is possible to obtain a failure still image 55 taken at the time the failure was detected, even though the timing is asynchronous.

The failure video 54 and failure still image 55 extracted by the same trigger signal are preferably associated with a failure flag or the like and stored in the main memory area 34. It is preferable that the second medical video is not stored after the failure still image 55 is extracted, but is instead deleted.

As a modified example of the fourth embodiment, a form in which a failure video 54 and a failure still image 55 are extracted by user operation will be described. During an endoscopic examination, a freeze operation in which the user presses the freeze button 12d provided on the operation unit 12b of the endoscope 12 generates a second trigger signal that serves as an extraction trigger. Upon receiving the second trigger signal manually generated from the endoscope system 11, the medical image processing device 17 extracts a failure video 54 from the medical video 50 and extracts a failure still image 55 from the second medical video.

When the user performs a freeze operation, the second medical video, in which the video is frozen for a certain period of time, is sent to the medical image processing device 17. The freeze operation also causes the trigger signal creation unit 27 to create a second trigger signal, which is sent to the medical image processing device 17. When the medical image processing device 17 receives the trigger signal, the extraction unit 39 extracts the failure video 54 from the medical video 50 and the failure still image 55 from the second medical video, and stores them in the main memory area 34.

It is preferable that the failure video 54 or failure still image 55 extracted in response to the trigger signal and the failure video 54 and failure still image 55 extracted in response to the second trigger signal are processed and saved so that they can be visually distinguished by the user. For example, when extracted in response to the second trigger signal, it is preferable to process the images of the failure video 54 and failure still image 55 so that they are surrounded by a red frame or to add a symbol such as a "△" to the upper right corner of the image. The word "Manual" may also be added to the file name.

In the above embodiment, the hardware structure of the processing units that perform various processes, such as the central control unit, image acquisition unit 21, input receiving unit 22, output control unit 23 realized by the processor device 14, and the fault determination unit 26, trigger signal creation unit 27, still image creation unit 28, and second medical video creation unit included in the fault identification unit 25, and the central control unit, data acquisition unit 31, input receiving unit 32, output control unit 33 realized by the medical image processing device 17, and the extraction range setting unit 36, mode control unit 37, temporary storage unit 38, extraction unit 39, image information management unit 40, selection unit 41, temporary storage unit for selection 42, and importance calculation unit 43 included in the fault video extraction unit 35, are various processors as shown below. Various types of processors include CPUs (Central Processing Units), which are general-purpose processors that execute software (programs) and function as various processing units, programmable logic devices (PLDs), such as FPGAs (Field Programmable Gate Arrays), which are processors whose circuit configuration can be changed after manufacture, and dedicated electrical circuits, which are processors with circuit configurations designed specifically for performing various processes.

A single processing unit may be configured with one of these various processors, or with a combination of two or more processors of the same or different types (for example, multiple FPGAs, or a combination of a CPU and an FPGA). Multiple processing units may also be configured with a single processor. Examples of multiple processing units configured with a single processor include, first, a configuration where a single processor is configured with a combination of one or more CPUs and software, as is typical of client or server computers, and this processor functions as multiple processing units. Second, a configuration where a processor is used to realize the functions of an entire system including multiple processing units on a single IC (Integrated Circuit) chip, as is typical of system-on-chip (SoC). In this way, the various processing units are configured with a hardware structure using one or more of the various processors listed above.

More specifically, the hardware structure of these various processors is an electrical circuit (circuitry) that combines circuit elements such as semiconductor devices. The hardware structure of the memory unit is a storage device such as an HDD (hard disk drive) or SSD (solid state drive).

10 Medical image processing system 11 Endoscope system 12 Endoscope 12a Insertion section 12b Operation section 12c Tip section 12d Freeze button 13 Light source device 14 Processor device 15 Display 16 User interface 17 Medical image processing device 18 Display 19 User interface 21 Image acquisition section 22 Input receiving section 23 Output control section 25 Fault identification section 26 Fault determination section 27 Trigger signal creation section 28 Still image creation section 31 Data acquisition section 32 Input receiving section 33 Output control section 34 Main memory area 35 Fault video extraction section 36 Extraction range setting section 37 Mode control section 38 Temporary storage section 39 Extraction section 40 Image information management section 41 Selection section 42 Temporary storage section for selection 43 Importance calculation section 50 Medical video 51 Normal frame image 52 Fault frame image 52a Fault frame image 52b Fault frame image 52c Fault frame image 53 No-fault abnormal frame image 54 Fault video 54a Fault video 54b Fault video 54c Fault video 55 Fault still image 60 Image display area 61 Image information display field 62 Patient ID
63 Patient name 64 Video playback toolbar 65 Command area C Disconnection R Lesion area T Non-lesion abnormal area

Claims (15)

an external device having a processor; and an endoscope system;
The processor:
acquiring real-time medical video from the endoscopy system during an endoscopy examination;
receiving a trigger signal having information about a failure occurrence generated based on a failure scene in the medical video from the endoscope system;
Temporarily store the medical video;
extracting a failure video and a failure still image including the failure scene from the temporarily stored medical video based on a timing of receiving the trigger signal;
The medical image processing system stores the failure video and the failure still image in a main storage area. The processor:
acquiring the medical video in which the trigger signal is embedded from the endoscope system;
2. The medical image processing system according to claim 1, wherein the failure video and the failure still image are extracted from the medical video based on the trigger signal. the endoscope system creates the failure still image upon detecting a failure;
The medical image processing system of claim 1 , wherein the processor acquires the failure still image from the endoscope system. The processor:
acquiring a second medical video from the endoscope system by freezing the video of the occurrence of the failure for a certain period of time;
Temporarily storing the second medical video;
2. The medical image processing system according to claim 1, wherein the failure still image is extracted from the temporarily stored second medical video based on the timing of receiving the trigger signal. The processor:
acquiring a second trigger signal generated by a freeze operation by a user in the endoscope system;
5. The medical image processing system according to claim 4, wherein the failure video and failure still images are extracted based on the second trigger signal. The processor:
The medical image processing system according to claim 5, wherein the failure still image and the failure video extracted in response to the second trigger signal are processed so that they can be visually distinguished from the failure still image and the failure video extracted in response to the trigger signal. The processor:
7. The medical image processing system according to claim 1, further comprising: a first extraction range for extracting, from the medical video, a timing at which the trigger signal is received and scenes before and after the timing of the reception of the trigger signal, before the start of the endoscopic examination. The processor:
In the first extraction range, a first limit period is set at the timing of receiving the trigger signal, the first limit period continuing at least until the recording range of the failure video is determined;
8. The medical image processing system according to claim 7, wherein if the trigger signal indicating the same type of fault is received during the first limited period, new extraction is not started and only fault information is acquired. The processor:
A medical image processing system according to any one of claims 1 to 6, wherein a second extraction range is set before the start of the endoscopic examination, and a range of consecutive trigger signals having information on the occurrence of the same type of fault is extracted from the medical video. The processor:
10. The medical image processing system according to claim 9, wherein, before the start of the endoscopic examination, a third extraction range is set in place of the second extraction range, and the third extraction range is used to extract the second extraction range and scenes before and after the second extraction range from the medical video. The processor:
In the second extraction range or the third extraction range, a second limit period is set at the timing of receiving the trigger signal, the second limit period continuing at least until the recording range of the failure video is determined;
11. The medical image processing system according to claim 10, wherein if a trigger signal indicating the same type of fault is received during the second limited period, new extraction is not started, and acquisition of fault information and extension of the extraction range are accepted. The processor:
12. The medical image processing system according to claim 10, wherein the recording range of the failure video is determined after a certain period of time has elapsed in the second extraction range or the third extraction range. The processor:
13. The medical image processing system according to claim 1, wherein an upper limit is set for the number of times the extraction is performed. The processor:
14. The medical image processing system according to claim 1, wherein personal information included in image information of the medical video is deleted or processed so as not to be viewable when the failure video and the failure still image are acquired. 1. A method of operating a medical imaging system comprising an external device having a processor and an endoscopy system, the method comprising:
acquiring , by the external device, real-time medical video from the endoscopy system during an endoscopy examination;
receiving, from the external device, a trigger signal having information on the occurrence of a failure, the trigger signal being generated based on a failure scene in the medical video;
The external device temporarily stores the medical video;
The external device extracts a failure video and a failure still image including the failure scene from the temporarily stored medical video based on a timing of receiving the trigger signal;
and a step in which the external device stores the failure video and the failure still image in a main memory area.