JP6625412B2 - Endoscope system and display method thereof - Google Patents

Description

  The present invention relates to an endoscope system that records and displays an image obtained by an endoscope, and a display method thereof.

  In recent years, endoscopes have been widely adopted in the medical field and the like. Medical images obtained by the endoscope are recorded on various media for diagnosis and recording of cases. In addition, with the increase in the capacity of the recording medium, recording of a moving image from an endoscope has also been performed. Such a medical image recording process is digitized, and the medical image may be stored in a file format readable by a computer.

  By the way, among the recorded medical images, scenes necessary as, for example, case recording moving images are generally part of the scenes. For example, in an endoscope, performing safe and secure surgery is the highest priority, and thus, as an important scene as a moving image of a case record, for example, excision, exfoliation treatment, hemostatic treatment, and the like can be considered. However, at the time of reproducing a medical image, it is not easy to confirm the recording position of such an important scene, and the reproduction operation for viewing the important scene is complicated.

  Therefore, in Patent Document 1, a feature point is detected at the time of recording, and information of the feature point is stored in synchronization with an endoscope image, so that an endoscope moving image corresponding to the feature point at the time of reproduction is obtained. A technique capable of starting the reproduction from the position is disclosed. Thus, in the device disclosed in Patent Document 1, for example, when recording the situation of an operation and confirming the recorded past image, the operator does not search for a point (time) at which reproduction is started, and searches for a point (time) from a feature point. Playback can begin.

JP 2011-36370 A

  However, the proposal of Patent Literature 1 aims to improve operability when confirming recorded past images, and to display or reproduce a useful moving image with a short and simple operation. However, it is not always possible to support certain medical acts during the course of such medical acts.

  For example, during endoscopic surgery, unexpected bleeding from the surgical area may occur. Since the bleeding may obscure the operation area, the operator needs to remove the blood from the operation area, check the bleeding position, and perform the hemostatic treatment. However, it is not easy for an operator to confirm the position of the bleeding and the degree of the bleeding because the blood in the observation range is covered by the blood due to the bleeding. In this case, the surgeon may check the bleeding position by reproducing the recorded observation image. Also in the proposal of Patent Document 1, it is possible to display an image portion at the time of bleeding from an endoscope image recorded in the past. However, only the proposal of Patent Literature 1 cannot simultaneously check a past image and a current live image on the same monitor, and cannot reliably support a medical practice of an operator.

  The present invention makes it possible to simultaneously display a current live image and a chase playback (time-shift playback) image from a playback position desired by the operator during a medical procedure by a simple operation, thereby enabling the operator to perform a medical procedure. To provide an endoscope system and a display method for the endoscope system capable of reliably supporting the endoscope.

An endoscope system according to one embodiment of the present invention includes a display unit that displays an endoscope image captured by an endoscope, and a treatment timing of a treatment tool that records the endoscope image and treats a subject. A recording unit that records cueing information in association with a recording position on the time axis of the endoscope image to be recorded, and a recording position related to the cueing information when a user operation is detected. From the one-screen display in which the endoscope image recorded in the recording unit is time-shifted and played back, and the display unit is controlled so that the endoscope image being captured by the endoscope is a live image. anda controller for switching the two-screen display of the time shift reproduction image by the time-shift playback with the live image, wherein, when the cueing information is a plurality of recording, based on the user operation One of the cue information that is more recorded have been selected to determine the origin of the time shift reproduction.
An endoscope system according to another aspect of the present invention includes a display unit that displays an endoscope image captured by an endoscope, and a treatment tool that records the endoscope image and treats a subject. When a signal relating to the treatment timing is given, a recording unit that records cueing information in association with a recording position on the time axis of the endoscopic image to be recorded, and a recording unit that detects cueing information when a user operation is detected. The endoscope image recorded in the recording unit is time-shifted and reproduced from the recording position as a starting point, and the display unit is controlled to make the endoscope image being captured by the endoscope a live image. A control unit for switching from a screen display to a two-screen display of the live image and a time-shifted reproduced image by the time-shifted reproduction, wherein the control unit controls the recording timing of the cue information and the user operation. If the time difference from the time is less than a predetermined threshold time, time shift reproduction is performed with the recording position related to the cue information as a starting point, and the time difference between the recording timing of the cue information and the timing of the user operation. Exceeds a predetermined threshold time, time-shift reproduction is performed starting from a timing that is a predetermined time before the timing of the user operation.
Further, an endoscope system according to another aspect of the present invention includes a display unit that displays an endoscope image captured by an endoscope, and a treatment tool that records the endoscope image and treats a subject. When a signal related to the treatment timing is given, a recording unit that records cueing information in association with a recording position on the time axis of the endoscopic image to be recorded, and, when a user operation is detected, the cueing information is related to the cueing information. The endoscope image recorded in the recording unit is time-shifted and reproduced from the recording position as a starting point, and the display unit is controlled to make the endoscope image being captured by the endoscope a live image. A control unit for switching from a screen display to a two-screen display of the live image and a time-shift reproduction image by the time-shift reproduction, wherein the control unit performs recording of the cue information and user operation If the time difference between the timing exceeds a predetermined threshold time, performs time shift reproduction starting timing of a predetermined time before the timing of the user operation.

In addition, the display method of the endoscope system according to one embodiment of the present invention displays the endoscope image captured by the endoscope on a display unit as a live image and records the endoscope image in a recording unit. Information recording for recording a cue information in association with a display recording step and a recording position on the time axis of the endoscope image in the display recording step when a signal regarding a treatment timing of a treatment tool for treating a subject is given; And detecting, when a user operation is detected, an endoscope image recorded in the recording unit in a time-shifted manner with the recording position related to the cue information as a starting point, and controlling the display unit to perform the internal Switching from one-screen display in which an endoscope image being captured by an endoscope is used as a live image to two-screen display of the live image and a time-shifted reproduced image by the time-shifted reproduction. Anda display step, said playback display step, when the cueing information is a plurality of recording, the selected one of said cue information based on the user operation is a plurality of recording time Determine the starting point of the shift reproduction.
Display method of the endoscope system, characterized in that.
Also, a display method of an endoscope system according to another aspect of the present invention displays an endoscope image captured by an endoscope on a display unit as a live image, and records the endoscope image in a recording unit. Information to record the cueing information in association with the recording position on the time axis of the endoscopic image in the display recording step, when a signal relating to the treatment timing of the treatment tool for treating the subject is given. When the recording step and the user operation are detected, the endoscope image recorded in the recording unit is time-shifted and played back from the recording position related to the cue information, and the display unit is controlled, Switching from one-screen display in which an endoscope image being captured by an endoscope is used as a live image to two-screen display of the live image and a time-shift reproduction image by the time-shift reproduction. A raw display step, wherein the reproduction display step includes, when a time difference between the recording timing of the cue information and the timing of the user operation is within a predetermined threshold time, the cue information related to the cue information. Performing time-shift reproduction with the recording position as a starting point, and when a time difference between the recording timing of the cue information and the timing of the user operation exceeds a predetermined threshold time, a timing that is a predetermined time before the timing of the user operation is set. Time shift reproduction is performed as a starting point.
Furthermore, a display method of an endoscope system according to another aspect of the present invention displays an endoscope image captured by an endoscope on a display unit as a live image, and records the endoscope image in a recording unit. Information to record the cueing information in association with the recording position on the time axis of the endoscopic image in the display recording step, when a signal relating to the treatment timing of the treatment tool for treating the subject is given. When the recording step and the user operation are detected, the endoscope image recorded in the recording unit is time-shifted and played back from the recording position related to the cue information, and the display unit is controlled, Switching from one-screen display in which an endoscope image being captured by an endoscope is used as a live image to two-screen display of the live image and a time-shift reproduction image by the time-shift reproduction A reproduction display step, wherein the reproduction display step includes a step of, when a time difference between the recording timing of the cue information and the timing of the user operation exceeds a predetermined threshold time, a predetermined time from the timing of the user operation. Time shift reproduction is performed starting from the previous timing.

  According to the present invention, the current live image and the time-shifted reproduced image from the reproduction position desired by the operator can be simultaneously displayed during the medical operation by a simple operation, thereby ensuring the medical operation of the operator. This has the effect that it can be supported.

FIG. 1 is a block diagram showing an endoscope system according to a first embodiment of the present invention. FIG. 5 is an explanatory diagram for explaining recording of cue information. 5 is a flowchart for explaining the operation of the first embodiment. FIG. 4 is an explanatory diagram for explaining the operation of the first embodiment. Explanatory drawing which shows an example of a screen display in a normal observation mode. FIG. 4 is an explanatory diagram illustrating an example of a screen display in a time shift observation mode. FIG. 7 is a flowchart showing an operation flow employed in the second embodiment of the present invention. FIG. 9 is an explanatory diagram for explaining an operation according to the second embodiment. FIG. 9 is an explanatory diagram for explaining an operation according to the second embodiment.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

(First Embodiment)
FIG. 1 is a block diagram showing an endoscope system according to a first embodiment of the present invention.

  In FIG. 1, an endoscope system 10 includes an endoscope 11 and an electric mess scissors 14. The endoscope 11 is configured to be inserted into a subject, image a subject such as a living tissue in the subject, and output the image as an image signal. The endoscope 11 includes an elongated insertion portion 11a to be inserted into a subject, and an operation portion 11b provided at a base end of the insertion portion 11a, and a switch is provided on the operation portion 11b. There are provided a plurality of operation units including the operation unit 12. An imaging signal obtained by the endoscope 11 is supplied to a video processor 21.

  The video processor 21 drives the endoscope 11 and captures an image signal from the endoscope 11 to perform image signal processing to generate an endoscope image. The video processor 21 can output an endoscope image to the recording / reproducing device 22 and the monitor 30.

  The electric knife scissors 14 are connected to the electric knife device 23. The foot switch 15 is connected to the electric scalpel device 23. The foot switch 15 instructs the electric scalpel device 23 to turn on / off a high-frequency current output based on a user operation. The electrosurgical device 23 generates a high-frequency current based on an on / off instruction by the foot switch 15. The high-frequency current is supplied to the electric scissors 14.

  The electric scalpel scissors 14 transmits the supplied high-frequency current from the distal treatment section to the living tissue. Thus, the electric scalpel scissors 14 can perform electroscalpel treatment such as cauterization and coagulation on living tissue.

  The controller 24 as a control unit is connected to the video processor 21, the recording / reproducing device 22, the electric scalpel device 23, and the monitor 30, so that these can be centrally controlled. The controller 24 has an operation panel (not shown). When communication is performed between the controller 24 and these devices, the setting state or operation of the connected device is displayed on the display screen of the operation panel. A setting screen for switches and the like can be displayed. Further, the controller 24 can perform an operation input such as a change of a set value by touching a desired operation switch and operating a touch panel in a predetermined area. The controller 24 may be configured by a processor such as a CPU (not shown) that operates according to a program stored in a memory (not shown) and controls each unit.

  The monitor 30 as a display unit receives an endoscope image from the video processor 21 and can display the input endoscope image on a display screen 30a. The recording / reproducing device 22 receives an endoscope image from the video processor 21 and can record the input endoscope image on a predetermined recording medium. The recording and reproducing device 22 can also reproduce the recorded endoscope image. The recording / reproducing device 22 can output the reproduced endoscope image to the monitor 30. The monitor 30 can also display the endoscope image provided from the recording / reproducing device 22 on the display screen 30a.

  In the present embodiment, the recording / reproducing device 22 as a recording unit is controlled by a controller 24 to control recording and reproduction. The recording / reproducing apparatus 22 is capable of reproducing a recorded image without waiting for the end of recording of an endoscopic image being recorded, that is, so-called time-shift reproducing. Further, the recording / reproducing device 22 is controlled by the controller 24 so that the cue information can be added to the endoscopic image being recorded at an arbitrary timing and recorded.

  That is, the recording / reproducing device 22 records the time for recording the cue information and the time managed when recording the endoscope image in association with each other. With this, when the cueing information is specified, the recording / reproducing device 22 sets the position (recording position) on the time axis of the endoscope image recorded at the time corresponding to the time when the specified cueing information was recorded. ) Can be played back. Note that if the recording / reproducing device 22 can reproduce the endoscope image recorded at the timing corresponding to the recording timing of the cue information, as long as the cue information is correlated with the endoscope image, It may be recorded.

  In the present embodiment, the controller 24 can instruct the recording / reproducing device 22 to record the cue information at a timing based on a state signal from a predetermined device in the endoscope system 10. I have. For example, the controller 24 may control the recording / reproducing device 22 so that the cue information is recorded at a timing when the ON instruction by the foot switch 15 is supplied to the electric scalpel device 23. The recording / reproducing device 22 may be controlled so that the cue information is recorded at the timing when the supply of the high-frequency current to the electric scissors 14 is started. That is, the controller 24 can control the recording / reproducing device 22 to record the cue information corresponding to the timing at which the treatment by the electric scissors 14 is started.

  In the example of FIG. 1, an example is described in which a device that determines the timing at which the treatment of the electric mess scissors 14 is started is used as the device that determines the recording timing of the cue information. The recording timing of the cue information may be determined by detecting the state of various devices that can be used, for example, a predetermined device such as an ultrasonic treatment tool.

  FIG. 2 is an explanatory diagram for explaining recording of cue information. FIG. 2 shows a series of frames 42 included in the endoscope image 41 recorded on the recording medium by the recording / reproducing device 22 by a square frame. The triangles in FIG. 2 indicate the cueing information H1 to H3. That is, in the example of FIG. 2, the cueing information H1 is recorded at the timing immediately before the frame f2, the cueing information H2 is recorded at the timing immediately before the frame fn + 2, and the cueing information H3 is recorded at the timing immediately before the frame fm + 2. Indicates that it was recorded.

  The recording / reproducing device 22 can perform time-shift reproduction from an arbitrary position of the endoscope image 41 during recording. Further, in the present embodiment, the recording / reproducing device 22 is controlled by the controller 24 to start time-shift reproduction starting from the recording position of the endoscope image recorded at the recording timing of the cue information. I can do it. In the example of FIG. 2, for example, time-shift reproduction can be started from the positions of frames f2, fn + 2, and fm + 2 corresponding to the recording timings of the cue information H1 to H3. Note that the time shift reproduction may be started from a frame position at a timing that is a predetermined time earlier than the recording position of the endoscope image recorded at the recording timing of the cue information.

  The controller 24 determines the start timing of the time shift reproduction based on the user operation, and performs the time shift reproduction at the time shift reproduction with the recording position of the endoscope image corresponding to the recording timing of the cue information as the reproduction start position. The recording / reproducing device 22 can be controlled so as to start.

  In the present embodiment, the operation of selecting the cue information that defines the start timing and the reproduction start position of the time shift reproduction can be set using a common operation unit. For example, the switch 12 can be set to such an operation unit in FIG. For example, when the switch 12 is operated during the recording of the endoscope image, the controller 24 determines the position of the endoscope image from the recording position of the endoscope image related to the recording timing of the cue information recorded most recently. Start time shift playback. Further, for example, when the switch 12 is operated within a predetermined time from the operation of the switch 12, the controller 24 determines cueing information to be selected retrospectively according to the number of times of operation of the switch 12, and determines the cueing information thus determined. The time-shift reproduction of the endoscopic image is started from the recording position of the endoscopic image related to the recording timing.

  Further, the controller 24 can set the reproduction time of the time shift reproduction to a time specified by the user operation or the system. For example, it is possible to set so that the time-shift reproduction is performed for a specified time such as 3 seconds, and the time-shift reproduction is continued to the recording position of the endoscope image corresponding to the timing when the user operates the switch 12. Can also be set to Further, the controller 24 may control the recording / reproducing device 22 to perform repetitive reproduction in which time shift reproduction is repeated for a predetermined time. Further, the controller 24 may employ not only normal reproduction but also slow reproduction, fast forward reproduction, reverse reproduction, and the like as time shift reproduction.

  The controller 24 can control the display on the monitor 30. The controller 24 provides the endoscope image from the video processor 21 to the recording / reproducing device 22 and the monitor 30 at the same time, thereby setting the normal observation mode for displaying the live image being recorded on the monitor 30 while recording the endoscope image. You can do it. For example, in the normal observation mode, the controller 24 performs one-screen display for displaying a live image on the entire display screen 30a of the monitor 30.

  In the present embodiment, in the time-shift observation mode for performing the time-shift reproduction, the controller 24 displays the endoscope image from the video processor 21, that is, the live image, and simultaneously performs the time-shift reproduction by the recording / reproduction device 22. A time-shifted reproduced image, which is a displayed endoscope image, is displayed. For example, in the time shift observation mode, the controller 24 displays a live image on the live image display area 31 of the display screen 30a and displays a time shift reproduced image on the time shift reproduced image display area 32 of the display screen 30a. I do.

  That is, when the operator operates the switch 12 during the observation in the normal observation mode in which the live image is displayed on the entire area of the display screen 30a, the mode shifts to the time shift observation mode, and the recording position of the cueing information immediately before the operation timing of the switch 12 is recorded. , The time-shift reproduction is started, and the display shifts to the two-screen display in which the live image and the time-shift reproduction image are simultaneously displayed in the regions 31 and 32 on the display screen 30a. Note that, for example, the operation unit 11b of the endoscope 11 may be provided with a switch for stopping the time shift reproduction.

  Next, the operation of the embodiment configured as described above will be described with reference to FIGS. FIG. 3 is a flowchart for explaining the operation of the present embodiment, and FIG. 4 is an explanatory diagram for explaining the operation of the present embodiment. FIG. 5 is an explanatory diagram showing an example of a screen display in the normal observation mode, and FIG. 6 is an explanatory diagram showing an example of a screen display in the time shift observation mode.

  Assume that the insertion section 11a of the endoscope 11 is inserted into a subject to perform endoscope observation. The endoscope 11 outputs the obtained imaging signal to the video processor 21. The video processor 21 generates an endoscope image by performing signal processing on an imaging signal. The controller 24 controls the video processor 21, the recording / reproducing device 22, and the monitor 30 to supply the endoscopic image generated by the video processor 21 to the recording / reproducing device 22 for recording, and monitor the endoscopic image. 30 to be displayed.

  FIG. 5 shows a display example of a screen display in this case (normal observation mode), in which the controller 24 displays an endoscope image in a live image display area 30b of substantially the entire display screen 30a of the monitor 30. The surgeon can observe the inside of the subject with the endoscope image displayed in the live image display area 30b, that is, the live image currently being photographed. This live image is displayed over substantially the entire area of the display screen 30a, has excellent visibility, and allows the operator to easily check the inside of the subject.

  Here, it is assumed that the surgeon operates the foot switch 15 to perform the treatment with the electric mess scissors 14. FIG. 3 shows the control of the controller 24 in the observation mode by the endoscope 11. The controller 24 determines whether or not the time shift reproduction is instructed by operating the switch 12 in step S1. In this case, the operation of the switch 12 has not been performed, and the controller 24 shifts the processing to step S7 to determine whether or not a state signal has been generated. When the surgeon turns on the foot switch 15, the electric scalpel device 23 applies a high-frequency current to the electric mess scissors 14 so that the electric scalpel 14 can be treated by the electric scalpel 14 and supplies a high-frequency current to the electric mess scissors 14. Is generated and output to the controller 24. When detecting that the state signal has been generated, the controller 24 instructs the recording / reproducing device 22 to record the cue information in step S8.

  FIG. 4 shows the recording positions of the cueing information H1 to H3 for each frame 42 of the endoscope image 41 recorded in the recording / reproducing device 22 in the same manner as in FIG. 4 shows that frames f1, f2,... Fn + 1, fn + 2,..., Fm + 1, fm + 2,..., Fr + 1, fr + 2, fr + 3,. This shows that the cue information H1 was recorded at the immediately preceding timing, the cue information H2 was recorded at the timing immediately before the frame fn + 2, and the cue information H3 was recorded at the timing immediately before the frame fm + 2.

  When the recording of the cue information is instructed by the controller 24, the recording / reproducing device 22 records the cue information in association with the frame of the endoscope image 41 recorded at the instructed timing. That is, the cueing information H1 to H3 in FIG. 4 is recorded in association with the frame of the endoscope image 41 recorded at the timing when the user operates the foot switch 15.

  The recording / reproducing device 22 only needs to be able to start time-shift reproduction from the frame of the endoscopic image recorded at the timing when the user operates the foot switch 15 using the cue information. Therefore, the recording / reproducing apparatus 22 may record and hold, for example, time information synchronized with time information from the start of recording of the endoscope image at the timing of the recording instruction of the cue information. Various recording methods can be considered as the delivery information.

  Now, for example, when the operator operates the foot switch 15 to perform the treatment of the electric mess scissors 14 while observing the affected part of the subject with an endoscopic image, the subject has bleeding. And In this case, when the operator confirms the bleeding by the endoscope image, the operator operates the switch 12 of the endoscope 11 to identify the bleeding position. The controller 24 determines that the operation of the switch 12 is a shift operation to the time shift observation mode, and shifts the processing from step S1 to step S2. The controller 24 determines the number of times the switch 12 has been operated in step S2 and determines whether a predetermined time has elapsed in step S3, thereby determining how many times the operator has operated the switch 12 continuously. . For example, when 0.5 seconds is set as the predetermined time, the number of times the switch is operated within 0.5 seconds from the previous switch operation is counted. The controller 24 selects the cue information that has been counted up by the counted number, and stores the selection result in a memory (not shown) (step S4).

  In step S5, the controller 24 specifies the selected cue information to the recording / reproducing device 22 and also specifies a reproducing method of time shift reproduction in the recording / reproducing device 22. For example, it is assumed that the controller 24 designates a three-second repetitive reproduction as a reproduction method. In addition, the controller 24 instructs the monitor 30 to display a live image and a time-shift reproduction image on two screens (step S6).

  Now, in the normal observation mode, it is assumed that the operator operates the switch 12 only once at the timing when the frame fr + 1 in FIG. 4 is recorded. In this case, the controller 24 selects the cue information H3 recorded immediately before the timing at which the switch 12 was operated. For example, when the surgeon operates the switch 12 twice within a predetermined time, the controller 24 selects the cueing information H2 recorded two times immediately before the timing at which the switch 12 was operated. .

  If the selected and specified cueing information is H3, the recording / reproducing device 22 starts time-shift reproduction from frame fm + 2 at a timing corresponding to the cueing information H3. The time-shift reproduced image reproduced by the recording / reproducing device 22 is supplied to the monitor 30. In this way, the monitor 30 is supplied with the live image from the video processor 21 and the time-shifted reproduced image from the recording / reproducing device 22, and is displayed on two screens.

  For example, it is assumed that the operator's operation of the switch 12 is performed at the timing of the frame fr + 1, and the two-screen display of the live image and the time-shift reproduction image is displayed with a delay of one frame period. In this case, as shown in FIG. 5, the live image of the frame fr + 1 displayed in the live image display area 30b is reduced to the live image display area 31 in the period of the next frame fr + 2, as shown in FIG. At the same time, the time-shift reproduced image of the frame fm + 2 is reduced and displayed in the time-shift reproduced image display area 32. The time-shifted reproduced image is displayed for three seconds, then returns to frame fm + 2, and is repeatedly reproduced and displayed.

  This repeated playback display is started at the timing when the user operates the foot switch 15, that is, at the timing when bleeding occurs due to the treatment of the electric mess scissors 14, and is repeatedly played back, for example, for several seconds. The bleeding position can be easily identified by referring to the reproduced image. Note that, as described above, a frame that is a predetermined time earlier than the frame corresponding to the timing at which the foot switch 15 is operated may be set as the starting point of the time shift reproduction. For example, if the predetermined time is set to 2 seconds, it is possible to perform time-shift reproduction starting from an endoscopic image about 2 seconds before bleeding occurs. In addition, a live image is also displayed at the same time as the time-shift reproduced image, so that the surgeon can compare the time-shift reproduced image with the live image. This makes it even easier for the operator to check the bleeding position and the like.

  When the operator operates the switch 12 in the time shift observation mode, the number of operations is determined in step S3, and new cueing information is selected based on the stored cueing information (step S4). ). For example, if the switch 12 is operated once in a case where the reproduction is repeatedly performed from the frame fm + 2 corresponding to the cue information H3, the controller 24 specifies the cue information H2 (step S5). Thereby, the reproduction is repeatedly performed from the frame fn + 2 corresponding to the cueing information H2, and is displayed in the time shift reproduction image display area 32.

  As described above, in the present embodiment, the live image and the time-shift reproduction image are simultaneously displayed on two screens from the normal observation mode in which the live image is displayed on the full screen by a very simple operation of operating the switch by the user. It is possible to shift to the time shift observation mode for display. Thus, even if bleeding or the like occurs due to a change in the state of the device in the endoscope system during an operation or the like, the operator can easily perform an endoscope image at a position corresponding to the bleeding timing using a live image. Can be observed together with the patient, and the medical practice of the operator can be reliably supported. For example, not only identification of a bleeding position but also confirmation of damage to the electric mess scissors which is likely to occur when power is supplied to the electric mess scissors can be easily confirmed.

(Second embodiment)
FIG. 7 is a flowchart showing an operation flow employed in the second embodiment of the present invention. In FIG. 7, the same procedures as those in FIG. The hardware configuration in the present embodiment is the same as in the first embodiment.

  In the proposal of Patent Literature 1, reproduction can be started from a characteristic point such as a time point when the electrocautery device is used, but a complicated operation is required for reproduction from other reproduction positions. In the present embodiment, the operator can enable reproduction from a reproduction position desired by the operator only by operating a single operation unit.

  In the first embodiment, each time the operator operates the switch 12, the frame corresponding to the recording position of the cue information is designated, and the time-shift reproduction of the endoscopic image is performed from the frame. . However, there may be a case where the user wants to check the endoscope image even at a timing that is not related to the treatment time by the electric scissors 14. For example, the timing when the tooth tip of the electric mess scissors that had small cracks due to aging is missing, or when the blood vessel is damaged by forceps or the like to cause bleeding may be irrelevant to the treatment time by the electric mess scissors 14. . In this case, when an operation for instructing reproduction from an arbitrary reproduction position is assigned to a switch different from the switch 12 for instructing reproduction from a reproduction position corresponding to the recording position of the cue information, the operator In such a case, it is necessary to operate a plurality of switches for cueing, and the operation becomes complicated.

  Therefore, in the present embodiment, a predetermined threshold time is set, and when the predetermined threshold time has elapsed from the recording time of the cue information, the controller 24 operates the switch 12 to determine the cue information. It is determined that the instruction is not an instruction for performing the time-shift reproduction retroactively to the recording position, but an instruction for performing the time-shift reproduction of the endoscope image recorded a predetermined time before the operation.

  The flow of FIG. 7 differs from the flow of FIG. 3 in that the processes of steps S11 to S14 are added between steps S3 and S6 of FIG. Step S11 is a process in which the controller 24 determines whether a predetermined threshold time has elapsed from the recording time of the cue information. The controller 24 instructs the recording / reproducing device 22 to record the cue information, and stores information on the time at which the instruction is performed in a memory (not shown). The controller 24 determines whether a predetermined threshold time has elapsed from the time when the recording of the cue information was instructed most recently to the time when the switch 12 was operated. If not, the controller 24 shifts the processing to step S4. If the elapsed time has elapsed, the operator's operation is not an operation for instructing reproduction from the recording position of the cue information, but an instruction for time-shift reproduction of the endoscopic image recorded a predetermined time ago. Is determined, and the process proceeds to step S12.

  In step S12, the controller 24 determines whether or not time shift reproduction is already being performed. If the time shift reproduction is not being performed, the controller 24 gives an instruction to the recording / reproducing device 22 in step S13 to obtain a reproduction start time a predetermined time before based on the number-of-times determination in step S2, and provides information on the specified time. Is stored in a memory (not shown).

  If the controller 24 determines in step S12 that time-shift reproduction is already being performed, the controller 24 determines in step S14 a predetermined time before the predetermined time based on the number-of-times determination in step S2 based on the stored time. An instruction is given to the recording / reproducing device 22 to obtain the reproduction start time, and information on the specified time is stored in a memory (not shown).

  Next, the operation of the embodiment configured as described above will be described with reference to FIGS. 8A and 8B. 8A and 8B are described by the same method as in FIG. 4, and FIG. 8A is an example in which the time from the recording of the latest cue information to the operation of the switch 12 is within a predetermined threshold time. FIG. 8B shows an example in which the time from the recording of the latest cue information to the operation of the switch 12 exceeds a predetermined threshold time.

  Now, as a result of operating the foot switch 15 and the like to output a state signal from the electric scalpel device 23 to the controller 24, the cueing information is recorded at the timing shown in FIG. 8A and the switch 12 is operated. . After determining the number of times the switch 12 has been operated in steps S2 and S3, the controller 24 determines in step S11 whether the time from the recording of the latest cue information to the operation of the switch 12 exceeds a predetermined threshold time. Is determined. Now, it is assumed that 30 seconds is set as the predetermined threshold time. The information on the predetermined threshold time is stored in a memory (not shown), and the user can appropriately set the predetermined threshold time by performing an input operation on the operation panel of the controller 24.

  In the example of FIG. 8A, the time T0 from the recording of the latest cue information H2 to the operation of the switch 12 is 10 seconds, and this time T0 does not exceed a predetermined threshold time of 30 seconds. In this case, the controller 24 shifts the processing from step S11 to step S4, and selects and stores the cueing information according to the number of times the switch 12 is operated. That is, in this case, similarly to the first embodiment, the time-shift reproduction is started from the frame corresponding to the recording timing of the cue information in response to the operation of the switch 12. Thus, also in this case, when the switch 12 is operated, the two screens of the live image and the time shift reproduction image are changed from the single screen display of the live image displayed substantially over the entire display screen 30a of the monitor 30. Switch to display.

  Next, by operating the foot switch 15 or the like, as a result of outputting a state signal from the electric scalpel device 23 to the controller 24, the cueing information is recorded at the timing shown in FIG. 8B and the switch 12 is operated. I do. In the example of FIG. 8B, the time T0 from the recording of the latest cueing information H1 to the operation of the switch 12 is 5 minutes, which exceeds the predetermined threshold time of 30 seconds. In this case, the controller 24 shifts the processing from step S11 to step S12, and determines whether or not the time shift operation has already been performed.

  If time shift reproduction has not been performed at the time of operation of the switch 12, the process proceeds to step S13. In this case, the controller 24 specifies the reproduction start time before the predetermined time T1 according to the number of times to the recording / reproduction device 22, and stores the information of the specified time in the memory. For example, the controller 24 sets T1 = predetermined time T1f (for example, 5 seconds) × number of times as the predetermined time T1 corresponding to the number of times. The predetermined time T1f can be changed by the operation panel of the controller 24. For example, if it is detected in steps S2 and S3 that the surgeon has operated the switch 12 only once, the controller 24 records in step S13 the time 5 seconds before the time when the switch 12 was operated. The time shift reproduction is started from the frame of the endoscope image. In the example of FIG. 8, the frame recorded five seconds before is frame fm + 1, and the endoscope image after frame fm + 1 is time-shifted and reproduced.

  For example, it is assumed that the operator operates the switch 12 at the timing of the frame fp + 1, and the two-screen display of the live image and the time-shift reproduction image is displayed with a delay of one frame period. In this case, the live image of the frame fp + 1 displayed in the live image display area 30b is reduced and displayed in the live image display area 31 in the next frame fp + 2, and at the same time, the frame fm + 1 is displayed in the time shift reproduced image display area 32. Is displayed in a reduced size. The time-shifted reproduced image is, for example, displayed as a moving image for 3 seconds, and then returns to frame fm + 1 and is repeatedly reproduced and displayed. Note that the reproduction method may be configured to be separately specifiable.

  On the other hand, when the time shift reproduction is being performed at the time of the operation of the switch 12, the process proceeds from step S12 to step S14. In this case, based on the stored time, the controller 24 specifies the reproduction start time before the predetermined time T2 corresponding to the number of times to the recording / reproducing device 22, and stores the information of the specified time in the memory. . For example, the controller 24 sets T2 = predetermined time T2f (for example, 3 seconds) × number of times as the predetermined time T2 corresponding to the number of times in this case. The predetermined time T2f can be changed by the operation panel of the controller 24. For example, if it is detected in steps S2 and S3 that the surgeon has operated the switch 12 only once, the controller 24 determines in step S14 the time recorded as the reproduction start position of the previous time-shift reproduction. Time-shift reproduction is started from the frame of the endoscopic image recorded three seconds before. In the example of FIG. 8, the frame recorded three seconds before is frame fn + 2, and the endoscope image after frame fn + 2 is time-shifted and reproduced.

  For example, when the operator performs the second operation of the switch 12 once, the current live image is displayed in the live image display area 31 and at the same time, the time starting from the frame fn + 2 is displayed in the time shift reproduction image display area 32. The shift reproduction image is displayed in a reduced size. The time-shifted reproduced image is displayed, for example, as a moving image for 3 seconds, then returns to frame fn + 2, and is repeatedly reproduced and displayed. Note that the reproduction method may be configured to be separately specifiable.

  Other operations are the same as those of the first embodiment.

  As described above, in the present embodiment, the same effects as those of the first embodiment can be obtained, and the time from the recording of the latest cue information to the operation of the switch for designating the time shift reproduction is determined by the predetermined time. If the time exceeds the threshold time, the surgeon determines that reproduction from a position other than the recording position of the cue information is desired, and performs time-shift reproduction from a predetermined time before the switch operation. It has become. Thereby, the operator can reproduce from a desired reproduction position. Moreover, in the present embodiment, a common operation unit may be operated in a case where time shift reproduction is performed from a position corresponding to the recording position of the cue information and a case where time shift reproduction is performed from a desired position. It has the advantage of excellent operability.

  Note that, in the first embodiment, an example has been described in which a live image is displayed on one screen in the normal observation mode, and a two-screen display of a live image and a time-shift reproduction image is performed when the mode shifts to the time-shift observation mode. In the second embodiment, the transition to the time-shift observation mode is not limited to the case where the two-screen display of the live image and the time-shift reproduction image is necessarily performed, and the time-shift reproduction image is also displayed in the time-shift observation mode. Screen display may be performed.

  The present invention is not limited to each of the above embodiments as it is, and can be embodied by modifying constituent elements in an implementation stage without departing from the scope of the invention. Various inventions can be formed by appropriately combining a plurality of constituent elements disclosed in the above embodiments. For example, some components of all the components shown in the embodiment may be deleted. Further, components of different embodiments may be appropriately combined.

    DESCRIPTION OF SYMBOLS 10 ... Endoscope system, 11 ... Endoscope, 11a ... Insertion part, 11b ... Operation part, 12 ... Switch, 14 ... Electric mess scissors, 15 ... Foot switch, 21 ... Video processor, 22 ... Recording / reproducing apparatus, 23 ... Electrocautery device, 24 controller, 30 monitor, 30a display screen, 31 live image display area, 32 time shift reproduction image display area.

Claims (9)

A display unit that displays an endoscope image captured by the endoscope,
While recording the endoscope image, when a signal regarding a treatment timing of a treatment tool for treating a subject is given, cueing information is recorded in association with a recording position on the time axis of the endoscopic image to be recorded. A recording unit;
When a user operation is detected, the endoscope image recorded in the recording unit is time-shifted and reproduced from the recording position related to the cue information, and the display unit is controlled by the endoscope. A control unit configured to switch from a one-screen display in which an endoscopic image during imaging is a live image to a two-screen display of the live image and a time-shift reproduction image by the time-shift reproduction;
Equipped with,
The control unit, when a plurality of pieces of the cue information are recorded, selects one of the plurality of pieces of the cue information recorded based on a user operation and determines a starting point of the time shift reproduction.
An endoscope system characterized in that:   A common operation unit for performing a user operation for shifting to the time-shift reproduction and a user operation for selecting one of the cue information.
  The endoscope system according to claim 1, further comprising:   When the time difference between the recording timing of the cueing information and the timing of the user operation is within a predetermined threshold time, the control unit performs time-shift reproduction with the recording position related to the cueing information as a starting point. If the time difference between the recording timing of the cue information and the timing of the user operation exceeds a predetermined threshold time, time-shift playback is performed starting from a timing that is a predetermined time before the timing of the user operation.
  The endoscope system according to claim 1, wherein:   A display unit that displays an endoscope image captured by the endoscope,
  While recording the endoscope image, when a signal regarding a treatment timing of a treatment tool for treating a subject is given, cueing information is recorded in association with a recording position on the time axis of the endoscopic image to be recorded. A recording unit;
  When a user operation is detected, the endoscope image recorded in the recording unit is time-shifted and reproduced from the recording position related to the cue information, and the display unit is controlled by the endoscope. A control unit configured to switch from a one-screen display in which an endoscopic image during imaging is a live image to a two-screen display of the live image and a time-shift reproduction image by the time-shift reproduction;
  With
  When the time difference between the recording timing of the cueing information and the timing of the user operation is within a predetermined threshold time, the control unit performs time-shift reproduction with the recording position related to the cueing information as a starting point. If the time difference between the recording timing of the cue information and the timing of the user operation exceeds a predetermined threshold time, time-shift playback is performed starting from a timing that is a predetermined time before the timing of the user operation.
  An endoscope system characterized in that:   A display unit that displays an endoscope image captured by the endoscope,
  While recording the endoscope image, when a signal regarding a treatment timing of a treatment tool for treating a subject is given, cueing information is recorded in association with a recording position on the time axis of the endoscopic image to be recorded. A recording unit;
  When a user operation is detected, the endoscope image recorded in the recording unit is time-shifted and reproduced from the recording position related to the cue information, and the display unit is controlled by the endoscope. A control unit configured to switch from a one-screen display in which an endoscopic image during imaging is a live image to a two-screen display of the live image and a time-shift reproduction image by the time-shift reproduction;
  With
  The control unit, when a time difference between the recording timing of the cue information and the timing of the user operation exceeds a predetermined threshold time, performs time-shift reproduction starting from a timing that is a predetermined time before the timing of the user operation. Do
  An endoscope system characterized in that:   If the time difference between the recording timing of the cue information and the timing of the user operation exceeds a predetermined threshold time, the control unit determines the starting point of the time-shift reproduction according to the number of times of the user operation.
  The endoscope system according to any one of claims 3 to 5, wherein:   A display recording step of displaying an endoscope image captured by an endoscope on a display unit as a live image, and recording the endoscope image in a recording unit,
  When a signal relating to the treatment timing of the treatment tool for treating the subject is given, an information recording step of recording cueing information in association with a recording position on the time axis of the endoscope image in the display recording step,
  When a user operation is detected, the endoscope image recorded in the recording unit is time-shifted and reproduced from the recording position related to the cue information, and the display unit is controlled by the endoscope. A reproduction display step of switching from a one-screen display in which an endoscope image during imaging is a live image to a two-screen display of the live image and a time-shift reproduction image by the time-shift reproduction;
  With
  In the reproduction display step, when a plurality of pieces of the cue information are recorded, one of the plurality of pieces of the cue information recorded is selected based on a user operation to determine a starting point of the time shift reproduction.
  A display method for an endoscope system, comprising:   A display recording step of displaying an endoscope image captured by an endoscope on a display unit as a live image, and recording the endoscope image in a recording unit,
  When a signal relating to the treatment timing of the treatment tool for treating the subject is given, an information recording step of recording cueing information in association with a recording position on the time axis of the endoscope image in the display recording step,
  When a user operation is detected, the endoscope image recorded in the recording unit is time-shifted and reproduced from the recording position related to the cue information, and the display unit is controlled by the endoscope. A reproduction display step of switching from a one-screen display in which an endoscope image during imaging is a live image to a two-screen display of the live image and a time-shift reproduction image by the time-shift reproduction;
  With
  The reproduction display step includes, if a time difference between the recording timing of the cue information and the timing of the user operation is within a predetermined threshold time, performs time-shift reproduction with the recording position related to the cue information as a starting point. When the time difference between the recording timing of the cue information and the timing of the user operation exceeds a predetermined threshold time, time-shift reproduction is performed starting from a timing that is a predetermined time before the timing of the user operation.
  A display method for an endoscope system, comprising:   A display recording step of displaying an endoscope image captured by an endoscope on a display unit as a live image, and recording the endoscope image in a recording unit,
  When a signal relating to the treatment timing of the treatment tool for treating the subject is given, an information recording step of recording cueing information in association with a recording position on the time axis of the endoscope image in the display recording step,
  When a user operation is detected, the endoscope image recorded in the recording unit is time-shifted and reproduced from the recording position related to the cue information, and the display unit is controlled by the endoscope. A reproduction display step of switching from a one-screen display in which an endoscope image during imaging is a live image to a two-screen display of the live image and a time-shift reproduction image by the time-shift reproduction;
  With
  When the time difference between the recording timing of the cue information and the timing of the user operation exceeds a predetermined threshold time, the reproduction display step performs time-shift reproduction starting from a timing that is a predetermined time before the timing of the user operation. I do
  A display method for an endoscope system, comprising:

Images