JP7126673B2 - Endoscope image processing program and endoscope system - Google Patents

Description

The present invention relates to an endoscope image processing program, an endoscope system, and an endoscope image processing method.

In order to assist the operation of a medical endoscope, for example, in the methods described in Patent Documents 1 and 2, a three-dimensional medical image acquired before conducting an examination is matched with the positional information of the tip of the endoscope. Therefore, it is possible to support easy and safe insertion of an endoscope into a hollow organ having a complicated shape. Such navigation of medical device operation for doctors using pre-operative and pre-examination three-dimensional image data has been proposed as various applications for the purpose of examination and surgery.

JP 2005-131317 A JP 2005-211525 A

Techniques for guiding medical devices into the body, such as conventional techniques, are based on the premise that three-dimensional anatomical data of a patient can be acquired in advance. On the other hand, from the viewpoint of medical resources and medical economy, the frequent use of three-dimensional medical images is not clinically highly compatible.

In addition, when estimating the three-dimensional shape of an organ using only endoscope position information and images, it is difficult to predict the target shape for organs that have branches (e.g., trachea, blood vessels, renal pelvis, etc.). is. For example, it is necessary to acquire more morphological characteristics of target organs by identifying bifurcations in endoscopic images, separating and tracking the lumens in the direction of insertion and the lumens that branch out. be.

An object of the present invention is to solve the above problems, and to provide an endoscopic image processing method and an endoscopic system that enable identification and tracking of bifurcations such as luminal tissue. is.

In order to solve the above problems, according to one aspect of the present invention, an endoscope image processing program is provided in a computer, an endoscope image acquisition step of acquiring an image of the inside of a hollow organ with an endoscope; a bifurcation extracting step of extracting a bifurcation in the hollow organ based on the image of the hollow organ acquired in the endoscopic image acquisition step; An endoscope image processing program for executing an identification information adding step for adding identification information. Furthermore, it is desirable that the bifurcation extracting step extracts the bifurcation in the hollow organ based on information on the brightness of the image in the hollow organ acquired in the endoscopic image acquisition step. Also, comparing the branch information extracted by the branch extracting step for the image acquired in the previous time period and the branch information extracted by the branch extracting step for the image acquired in the later time period. a tracking step of obtaining a correspondence between the bifurcation extracted by the bifurcation extracting step for the image acquired in the preceding time period and the bifurcation extracted by the bifurcation extraction step for the image acquired in the subsequent time period by It is recommended to run

According to another aspect of the present invention, an endoscope system comprises an endoscope for obtaining an image of the inside of a hollow organ, and an image of the inside of the hollow organ obtained by the endoscope. and an identification information providing unit for providing identification information to the bifurcation in the hollow organ extracted by the bifurcation extracting unit. Further, comparing the branch information extracted by the branch extraction unit for the image acquired in the previous time period with the branch information extracted by the branch extraction unit for the image acquired in the later time period. A tracking unit that obtains a correspondence between the bifurcation extracted by the bifurcation extraction unit for the image acquired in the previous time period and the bifurcation extracted by the bifurcation extraction unit for the image acquired in the subsequent time period. shall be provided.

According to another aspect of the present invention, an endoscopic image processing method comprises: an endoscopic image obtaining step of obtaining an image inside a hollow organ with an endoscope; a bifurcation extraction step of extracting a bifurcation in the hollow organ based on the image of the inside of the hollow organ obtained; and an identification information adding step of adding identification information to the bifurcation in the hollow organ extracted by the bifurcation extraction step. and

According to the present invention, it is possible to provide an endoscopic image processing method and an endoscopic system that enable identification and tracking of bifurcations in luminal tissue.

It is a figure which shows the branch part extraction of a present Example, and the algorithm of a trace. It is a figure which shows the endoscope image|video in a model, and a bifurcation part extraction result. It is a figure which shows the branch part extracted.

Examples of the present invention will be described below, but embodiments of the present invention are not limited to the examples described below.

The present invention is a dynamic image processing algorithm that enables identification and tracking of bifurcations from images of a moving camera within a luminal structure with multiple bifurcations.

An embodiment of the present invention comprises an endoscope imaging system inserted into a hollow organ having branches, an image processing device that receives images from the system and performs necessary processing, and a visualization device that presents the processing results. video processing system.

The algorithm of the invention is presented in FIG. The algorithm includes steps 101 of extracting potential organ bifurcations from a single endoscopic image, 102 of tracking the bifurcations in a sequence of images as the endoscope moves, and displaying the tracking results. 103.

In the bifurcation extraction step 101, a region that is highly likely to be a bifurcation is extracted from the brightness, saturation, secondary differential value, etc. in the endoscopic image, and labeling processing (identification is performed on the region extracted as the bifurcation) process for assigning identification information such as a number). At this time, it is desirable to remove unnecessary information such as noise by deleting labels having a small area.

Each extracted bifurcation is given an ID and status, and by tracking each bifurcation, a branch in the direction of travel of the endoscope (Master branch) and a branch that becomes an unobserved area (Sub branch) It is characterized by being able to separate the sub-branch) and identify when the endoscope enters the sub-branch.

In the trace step 102, first, the Branches data (a structure (branch) array storing branch information) is initialized. Each branch is given a status variable (master, sub, lost, etc.). Among the blobs extracted by labeling, the one closest to the center of gravity of the master branch in the previous image is taken as the master branch of the current frame. Search for continuous blobs while referring to the sub branch of the previous image. "Continuity" is judged by the distance between the centroids of the blobs.

In step 103 for displaying the tracking results, only sub branches are extracted from the Branches data. Furthermore, it is limited to Branches detected continuously for 10 frames or more. A scan line for generating a panorama image and an overlapping portion of each blob are marked as a branch portion (given identification information).

FIG. 2 is a diagram showing an endoscopic image and branch point extraction results in a model. 201 is a master branch (main branch) and 202 is a sub branch.

FIG. 3 is a diagram showing extracted branching portions and branching points. The upper image in FIG. 3 is the original image, and the lower image in FIG. 3 is the result of extracting the branching portion from the original image, which is the extracted branching portion marked in green. Reference numeral 301 denotes an image in which the extracted bifurcation is processed to make it easier to see.

The hollow organ having a bifurcation in this embodiment includes, for example, the trachea, blood vessels, renal pelvis, etc., and is expected to be applied to education of endoscope operation and diagnosis/treatment support technology.

According to this embodiment, even when diagnosing or performing surgery on a hollow organ having many bifurcations using an endoscope, it is possible to diagnose already observed bifurcations and unobserved organs that have not yet been observed. can be easily recognized and diagnosed by a person. This has the effect of improving the efficiency of surgery and preventing misdiagnosis.

INDUSTRIAL APPLICABILITY The present invention is industrially applicable as an endoscopic image processing program, an endoscopic system, and an endoscopic image processing method.

101 step of extracting branch candidates 102 step of tracing branch 103 step of displaying trace result 201 master branch 202 sub-branches 301 branch

Claims (3)

an endoscopic image acquisition step of acquiring an image inside a hollow organ using an endoscope; a bifurcation extracting step of extracting a bifurcation, an identification information adding step of adding identification information to the bifurcation in the hollow organ extracted by the bifurcation extracting step, and an image acquired in a previous time period for the bifurcation By comparing the information on the bifurcation extracted by the extracting step and the information on the bifurcation extracted by the bifurcation extracting step for the image acquired in the later time period, the image obtained in the previous time period is Endoscope image processing for executing a tracking step of obtaining a correspondence between the bifurcation extracted by the bifurcation extraction step and the bifurcation extracted by the bifurcation extraction step with respect to an image acquired in the subsequent time zone program. 2. The method according to claim 1, wherein in said bifurcation extracting step, the bifurcation within said hollow organ is extracted based on information relating to the brightness of the image within said hollow organ obtained in said endoscope image obtaining step. endoscopic image processing program. An endoscope that acquires an image inside a hollow organ, a bifurcation extraction unit that extracts a bifurcation within the hollow organ based on the image of the interior of the hollow organ acquired by the endoscope, and the bifurcation. an identification information adding unit that adds identification information to the bifurcation in the hollow organ extracted by the extracting unit; By comparing the information of the branch extracted by the branch extraction unit for the image acquired in the time period, the branch extracted by the branch extraction unit and the An endoscope system comprising a tracking unit that obtains the correspondence of the bifurcation extracted by the bifurcation extraction unit for images acquired in a later time period.

Images