JP7315809B2 - Colonoscopy Auxiliary Apparatus, Colonoscopy Auxiliary Method and Colonoscopy Auxiliary Program - Google Patents

Description

The present invention relates to a colonoscopy assisting device, a colonoscopy assisting method, and a colonoscopy assisting program.

In recent years, colonoscopies have been performed by doctors as a screening test for the large intestine. This colonoscopy is an examination for detecting polyps present in the large intestine, etc., by using dedicated medical equipment.

In colonoscopy as described above, a doctor visually confirms image data and the like to diagnose the presence or absence of polyps (hereinafter also referred to as image diagnosis). Therefore, the accuracy of polyp detection may be affected by the experience of the doctor, the degree of fatigue, and the like.

Therefore, in recent years, image diagnosis support has been provided by utilizing AI (Artificial Intelligence). This enables doctors to improve the accuracy of polyp detection in colonoscopy (see Non-Patent Documents 1 and 2).

Jikei University and others press release August 17, 2018 https://eirl.ai/ja/news/2018/08/17/247/ National Cancer Center, etc. Press Release July 10, 2017 https://www.ncc.go.jp/jp/information/pr_release/2017/0710/press_release_20170710.pdf

Here, the support for image diagnosis by AI as described above is performed by, for example, displaying polyp candidates (hereinafter simply referred to as polyp candidates) included in the image data together on the image data. This reduces the burden on doctors.

However, if, for example, many structures other than polyps are displayed as polyp candidates, the number of image data that the doctor needs to check intensively increases, and as a result, the burden on the doctor cannot be reduced. . Therefore, in recent years, there has been a demand for a new method capable of more accurately identifying polyp candidates contained in image data.

Accordingly, an object of the present invention is to provide a colonoscopy assisting device, a colonoscopy assisting method, and a colonoscopy assisting program that enable more accurate identification of polyp candidates contained in image data. is to provide

A colonoscopy assisting device in the present invention for achieving the above object is a colonoscopy assisting device for detecting polyps in the large intestine by performing machine learning using image data, comprising a plurality of a first learning data generation unit for generating a plurality of learning data by adding identification information corresponding to the polyp contained in each image data to each of the image data; A first machine learning unit that generates a first learning model by performing machine learning using a plurality of first learning data, and the polyp included in a plurality of second learning data among the plurality of learning data Image data included in the plurality of first learning data based on a comparison result between the corresponding identification information and the identification information output from the first learning model in response to the input of the plurality of second learning data. a plurality of third learning data by specifying a structure other than the polyp from each and adding identification information corresponding to the specified structure other than the polyp to each of the image data included in the plurality of first learning data; and a second machine learning unit that generates a second learning model by performing machine learning using the plurality of third learning data. .

In addition, a colonoscopy assisting method in the present invention for achieving the above object is a colonoscopy that causes a computer to execute processing for detecting polyps in the colon by performing machine learning using image data. An inspection assistance method, wherein identification information corresponding to the polyp included in each image data is added to each of a plurality of image data to generate a plurality of learning data, and among the plurality of learning data, generating a first learning model by performing machine learning using a plurality of first learning data, and identification information corresponding to the polyp included in a plurality of second learning data among the plurality of learning data; and and identifying information other than the polyp from each of the image data included in the plurality of first learning data based on a comparison result with the identification information output from the first learning model in response to the input of the plurality of second learning data. generating a plurality of third learning data by identifying the structure of the polyp and adding identification information corresponding to the identified structure other than the polyp to each of the image data included in the plurality of first learning data; A second learning model is generated by performing machine learning using a plurality of third learning data.

In addition, a colonoscopy assistance program in the present invention for achieving the above object is a colonoscopy program that causes a computer to execute processing for detecting polyps in the colon by performing machine learning using image data. a complementary program for generating a plurality of learning data by adding identification information corresponding to the polyp contained in each image data to each of a plurality of image data; Identification information corresponding to the polyp that generates a first learning model by performing machine learning using a plurality of first learning data, and is included in a plurality of second learning data among the plurality of learning data; Based on the result of comparison with the identification information output from the first learning model in response to the input of the plurality of second learning data, from each of the image data included in the plurality of first learning data generating a plurality of third learning data by identifying a structure and adding identification information corresponding to the identified structure other than the polyp to each of the image data included in the plurality of first learning data; The second learning model is generated by performing machine learning using the third learning data.

According to the colonoscopy assisting device, colonoscopy assisting method, and colonoscopy assisting program of the present invention, it is possible to more accurately identify polyp candidates contained in image data.

FIG. 1 is a diagram showing a configuration example of a colonoscopy assisting apparatus 1 according to an embodiment of the present invention. FIG. 2 is a diagram for explaining a specific example of colonoscopy assistance processing. FIG. 3 is a diagram for explaining a specific example of polyp identification information. FIG. 4 is a diagram for explaining a specific example of polyp identification information. FIG. 5 is a flowchart for explaining an outline of colonoscopy assisting processing according to the first embodiment. FIG. 6 is a flowchart for explaining an outline of colonoscopy assisting processing according to the first embodiment. FIG. 7 is a flowchart for explaining details of the colonoscopy assisting process according to the first embodiment. FIG. 8 is a flowchart for explaining details of the colonoscopy assisting process according to the first embodiment. FIG. 9 is a flowchart for explaining details of the colonoscopy assisting process according to the first embodiment. FIG. 10 is a diagram illustrating details of the colonoscopy assisting process according to the first embodiment. FIG. 11 is a diagram illustrating details of the colonoscopy assisting process according to the first embodiment. FIG. 12 is a diagram illustrating details of the colonoscopy assisting process according to the first embodiment. FIG. 13 is a diagram illustrating details of the colonoscopy assisting process according to the first embodiment. FIG. 14 is a diagram illustrating details of the colonoscopy assisting process according to the first embodiment. FIG. 15 is a diagram illustrating details of the colonoscopy assisting process according to the first embodiment.

BEST MODE FOR CARRYING OUT THE INVENTION Hereinafter, embodiments of the present invention will be described with reference to the drawings. However, such an embodiment does not limit the technical scope of the present invention.

FIG. 1 is a diagram showing a configuration example of a colonoscopy assisting apparatus 1 according to an embodiment of the present invention. The colonoscopy assisting device 1 is a computer device, and may be, for example, a general-purpose PC (Personal Computer).

The colonoscopy assisting device 1 has a hardware configuration of a general-purpose computer device. For example, as shown in FIG. and Each unit is connected to each other via a bus 105 . Note that the CPU 101 may operate in cooperation with a GPU (Graphics Processing Unit) (not shown).

The storage medium 104 is, for example, a program storage area (not shown) for storing a program (not shown) for performing a process for assisting a doctor's colonoscopy (hereinafter also referred to as colonoscopy assisting process). ).

The storage medium 104 also has a storage unit 110 (hereinafter also referred to as a storage area 110) that stores information used when performing colonoscopy assisting processing, for example. Specifically, the storage medium 104 stores, for example, a plurality of image data for diagnosis by a doctor in colonoscopy.

Note that the storage medium 104 may be, for example, an HDD (Hard Disk Drive) or an SSD (Sokid State Drive).

The CPU 101 executes a program loaded from the storage medium 104 to the memory 102 to perform colonoscopy assisting processing.

The network interface 103 communicates with, for example, the operation terminal 2 operated by the operator. The network interface 103 also communicates with the output device 3 (the output device 3 that displays image data) that is viewed by a doctor when performing colonoscopy, for example.

[Specific example of colonoscopy assisting process]
Next, a specific example of colonoscopy assistance processing will be described. FIG. 2 is a diagram for explaining a specific example of colonoscopy assistance processing. Specifically, FIG. 2A shows a specific example in which one sheet of image data IM11 to be diagnosed is displayed on the output device 3, and FIG. This is a specific example when the other one of them is displayed on the output device 3 . FIG. 2C is a specific example of displaying identification information corresponding to a polyp together with image data IM11, and FIG. This is a specific example when is displayed.

The image data IM11 shown in FIG. 2A shows the polyp candidate PO11, and the image data IM12 shown in FIG. 2B shows the polyp candidates PO12 and PO13. Note that the image data IM11 and the image data M12 will also be collectively referred to as image data IM hereinafter. Further, hereinafter, the polyp candidates PO11, PO12 and PO13 are also collectively referred to as polyp candidates PO.

First, the doctor diagnoses whether each of the polyp candidates PO11, PO12, and PO13 is a polyp by referring to the image data IM11 and IM12. Specifically, the doctor determines whether each polyp candidate PO is an adenoma, hyperplasia, SSA/P (Sessile Serrated Adenoma/Polyp), or LST (Laterally spreading Tumor). I do.

Then, for example, the doctor diagnoses that the polyp candidate PO11 is an adenoma, which is a type of polyp, and that the polyp candidate PO12 is a hyperplasia, which is a type of polyp. On the other hand, the doctor, for example, diagnoses that the polyp candidate PO13 is a structure other than a polyp.

After that, the colonoscopy assisting apparatus 1 generates identification information (annotations) of polyp candidates PO11 and PO12 diagnosed as polyps based on, for example, diagnostic results input by a doctor.

Specifically, the colonoscopy assisting apparatus 1, for example, uses type information ID11a indicating that the polyp candidate PO11 is an adenoma as identification information corresponding to the polyp candidate PO11 included in the image data IM11, and the image data IM11 position information ID 11b indicating the coordinates of the polyp candidate PO 11 in .

Further, the colonoscopy assisting apparatus 1, for example, uses type information ID12a indicating that the polyp candidate PO12 is hyperplasia as identification information corresponding to the polyp candidate PO12 included in the image data IM12, and Information including location information ID 12b indicating the coordinates of the polyp candidate PO 12 is generated. A specific example of the data structure of the identification information will be described below.

[Specific examples of polyp identification information]
3 and 4 are diagrams for explaining specific examples of polyp identification information.

The identification information shown in FIG. 3 includes "image name" in which the name of the image data IM is stored, "type information" in which polyp type information is stored, and "position information (horizontal)" in which polyp position information is stored. coordinates)”, “position information (vertical coordinates)”, “position information (horizontal length)”, and “position information (vertical length)” as items. "Position information (horizontal coordinate)" stores the coordinate corresponding to the horizontal axis among the upper left coordinates of the cursor containing the polyp. Of the upper left coordinates, the coordinates corresponding to the vertical axis are stored. "Position information (horizontal length)" stores the horizontal length (size) of the cursor including the polyp. The vertical length (size) of is stored. That is, in "Position information (horizontal coordinate)", "Position information (vertical coordinate)", "Position information (horizontal length)" and "Position information (vertical length)", the cursor including the polyp Each of the information necessary to identify the is stored.

Specifically, in the identification information shown in FIG. 3, in the first row of information (information corresponding to the polyp candidate PO11 in FIG. 2), "IM11" is stored as the "image name", and "IM11" is stored as the "type information". "Adenoma" is stored, "22" is stored as "position information (horizontal coordinate)", "81" is stored as "position information (vertical coordinate)", and "position information (horizontal length)" is stored. "28" is stored as "position information (vertical length)", and "44" is stored as "position information (vertical length)".

Further, in the identification information shown in FIG. 3, in the information on the second line (information corresponding to the polyp candidate PO12 in FIG. 2), "IM12" is stored as the "image name", and "hyperplasia" is stored as the "type information". ” is stored as “position information (horizontal coordinate)”, “37” is stored as “position information (vertical coordinate)”, and “69” is stored as “position information (horizontal length)”. "34" is stored, and "49" is stored as "positional information (longitudinal length)".

That is, as shown in FIGS. 2(C) and 2(D), the colonoscopy assisting apparatus 1 generates information that can be displayed together with the display of each polyp candidate PO. conduct.

After that, for example, an operator who performs the colonoscopy assisting process (hereinafter also simply referred to as an operator) performs each image data IM stored in the storage medium 104 and each image data IM. and identification information for each of the polyp candidate POs included. Specifically, the worker generates learning data including, for example, the image data IM11 and the information on the first row in the identification information described with reference to FIG. Also, the operator generates learning data including, for example, the image data IM12 and the information on the second line in the identification information described with reference to FIG. Then, the colonoscopy assisting device 1 generates a learning model by using the generated plurality of learning data.

As a result, the colonoscopy assisting apparatus 1 uses the generated learning model to determine the polyp candidates included in the new image data IM (for example, the image data IM newly captured in the colonoscopy). It becomes possible to specify the PO. Then, when displaying the new image data IM on the output device 3, the colonoscopy assisting apparatus 1 can also display the identification information corresponding to each polyp candidate PO.

Specifically, as shown in FIG. 4A, when new image data IM21 is input, the colonoscopy assisting apparatus 1 displays the image data IM21 when the polyp candidate PO21 is an adenoma. and the position information ID 21b of the polyp candidate PO 21 (cursor corresponding to the position information ID 21b) can be displayed together.

Further, as shown in FIG. 4B, when image data IM22 is input, the colonoscopy assisting apparatus 1 indicates that the polyp candidate PO22 is an adenoma when displaying the image data IM22. position information ID 22b of the candidate polyp PO 22 (cursor corresponding to the position information ID 22b); classification information ID 23a indicating that the candidate polyp PO 23 is hyperplasia; (cursor corresponding to ID 23b) can be displayed together.

As a result, when diagnosing new image data IM, the doctor can refer to the identification information that is also displayed, thereby reducing the burden associated with diagnosing each image data IM.

However, for example, when many structures other than polyps are displayed on the output device 3 as polyp candidates PO, the doctor cannot reduce the burden associated with image diagnosis. Specifically, for example, when many structures other than polyps are included in the polyp candidate POs shown in FIGS. can no longer be suppressed, and the burden associated with image diagnosis cannot be reduced. Therefore, at sites such as hospitals where colonoscopy is performed, there is a demand for a new method capable of more accurately identifying polyp candidates PO contained in image data. An outline of the colonoscopy assisting process according to the present embodiment will be described below.

[Outline of the first embodiment]
5 and 6 are flowcharts for explaining an outline of the colonoscopy assisting process according to the first embodiment.

As shown in FIG. 5, the first learning data generator of the colonoscopy assisting device 1 adds identification information corresponding to the polyp contained in each image data IM to each of the plurality of image data IM. By doing so, a plurality of learning data are generated (S11).

Then, the first machine learning unit of the colonoscopy assisting apparatus 1 performs machine learning using part of the plurality of learning data generated in the process of S11 (hereinafter also referred to as a plurality of first learning data). to generate a first learning model (S12).

Subsequently, as shown in FIG. 6, the second learning data generation unit of the colonoscopy assisting apparatus 1 generates, for example, another part of the plurality of learning data generated in the process of S11 (hereinafter referred to as a plurality of learning data). (also referred to as second learning data), the identification information output from the first learning model generated in the process of S12 is acquired (S21).

Then, the second learning data generation unit of the colonoscopy assisting apparatus 1 generates the identification information corresponding to the polyp included in the plurality of second learning data generated in the process of S11 and the identification information acquired in the process of S21. structure other than the polyp is identified from the image data IM included in the plurality of first learning data generated in the process of S11 (S22). Structures other than polyps include, for example, at least one of a blister, intestinal contents (eg, stool, etc.), ileocecal valve, and colonic lumen. Structures other than polyps also include, for example, light reflecting regions.

Furthermore, the second learning data generation unit of the colonoscopy assisting device 1 generates structures other than the polyp identified in the process of S22 for the image data IM included in the plurality of first learning data generated in the process of S11. By further adding identification information corresponding to , a plurality of third learning data are generated (S23).

After that, the second machine learning unit of the colonoscopy assisting device 1 generates a second learning model by performing machine learning using the plurality of third learning data generated in the process of S23 (S24). .

That is, the colonoscopy assisting apparatus 1 uses not only learning data containing identification information corresponding to polyps, but also learning data containing identification information corresponding to structures other than polyps, so that the learning model (second learning model ).

As a result, the colonoscopy assisting apparatus 1 can more accurately identify the polyp candidates PO included in the image data IM captured during the colonoscopy. Therefore, it becomes possible for a doctor to reduce the burden associated with colonoscopy.

[Details of the first embodiment]
Next, colonoscopy assistance processing in the first embodiment will be described. 7 to 9 are flowcharts for explaining the details of colonoscopy assistance processing according to the first embodiment. 10 to 15 are diagrams for explaining the details of colonoscopy assistance processing according to the first embodiment.

As shown in FIG. 7, the colonoscopy assisting apparatus 1 acquires a combination of type information and position information corresponding to each of the plurality of image data IM from the identification information 131 stored in the storage area 110 ( S31). A specific example of the identification information 131 will be described below.

[Specific example of identification information]
FIG. 10 is a diagram for explaining a specific example of the identification information 131. As shown in FIG. Specifically, FIG. 10A is a diagram illustrating a specific example of the identification information 131a included in the first learning data. Also, FIG. 10B is a diagram illustrating a specific example of the identification information 131b included in the second learning data. Note that the identification information 131 shown in FIG. 10 has, for example, the same items as the identification information described in FIG.

Specifically, in the identification information 131a shown in FIG. 10A, in the information on the first line, "IM101" is stored as the "image name", "adenoma" is stored as the "type information", and "location information" is stored. "42" is stored as "information (horizontal coordinate)", "71" is stored as "positional information (vertical coordinate)", "42" is stored as "positional information (horizontal length)", "36" is stored as "positional information (vertical length)".

In the identification information 131a shown in FIG. 10A, in the information on the second line, "IM102" is stored as the "image name", "adenoma" is stored as the "type information", and "position information ( "29" is stored as "horizontal coordinate)", "89" is stored as "position information (vertical coordinate)", "25" is stored as "position information (horizontal length)", and "position "22" is stored as "information (vertical length)".

Further, in the identification information 131a shown in FIG. 10A, in the information on the third line, "IM102" is stored as the "image name", "hyperplasia" is stored as the "type information", and "position information" is stored. "83" is stored as "(horizontal coordinate)", "52" is stored as "position information (vertical coordinate)", "8" is stored as "position information (horizontal length)", and " "9" is stored as position information (vertical length). Description of other information included in FIG. 10A is omitted.

That is, the identification information 131a shown in FIG. 10A includes, for example, one polyp (polyp corresponding to the information on the first row) included in the image data IM whose "image name" is "IM101". indicates that the doctor has determined that Further, the identification information 131a shown in FIG. 10A includes, for example, two polyps (polyp ) indicates that the doctor has determined that it contains

Further, in the identification information 131b shown in FIG. 10B, in the information on the first line, "IM111" is stored as the "image name", "hyperplasia" is stored as the "type information", and "position information" is stored. "19" is stored as "(horizontal coordinate)", "42" is stored as "position information (vertical coordinate)", "12" is stored as "position information (horizontal length)", and " "5" is stored as position information (vertical length).

Further, in the identification information 131b shown in FIG. 10B, in the information on the second line, "IM111" is stored as the "image name", "adenoma" is stored as the "type information", and "position information ( "86" is stored as "horizontal coordinate)", "39" is stored as "position information (vertical coordinate)", "9" is stored as "position information (horizontal length)", and "position "11" is stored as information (vertical length). Description of other information included in FIG. 10B is omitted.

That is, the identification information 131a shown in FIG. 10B includes, for example, two polyps (polyp ) indicates that the doctor has determined that it contains

Then, in the process of S31, the colonoscopy assisting apparatus 1 selects, for example, "type information", "position information (horizontal coordinates)", "position Acquire a combination of information (vertical coordinate), position information (horizontal length), and position information (vertical length).

Returning to FIG. 7, the colonoscopy assisting apparatus 1 adds a combination of the type information and the position information acquired in the process of S31 to each of the plurality of image data IM, thereby obtaining a plurality of learning data. Generate (S32).

Then, the colonoscopy assisting device 1 generates a first learning model by performing machine learning using the plurality of first learning data generated in the process of S32 (S33).

That is, the colonoscopy assisting device 1 generates the first learning model by learning a part of the plurality of learning data generated in the process of S32.

After that, the operator verifies the first learning model by using, for example, a plurality of learning data other than the first learning data and the second learning data among the plurality of learning data generated in the process of S32. conduct.

Subsequently, as shown in FIG. 8, the colonoscopy assisting apparatus 1 outputs from the first learning model generated in the process of S32 according to the input of the plurality of second learning data generated in the process of S31. A combination of the type information and the position information to be received is acquired (S41).

Then, the colonoscopy assisting apparatus 1 identifies, among the combinations obtained in the process of S41, combinations that are not included in the combinations corresponding to the polyps included in the plurality of second learning data generated in the process of S31. (S42). A specific example of the processing of S42 will be described below.

[Specific example of processing in S42]
FIG. 11 is a specific example of a combination of type information and position information acquired in the process of S41.

In addition to the information included in the identification information 131b described in FIG. 10B, the information shown in FIG. 11 includes, for example, information on the third line and information on the sixth line (information corresponding to the underlined portion). is Therefore, in the process of S42, the colonoscopy assisting apparatus 1 specifies, for example, the information on the third line and the information on the sixth line among the information shown in FIG. 11 .

As a result, the colonoscopy assisting apparatus 1 identifies information corresponding to structures other than polyps that the first learning model has determined to be polyps (that is, structures that the first learning model has misidentified as polyps). it becomes possible to

In addition, in the processing of S42, the colonoscopy assisting apparatus 1 determines that the information matches when the difference in the position information included in each combination to be compared is within a predetermined range. may

Returning to FIG. 8, the colonoscopy assisting apparatus 1 generates partial image data corresponding to the combination specified in the process of S42 from each of the image data IM included in the plurality of second learning data generated in the process of S31. (hereinafter also referred to as first partial image data) is specified (S43).

In this case, the doctor identifies correct type information corresponding to the combination identified in the process of S42 by referring to the first partial image data identified in the process of S43. Then, the colonoscopy assisting device 1 corrects the information included in the combination specified in the process of S42 based on the type information specified by the doctor.

Specifically, for example, if the correct type information corresponding to each of the information on the third line and the information on the sixth line among the information described in FIG. As shown in the underlined portion of FIG. 12, the auxiliary device 1 changes the information set in the "type information" in the information on the third line to "blister" and sets it in the "type information" in the information on the sixth line. Change the information provided to "ileocecal valve".

Then, the colonoscopy assisting apparatus 1 selects the first partial image data specified in the process of S43 from the image data IM included in the plurality of first learning data generated in the process of S31 and has a predetermined relationship with the first partial image data. Two partial image data are specified (S44).

Specifically, the colonoscopy assisting apparatus 1, for example, extracts the first partial image data and the size specified in the process of S43 from the image data IM included in the plurality of first learning data generated in the process of S31. and the partial image data corresponding to the structure whose shape difference is within a predetermined range (that is, partial image data that can be judged to be similar to the first partial image data) as the second partial image data.

After that, as shown in FIG. 9, the colonoscopy assisting apparatus 1 uses the combination of the type information and the position information corresponding to the second partial image data specified in the process of S44 and corresponds to a structure other than the polyp. A combination of type information and position information to be used is generated (S51).

Then, the colonoscopy assisting apparatus 1 combines the type information and position information generated in the process of S51 for each of the image data IM included in the plurality of first learning data generated in the process of S31. By adding, a plurality of third learning data are generated (S52).

Specifically, for example, when each of the combinations shown in FIG. 13 is specified in the process of S51, the colonoscopy assisting apparatus 1 selects the combination shown in FIG. 13 as shown in the underlined portion of FIG. The corresponding new identification information 131a is added to the identification information 131a described with reference to FIG. 10(A).

Subsequently, the colonoscopy assisting device 1 generates a second learning model by performing machine learning using the plurality of third learning data generated in the process of S52 (S53).

As a result, the colonoscopy assisting apparatus 1 uses not only the learning data containing the identification information 131a corresponding to the polyp, but also the learning data containing the identification information 131a corresponding to structures other than the polyp. 2 learning models).

After that, the operator verifies the second learning model generated in the process of S53, for example. Specifically, for example, the operator verifies the second learning model by using a plurality of learning data other than the first learning data and the second learning data among the plurality of learning data generated in the process of S32. I do. In addition, the operator may, for example, identify the identification information corresponding to the changed state of the combination specified in the process of S42 (for example, the information on the third line and the information on the sixth line included in the information described with reference to FIG. 12). and the image data IM corresponding to the identification information are used to verify the second learning model.

In this manner, the colonoscopy assisting apparatus 1 generates a plurality of learning data by adding the identification information 131 corresponding to the polyp included in each image data IM to each of the plurality of image data IM. do. Then, the colonoscopy assisting device 1 generates a first learning model by performing machine learning using a plurality of first learning data out of the plurality of learning data.

Thereafter, the colonoscopy assisting apparatus 1 receives the identification information 131 corresponding to the polyp included in the plurality of second learning data among the plurality of learning data, and the first A structure other than the polyp is specified from each of the image data IM included in the plurality of first learning data based on the result of comparison with the identification information 131 output from the learning model, and identification corresponding to the specified structure other than the polyp is performed. A plurality of third learning data are generated by adding the information 131 to each of the image data IM included in the plurality of first learning data. Then, the colonoscopy assisting device 1 generates a second learning model by performing machine learning using a plurality of third learning data.

That is, the colonoscopy assisting apparatus 1 uses not only the learning data containing the identification information 131 corresponding to the polyp but also the learning data containing the identification information 131 corresponding to structures other than the polyp so that the learning model (second learning model).

As a result, the colonoscopy assisting apparatus 1 can more accurately identify the polyp candidates PO included in the image data IM captured during the colonoscopy. Therefore, it becomes possible for a doctor to reduce the burden associated with colonoscopy.

Specifically, by using the second learning model, the colonoscopy assisting apparatus 1, for example, determines that the type information ID 23a corresponding to the image data IM described with reference to FIG. It becomes possible to determine that it is a blister. Therefore, in this case, the colonoscopy assisting apparatus 1, for example, as shown in FIG. It is possible to control so that the corresponding cursor) is not displayed on the output device 3 .

It should be noted that even after the second learning model is generated in the process of S53, the colonoscopy assisting apparatus 1 does not capture new image data IM due to the further execution of the colonoscopy. In this case, additional machine learning may be performed using learning data including the new image data IM.

As a result, the colonoscopy assisting device 1 can further improve the determination accuracy of the second learning model.

1: Colonoscopy Auxiliary Device 2: Operation Terminal 3: Output Device 101: CPU
102: Memory 103: Network Interface 104: Storage Medium 105: Bus

Claims (9)

A colonoscopy assisting device for detecting polyps in the colon by performing machine learning using image data,
a first learning data generation unit that generates a plurality of learning data by adding identification information corresponding to the polyp contained in each image data to each of the plurality of image data;
a first machine learning unit that generates a first learning model by performing machine learning using a plurality of first learning data out of the plurality of learning data;
identification information corresponding to the polyp contained in a plurality of second learning data among the plurality of learning data; and identification information output from the first learning model according to the input of the plurality of second learning data. a structure other than the polyp is identified from each of the image data included in the plurality of first learning data based on the comparison result of the plurality of first learning data, and identification information corresponding to the identified structure other than the polyp is provided to the plurality of first learning data a second learning data generation unit that generates a plurality of third learning data by adding to each of the image data included in the data;
a second machine learning unit that generates a second learning model by performing machine learning using the plurality of third learning data;
A colonoscopy assisting device characterized by: In claim 1,
The second learning data generation unit is
A specific identification that is not included in the identification information corresponding to the polyp included in the plurality of second learning data, among the identification information output from the first learning model in response to the input of the plurality of second learning data. identify the information,
identifying first partial image data corresponding to the specific identification information from the image data included in the plurality of second learning data;
identifying second partial image data having a predetermined relationship with the first partial image data from among the image data included in the plurality of first learning data;
By adding identification information corresponding to the second partial image data and corresponding to a structure other than the polyp to each of the image data included in the plurality of first learning data, the plurality of generating third learning data;
A colonoscopy assisting device characterized by: In claim 2,
The second learning data generation unit identifies partial image data satisfying a predetermined condition for a similarity relationship of the first partial image data as the second partial image data.
A colonoscopy assisting device characterized by: In claim 1,
The plurality of first learning data and the plurality of second learning data are different learning data,
A colonoscopy assisting device characterized by: In claim 1,
said structures other than polyps include at least one of vesicles, intestinal contents including stool, ileocecal valves and large intestine lumen;
A colonoscopy assisting device characterized by: In claim 1,
the structure other than the polyp comprises a light reflecting region;
A colonoscopy assisting device characterized by: In claim 1,
the identification information corresponding to the polyp is a combination of the type information of the polyp and the position information of the polyp in the image data containing the polyp;
The identification information corresponding to the structure other than the polyp is a combination of the type information of the structure other than the polyp and the position information of the structure other than the polyp in the image data including the structure other than the polyp.
A colonoscopy assisting device characterized by: A colonoscopy assisting method for causing a computer to execute processing for detecting polyps in the large intestine by performing machine learning using image data,
generating a plurality of learning data by adding identification information corresponding to the polyp contained in each image data to each of the plurality of image data;
generating a first learning model by performing machine learning using a plurality of first learning data out of the plurality of learning data;
identification information corresponding to the polyp contained in a plurality of second learning data among the plurality of learning data; and identification information output from the first learning model according to the input of the plurality of second learning data. a structure other than the polyp is identified from each of the image data included in the plurality of first learning data based on the comparison result of the plurality of first learning data, and identification information corresponding to the identified structure other than the polyp is provided to the plurality of first learning data generating a plurality of third learning data by adding to each of the image data included in the data;
generating a second learning model by performing machine learning using the plurality of third learning data;
A colonoscopy assisting method characterized by: A colonoscopy supplementary program that causes a computer to execute processing for detecting polyps in the colon by performing machine learning using image data,
generating a plurality of learning data by adding identification information corresponding to the polyp contained in each image data to each of the plurality of image data;
generating a first learning model by performing machine learning using a plurality of first learning data out of the plurality of learning data;
identification information corresponding to the polyp contained in a plurality of second learning data among the plurality of learning data; and identification information output from the first learning model according to the input of the plurality of second learning data. a structure other than the polyp is identified from each of the image data included in the plurality of first learning data based on the comparison result of the plurality of first learning data, and identification information corresponding to the identified structure other than the polyp is provided to the plurality of first learning data generating a plurality of third learning data by adding to each of the image data included in the data;
generating a second learning model by performing machine learning using the plurality of third learning data;
A colonoscopy assistance program characterized by:

Images