JP7629185B2 - Apparatus and method for determining possibility of cancer recurrence - Google Patents

Description

The present invention relates to a recurrence determination device and a recurrence determination method for determining the possibility of cancer recurrence.

There are various known methods for treating cancer, but in clinical practice, it is necessary to take into account the possibility of recurrence as well as the patient's wishes and quality of life. However, since there are various differences between patients in terms of the severity of cancer, physical condition, etc., it is extremely difficult to make treatment decisions that take into account the possibility of recurrence.

Therefore, the present invention aims to provide a new device and method that can easily determine the possibility of cancer recurrence after treatment.

In order to achieve the above object, the present invention provides a method for determining the possibility of cancer recurrence, comprising:
The apparatus includes an image acquisition unit, an extraction unit, a generation unit, a clustering unit, a combination pattern determination unit, a classification unit, a judgment unit, and an output unit,
The image acquisition unit acquires a slide image of a cancer tissue of a subject,
The extraction unit extracts a plurality of partial images from the slide image,
the generating unit uses a first model to determine whether each of the partial images corresponds to a cancer cell region or a stromal region, and generates a feature vector;
the clustering unit clusters each of the partial images into a corresponding cluster based on the type and the feature vector using a second model;
the combination pattern determination unit determines a combination pattern of cluster groups that indicates what combination each of the clusters of the plurality of partial images extracted from the slide image corresponds to for a plurality of cluster groups obtained by grouping clusters according to a combination of the type and the presence or absence of recurrence;
The determination unit determines a possibility of recurrence of the cancer tissue in the slide image based on a combination pattern of the cluster groups of the slide image by a third model;
The output unit outputs a result of a determination of a possibility of recurrence of the cancer tissue in the slide image,
The first model is
The model uses, as learning data, partial images extracted from learning slide images of cancer tissues of a plurality of cancer patients, and type information indicating whether the partial images correspond to a cancer cell region or a stromal region, and generates, through learning, a model that determines whether a region corresponds to a cancer cell region or a stromal region, and generates a feature vector;
The second model is
a clustering model that classifies the cancer cell regions into a plurality of clusters and the stromal region into a plurality of clusters, the clustering model being generated by learning using type information of the partial images and feature vectors of the partial images as learning data;
The third model is
a model for determining the presence or absence of recurrence, the model being generated by learning using recurrence information regarding the presence or absence of recurrence after adjuvant treatment following surgery and combination patterns of cluster groups for the plurality of cancer patients as learning data;
The combination pattern of the cluster group of the cancer patients is
For each cluster obtained by the second model, the partial images derived from cancer patients who have recurred after adjuvant treatment following surgery and the partial images derived from cancer patients who have not recurred after adjuvant treatment following surgery are counted, and if the count number of cases with recurrence is significantly large, the cluster is determined as a cluster with recurrence, and if the count number of cases without recurrence is significantly large, the cluster is determined as a cluster without recurrence;
The method is characterized in that, for each of the learning slide images of the multiple cancer patients, the clusters of the multiple partial images contained therein are determined as a pattern indicating in what combination they fall into a cluster group with recurrence in the cancer cell region, a cluster group with no recurrence in the cancer cell region, a cluster group with recurrence in the stromal region, and a cluster group with no recurrence in the stromal region.

The method for determining the possibility of cancer recurrence of the present invention comprises the steps of:
The method includes an image acquisition step, an extraction step, a generation step, a clustering step, a combination pattern determination step, a classification step, a judgment step, and an output step, and these steps are executed by a computer;
The image acquisition step includes acquiring a slide image of a cancer tissue of a subject,
The extracting step extracts the slide image into a plurality of partial images,
the generating step includes using a first model to determine whether each of the partial images corresponds to a cancer cell region or a stromal region, and to generate a feature vector;
the clustering step includes clustering each of the partial images into a corresponding cluster based on the type and the feature vector using a second model;
The combination pattern determination step determines a combination pattern of cluster groups that indicates what combination each of the clusters of the plurality of partial images extracted from the slide image corresponds to for a plurality of cluster groups obtained by grouping clusters according to a combination of the type and the presence or absence of recurrence,
The determination step includes:
A third model is used to determine the possibility of recurrence of the cancer tissue in the slide image based on the combination pattern of the cluster group of the slide image;
The output step outputs a result of the assessment of the possibility of recurrence of the cancer tissue in the slide image,
The first model is
The model uses, as learning data, partial images extracted from learning slide images of cancer tissues of a plurality of cancer patients, and type information indicating whether the partial images correspond to a cancer cell region or a stromal region, and generates, through learning, a model that determines whether a region corresponds to a cancer cell region or a stromal region, and generates a feature vector;
The second model is
a clustering model that classifies the cancer cell regions into a plurality of clusters and the stromal regions into a plurality of clusters, the clustering model being generated by learning using type information of the partial images and feature vectors of the partial images as learning data;
The third model is
a model for determining the presence or absence of recurrence, the model being generated by learning using recurrence information regarding the presence or absence of recurrence after adjuvant treatment following surgery and combination patterns of cluster groups for the plurality of cancer patients as learning data;
The combination pattern of the cluster group of the cancer patients is
For each cluster obtained by the second model, the partial images derived from cancer patients who have recurred after adjuvant treatment following surgery and the partial images derived from cancer patients who have not recurred after adjuvant treatment following surgery are counted, and if the count number of cases with recurrence is significantly large, the cluster is determined as a cluster with recurrence, and if the count number of cases without recurrence is significantly large, the cluster is determined as a cluster without recurrence;
The method is characterized in that, for each of the learning slide images of the multiple cancer patients, the clusters of the multiple partial images contained therein are determined as a pattern indicating in what combination they fall into a cluster group with recurrence in the cancer cell region, a cluster group with no recurrence in the cancer cell region, a cluster group with recurrence in the stromal region, and a cluster group with no recurrence in the stromal region.

The program of the present invention is characterized in that it causes a computer to execute the recurrence determination method of the present invention for determining the possibility of cancer recurrence.

The recording medium of the present invention is a computer-readable recording medium on which the program of the present invention is recorded.

According to the present invention, for example, by using a model generated by learning using partial images of the cancer cell region and partial images of the stromal region derived from slide images of a cancer patient who has undergone adjuvant treatment after surgery and has been diagnosed with the presence or absence of subsequent recurrence, it is possible to more easily determine the possibility of recurrence in a subject after adjuvant treatment. Therefore, according to the present invention, it is possible to determine whether or not further adjuvant treatment is required after surgery and to select the type of adjuvant treatment, taking into account the possibility of recurrence in a cancer patient after surgery.

FIG. 1 is a block diagram illustrating an example of a recurrence determination device according to the first embodiment. FIG. 2 is a block diagram showing an example of a hardware configuration of the recurrence determination device of the first embodiment.

In the recurrence assessment device of the present invention, for example, in the type information, the types of the cancer cell regions are 2 to 20 types, and the types of the stromal regions are 2 to 10 types.

In the recurrence assessment device of the present invention, for example, in the type information, the types of the cancer cell regions are cancer infiltrated regions and cancer main regions, and the types of the stromal regions are stromal mature regions and stromal immature regions.

In the recurrence determination device of the present invention, for example, the cluster groups include a cluster group with recurrence in a wet cancer region, a cluster group with no recurrence in a wet cancer region, a cluster group with recurrence in a main cancer region, a cluster group with no recurrence in a main cancer region, a cluster group with recurrence in a mature interstitial region, a cluster group with no recurrence in a mature interstitial region, a cluster group with recurrence in an immature interstitial region, and a cluster group with no recurrence in an immature interstitial region.

In the recurrence determination device of the present invention, for example, the number of clusters in the clustering unit is 100 to 1000.

In the recurrence assessment device of the present invention, for example, the third model is a model set including a model with adjuvant treatment generated by learning using, as the learning data, recurrence information regarding the presence or absence of recurrence after adjuvant treatment and a combination pattern of cluster groups for cancer patients among the plurality of cancer patients who have undergone any treatment after surgery, and a model without adjuvant treatment generated by learning using, as the learning data, recurrence information regarding the presence or absence of recurrence after adjuvant treatment and a combination pattern of cluster groups for cancer patients among the plurality of cancer patients who have not undergone any adjuvant treatment,
The determination unit is
using the model set as the third model;
The likelihood of recurrence is determined using the model with the adjuvant treatment, and the likelihood of recurrence is determined using the model without the adjuvant treatment.

In the recurrence assessment method of the present invention, for example, the assessment unit performs a comprehensive assessment based on the assessment result of the possibility of recurrence using the model with supplemental treatment and the assessment result of the possibility of recurrence using the model without supplemental treatment.

In the recurrence determination method of the present invention, for example, in the type information, the types of the cancer cell regions are 2 to 20 types, and the types of the stromal regions are 2 to 10 types.

In the recurrence determination method of the present invention, for example, in the type information, the types of the cancer cell regions are cancer infiltrated regions and cancer main regions, and the stromal regions are stromal mature regions and stromal immature regions.

In the recurrence determination method of the present invention, for example, the cluster groups include a cluster group with recurrence in the wet cancer region, a cluster group with no recurrence in the wet cancer region, a cluster group with recurrence in the main cancer region, a cluster group with no recurrence in the main cancer region, a cluster group with recurrence in the mature interstitial region, a cluster group with no recurrence in the mature interstitial region, a cluster group with recurrence in the immature interstitial region, and a cluster group with no recurrence in the immature interstitial region.

In the recurrence determination method of the present invention, for example, the number of clusters in the clustering step is 100 to 1,000.

In the recurrence assessment method of the present invention, for example, the third model is a third model set including a model with adjuvant treatment generated by learning using, as the learning data, recurrence information regarding the presence or absence of recurrence after adjuvant treatment and a combination pattern of cluster groups for cancer patients among the plurality of cancer patients who have undergone any treatment after surgery, and a model without adjuvant treatment generated by learning using, as the learning data, recurrence information regarding the presence or absence of recurrence after adjuvant treatment and a combination pattern of cluster groups for cancer patients among the plurality of cancer patients who have not undergone any adjuvant treatment,
The determination step includes:
using the third model set as the third model;
The likelihood of recurrence is determined using the model with the adjuvant treatment, and the likelihood of recurrence is determined using the model without the adjuvant treatment.

In the recurrence assessment method of the present invention, for example, the assessment step performs a comprehensive assessment based on the assessment result of the possibility of recurrence using the model with supplemental treatment and the assessment result of the possibility of recurrence using the model without supplemental treatment.

According to the present invention, for example, the possibility of recurrence in a cancer patient when treatment (specifically, adjuvant treatment after surgery) is performed can be determined. The type of cancer that is the subject of the present invention is not particularly limited, and examples thereof include any cancer from which pathological images can be obtained, such as digestive system cancer, respiratory system cancer, reproductive system cancer, skin cancer, and nervous system cancer. The means of the adjuvant treatment are not particularly limited, and examples thereof include drug therapy (chemotherapy), radiation therapy, immunotherapy, transplantation, etc.

In the present invention, the target cancer is not particularly limited, but in assessing recurrence, it is preferable, for example, to match the type of cancer of the subject with the type of cancer in the model set. Furthermore, since the present invention can assess the possibility of recurrence after adjuvant treatment performed after surgery, it is preferable to use, for example, a model set generated using learning data related to adjuvant treatment that matches the expected method of adjuvant treatment.

In the present invention, the first model, the second model, and the third model are collectively referred to as a model set.

The following describes an embodiment of the present invention with reference to the drawings. The present invention is not limited to the following embodiment. In each of the drawings, the same parts are given the same reference numerals. The explanations of each embodiment can be used interchangeably unless otherwise specified. The configurations of each embodiment can be combined unless otherwise specified.

[Embodiment 1]
An example of a determination device for determining the possibility of cancer recurrence (hereinafter also referred to as a recurrence determination device) and a determination method (hereinafter also referred to as a recurrence determination method) of the present invention will be described with reference to the drawings. In the present invention, determination can be read as estimation.

FIG. 1 is a block diagram showing an example of the configuration of the recurrence determination device 1 of this embodiment. The recurrence determination device 1 includes an image acquisition unit 11, an extraction unit 12, a generation unit 13, a clustering unit 14, a combination pattern determination unit 15, a determination unit 16, and an output unit 17. The recurrence determination device 1 further includes, for example, a model set 19. The recurrence determination device 1 further includes, for example, a storage unit 18, and the model set 19 is stored in the storage unit 18. The model set 19 includes a first model 191, a second model 192, and a third model 193. The recurrence determination device 1 is also referred to as, for example, a recurrence determination system.

2 shows an example of a block diagram of the hardware configuration of the recurrence determination device 1. The recurrence determination device 1 has, for example, a CPU (central processing unit) 101, a memory 102, a bus 103, an input device 104, a display unit 105, a communication device 106, a storage device 107, and an imaging device 109. Each unit of the recurrence determination device 1 is connected to each other via the bus 103 by its respective interface (I/F).

The CPU 101 is a processor that is responsible for the overall control of the recurrence determination device 1, and is not limited to a CPU and may be another processor. In the recurrence determination device 1, the CPU 101 executes, for example, the program of the present invention and other programs, and also reads and writes various types of information.

The recurrence determination device 1 can be connected to a communication network, for example, by a communication device 106 connected to the bus 103, and can also be connected to an external device via the communication network. The external device is not particularly limited, and examples thereof include a terminal, a PC (personal computer), a server, etc. The terminal is, for example, a smartphone, a mobile phone, a tablet, etc. The connection method between the recurrence determination device 1 and the external device is not particularly limited, and may be, for example, a wired connection or a wireless connection. The wired connection may be, for example, a cord connection or a cable connection for using a communication network. The wireless connection may be, for example, a connection using a communication network or a connection using wireless communication. The communication network is not particularly limited, and for example, a known communication network may be used, and examples thereof include the Internet line, a telephone line, a LAN (Local Area Network), and WiFi (Wireless Fidelity).

The memory 102 includes, for example, a main memory, which is also referred to as a primary storage device. When the CPU 101 performs processing, the memory 102 reads various operation programs 108, such as the program of the present invention, stored in, for example, an auxiliary storage device described below, and the CPU 101 receives data from the memory 102 and executes the program 108. In addition, the memory 102 reads, for example, a model set 19 (first model 191, second model 192, third model 193) in the present invention stored in, for example, an auxiliary storage device described below, and the CPU 101 receives data from the memory 102 and executes the program 108 using the model set 19. The main memory is, for example, a RAM (random access memory). The memory 102 further includes, for example, a ROM (read only memory).

The storage device 107 includes, for example, the storage unit 18 described above, and is also called an auxiliary storage device in contrast to the main memory (primary storage device). The storage device 107 includes, for example, a storage medium and a drive for reading and writing data from and to the storage medium. The storage medium is not particularly limited, and may be, for example, an internal or external type, and may be, for example, a hard disk drive (HDD), a CD-ROM, a CD-R, a CD-RW, an MO, a DVD, a flash memory, a memory card, or the like, and the drive is not particularly limited. The storage device 107 may be, for example, a hard disk drive (HDD) in which a storage medium and a drive are integrated. The storage device 107 stores, for example, the program 108 and the model set 19, as described above. Various information and slide images input to the recurrence assessment device 1, information obtained by the recurrence assessment device 1, and the like may be stored in the storage device 107 (storage unit 18), for example.

The recurrence assessment device 1 may further include, for example, an input device 104. The input device 104 is, for example, a scanner, a touch panel, a keyboard, etc. The recurrence assessment device 1 may input, for example, subject information, slide images, etc., through the input device 104, or may input the information through the input device 104 via the communication device 106.

The recurrence assessment device 1 may further include, for example, an imaging device 109, which may take images of slides of cancer tissue, and store the obtained slide images in the storage unit 18. The imaging device 109 is not particularly limited and may be, for example, a camera such as a CCD camera.

The recurrence determination device 1 may have, for example, a display unit 105. The display unit 105 is, for example, a display, and specific examples include an LED display and a liquid crystal display. The recurrence determination device 1 may, for example, display the obtained determination result on the display unit 105 via the output unit 17, or may output the result from the output unit 16 to an external terminal or the like via the communication device 106.

The recurrence determination device 1 will now be described in more detail.

In the recurrence assessment device 1, the image acquisition unit 11 acquires slide images of the subject's cancer tissue. The subject is, for example, a cancer patient, and the slide images can be acquired, for example, by imaging the cancer tissue that has been surgically removed from the cancer patient.

The type of slide image is not particularly limited, and may be, for example, an image of a cancer tissue slice that has been stained by immunostaining or the like. The actual dimensions of the slice shown in one slide image are, for example, 20 to 40 mm in length and 20 to 40 mm in width.

The slide image of the subject may be stored in the storage unit 18, for example, in association with information about the subject. The information about the subject may include, for example, identification information such as name, cancer morbidity information including type and stage of cancer, treatment information, etc.

The extraction unit 12 extracts a plurality of partial images from the slide image. The extraction of the partial images is also referred to as, for example, cutting out the partial images. The number of partial images obtained from one slide is not particularly limited, and may be, for example, 1,000 to 40,000. The method of extracting partial images from the slide image is not particularly limited, and may be performed, for example, as follows. That is, in the recurrence assessment device 1, the slide image is displayed on the display unit 105. Then, the operator selects a partial area from the slide image on the display unit 105 and inputs the selection via the input device 104, thereby extracting the partial image. The partial area can be selected, for example, by specifying an area with a pointer, specifying coordinates, or the like on the slide image displayed on the display unit 105.

In addition, in the recurrence assessment device 1, the extraction unit 12 may, for example, automatically extract partial images. Automatic extraction of partial images can be performed, for example, as follows. That is, for example, a virtual magnification (for example, 5x, 10x, 20x, 40x) is specified based on the microscope magnification when the slide image was captured. As a specific example, for example, if pixels are acquired every other pixel from an image file of the slide image captured at a magnification of 40x, an image captured at a virtual magnification of 20x can be acquired. Then, the size of the partial image (for example, a square with one side of 128, 256, or 512 pixels) is specified, and the partial image is cut out in a tile shape from the slide image. The determination of whether tissue is included in the partial image can be made, for example, by extracting pixels with a saturation equal to or greater than a predetermined threshold value for each pixel value of the partial image, and determining the proportion of the pixels (for example, 30% or more) that are present relative to the total number of pixels in one partial image to determine whether the partial image includes tissue.

The generation unit 13 uses the first model 191 to determine whether each of the partial images corresponds to a cancer cell region or a stromal region, and to generate a feature vector. According to the first model 191, by providing the partial image to the model, the type of object depicted in the partial image (i.e., whether it is a cancer cell region or a stromal region, etc.) can be estimated, and a feature vector for the partial image can be generated.

The clustering unit 14 uses the second model 192 to cluster each of the partial images into a corresponding cluster based on the type and the feature vector. According to the second model 192, by providing the type of the partial image and its feature vector to the model, it is possible to determine which of multiple clusters the partial image belongs to.

The combination pattern determination unit 15 determines a combination pattern of cluster groups that indicates what combination each of the clusters of the multiple partial images extracted from the slide image corresponds to for multiple cluster groups obtained by grouping clusters according to the combination of the type and the presence or absence of recurrence. As described below, the multiple clusters classified by clustering can be further classified into multiple groups (cluster groups) according to the combination of the type and the recurrence information on the presence or absence of recurrence. Therefore, it is possible to determine the combination pattern of the cluster group according to what combination each of the clusters of the multiple partial images derived from the slide image of the subject corresponds to.

The determination unit 16 determines the possibility of recurrence of the cancer tissue in the slide image based on the combination pattern of the cluster groups of the slide image using the third model 193. According to the third model 193, it is possible to determine whether the subject has a possibility of recurrence or not based on the combination pattern of the slide image of the subject.

The output unit 17 outputs the result of the assessment of the possibility of recurrence of the cancer tissue in the slide image of the subject.

The recurrence determination method of the present invention includes, for example, an image acquisition step, an extraction step, a generation step, a clustering step, a combination pattern determination step, a classification step, a determination step, and an output step, as described above, and these steps are executed by a computer. The recurrence determination method of the present invention can be executed, for example, by using the recurrence determination device of the present invention. For example, unless otherwise specified, the recurrence determination method of the present invention can be implemented by referring to the description of the recurrence determination device of the present invention.

The image acquisition process acquires slide images of the subject's cancer tissue, and this process can be performed, for example, by the image acquisition unit 11 of the recurrence assessment device 1.

The extraction process extracts the slide image into a plurality of partial images, and this process can be performed, for example, by the extraction unit 12 of the recurrence assessment device 1.

The generation step uses the first model to determine whether each partial image corresponds to a cancer cell region or a stromal region, and to generate a feature vector. This step can be executed by the generation unit 13 of the recurrence determination device 1, for example, using the first model 191.

The clustering step uses the second model to cluster each of the partial images into a corresponding cluster based on the type and the feature vector. This step can be executed by the clustering unit 14 of the recurrence determination device, for example, using the second model 192.

The combination pattern determination step determines a combination pattern of cluster groups that indicates the combination of each cluster of the multiple partial images extracted from the slide image for multiple cluster groups obtained by grouping clusters according to the combination of the type and the presence or absence of recurrence. This step can be executed, for example, by the combination pattern determination unit 15 of the recurrence assessment device 1.

The determination step determines the possibility of recurrence of the cancer tissue in the slide image based on the combination pattern of the cluster groups in the slide image using a third model. This step can be executed by the determination unit 16 of the recurrence determination device 1, for example, using the third model 193.

The output process outputs the assessment result of the possibility of recurrence of the cancer tissue in the slide image, and this process can be executed, for example, by the output unit 17 of the recurrence assessment device 1.

[Embodiment 2]
The model set in the recurrence determination device and recurrence determination method of the present invention is the model set of the present invention. The model set of the present invention can be applied to the descriptions in other embodiments including the first embodiment. Hereinafter, the model set of the present invention will be described using an example, but the present invention is not limited to this example.

The model set of the present invention is a model set for the recurrence determination device of the present invention or a model set for the recurrence determination method of the present invention, and includes the first model, the second model, and the third model. Each model is a learning model, and the form thereof is not particularly limited, and is, for example, a support vector machine.

The first model is
The model uses, as learning data, partial images extracted from learning slide images of cancer tissues of a plurality of cancer patients, and type information indicating whether the partial images correspond to a cancer cell region or a stromal region, and generates, through learning, a model that determines whether a region corresponds to a cancer cell region or a stromal region, and generates a feature vector;
The second model is
a clustering model that classifies the cancer cell regions into a plurality of clusters and the stromal regions into a plurality of clusters, the clustering model being generated by learning using type information of the partial images and feature vectors of the partial images as learning data;
The third model is
a model for determining the presence or absence of recurrence, the model being generated by learning using recurrence information regarding the presence or absence of recurrence after adjuvant treatment following surgery and combination patterns of cluster groups for the plurality of cancer patients as learning data;
The combination pattern of the cluster group of the cancer patients is
For each cluster obtained by the second model, the partial images derived from cancer patients who have recurred after adjuvant treatment following surgery and the partial images derived from cancer patients who have not recurred after adjuvant treatment following surgery are counted, and if the count number of cases with recurrence is significantly large, the cluster is determined as a cluster with recurrence, and if the count number of cases without recurrence is significantly large, the cluster is determined as a cluster without recurrence;
For each of the multiple learning slide images of the cancer patients, a pattern is determined that indicates in what combination each of the clusters of the multiple partial images contained therein corresponds to a cluster group with recurrence in the cancer cell region, a cluster group with no recurrence in the cancer cell region, a cluster group with recurrence in the stromal region, and a cluster group with no recurrence in the stromal region.

As an example of generating a model set of the present invention, model set 19 including first model 191, second model 192, and third model 193 in the first embodiment will be described.

The model set may be generated without distinguishing the type of cancer (the type of tissue in which the cancer occurs), for example. However, since this leads to more reliable judgments, it is preferable to generate the model set according to the type of cancer (for example, each type of colorectal cancer, pancreatic cancer, lung cancer, etc.). In the following examples, the target cancer is assumed to be "specific cancer X," but the present invention is not limited to this example in any way.

To generate model set 19, information on cancer patients who have undergone adjuvant treatment after surgery and have been clinically diagnosed as having a recurrence as a result of the treatment is used. The information includes, for example, slide images of cancer tissues removed by surgery; morbidity information such as the type and stage of cancer; the presence or absence of adjuvant treatment, the contents of the adjuvant treatment, and progress information after the adjuvant treatment; and identification information such as an ID that identifies the patient, all of which are linked to one another. In the explanation of model set 19, the cancer patients are referred to as learning cancer patients in order to distinguish them from subjects for whom the possibility of recurrence is to be determined. In addition, in the present invention, subjects whose cancer has not recurred due to adjuvant treatment are also referred to as cancer patients for convenience in terms of their history of cancer.

As described above, the first model 191 is a model that determines whether a region is a cancer cell region or a stromal region, and generates a feature vector. The first model 191 can be generated, for example, as follows.

First, as the learning slide images, slide images of cancer tissues before adjuvant treatment of a cancer patient who has undergone adjuvant treatment after surgery for cancer X and has been clinically diagnosed as having a recurrence as a result are used. The number of slide images is not particularly limited, but in order to generate a more reliable model, for example, the number of cancer patients for learning is preferably 400 or more.

The type of slide image is not particularly limited, and may be, for example, an image of a cancer tissue slice that has been stained by immunostaining or the like. The actual dimensions of the slice shown in one slide image are, for example, 20 to 40 mm in length and 20 to 40 mm in width.

Next, a plurality of partial images are extracted from the slide image. The extraction of the partial images is also referred to as, for example, cutting out the partial images. The number of partial images obtained from one slide is not particularly limited, and may be, for example, 1,000 to 40,000. The method of extracting partial images from the slide image is not particularly limited, and for example, based on pathology, a portion of the slide image showing a cancer cell region and a portion of the slide image showing a stromal region can be selected and extracted. The partial images extracted from the slide image are also referred to as, for example, ROI (Region of Interest) images. The partial images are, for example, tagged (annotated) and used for learning.

Then, the first model 19 can be generated by learning using the partial image and type information of the partial image (i.e., information on whether it is a cancer cell region or a stromal region) as learning data. With the first model 19, even if it is unclear whether a partial image is a cancer cell region or a stromal region, for example, the type can be determined and a feature vector of the partial image can be generated.

The type information is not limited to two types, for example, whether it is a cancer cell region or a stromal region, and the cancer cell region may be further classified, for example, into 2 to 20 types, and the stromal region may be further classified, for example, into 2 to 10 types. As a specific example, the cancer cell region may be classified, for example, into two types, an invasive part (also called an advanced part) region and a main part region, and the stromal region may be classified, for example, into two types, a mature region and an immature region.

As described above, the second model 192 is a clustering model that classifies cancer cell regions into multiple clusters and stromal regions into multiple clusters. The second model 192 can be generated, for example, as follows.

The second model 192 can be generated by performing learning using the partial image type information and the feature vectors of the partial images linked thereto, which were also used to generate the first model 191, as learning data. According to the second model 192, as described below, multiple partial images extracted from a slide image of a subject can be classified into corresponding clusters.

In the learning, for example, the number of clusters to be classified can be set arbitrarily. As a specific example, for rough classification, the number of clusters can be about 10 to 50, for medium classification, the number of clusters can be about 50 to 100, and for detailed classification, the number of clusters can be about 100 to 1000.

As described above, the third model 193 is a model that determines whether or not there is a possibility of recurrence. The third model 193 can be generated, for example, as follows.

First, the second model 192 is used to perform clustering for each partial image extracted from the learning slide image of the learning cancer patient. Since the presence or absence of recurrence is known for the learning cancer patient, the partial images derived from the learning cancer patient who recurred after supplementary treatment and the partial images derived from the learning cancer patient who did not recur after supplementary treatment are counted for each cluster. As a result, for each cluster, the count number of partial images derived from the learning cancer patient with recurrence and the count number of partial images derived from the learning cancer patient with recurrence are obtained. Then, for each cluster, the count number of recurrence and the count number of no recurrence are compared, and, for example, if the count number of recurrence is significantly larger than the other, the cluster is set as a cluster with recurrence, and if the count number of no recurrence is significantly larger than the other, the cluster is set as a cluster without recurrence. The degree of significance is not particularly limited and can be set arbitrarily, and may be set, for example, as a difference in number (e.g., 50 or more times larger, etc.) or as a ratio to the other (e.g., 1.5 times larger, etc.). Additionally, clusters that do not show significant differences between the two may be excluded, for example.

Next, a combination pattern of cluster groups is determined for each of the multiple learning slide images of cancer patients.

Here, as an example, a case will be described in which the type information includes four types, for example, the infiltrated area of cancer cells, the main area of cancer cells, the mature area of the stroma, and the immature area of the stroma. The multiple clusters classified by clustering based on a combination of the type information and the presence or absence of recurrence are classified into eight types of groups, as shown in Table 1 below. Suppose that the multiple partial images from the learning slide image of the learning cancer patient A are cluster Nos. 1, 15, 19, and 27. In this case, the combination pattern of the cluster groups of the learning slide image is group 1/group 4/group 5/group 7.

Such a combination pattern is determined for each of the multiple learning slide images of cancer patients. Then, the combination patterns and the recurrence information (information regarding the presence or absence of recurrence) linked thereto for the multiple learning cancer patients are used as learning data, and a third model 193 can be generated by performing learning. According to the third model 193, as described below, it is possible to determine the possibility of recurrence from the combination pattern of the cluster group of the slide image of the subject.

As exemplified in the first embodiment, in the combination pattern process using the combination pattern determination unit 15, a combination pattern of cluster groups is determined for the slide images of the subject. In this case, taking Table 1 as an example, if the clusters of the multiple partial images derived from the slide images of the subject are cluster Nos. 1, 15, 19, and 27, the combination pattern of the cluster groups of the slide images can be determined to be the combination pattern of group 1/group 4/group 5/group 7.

(Variation 1)
The third model may be, for example, a third model (3M) generated using learning data from all learning patients, regardless of whether or not any auxiliary treatment was performed, as described above, or may be a set of two third models generated by dividing the learning data from the learning patients into two types and using them for learning, as in this modified example. That is, it may be a set of a third model with auxiliary treatment (3M(+)) generated using learning data from learning patients who have received any auxiliary treatment, and a third model without auxiliary treatment (3M(-)) generated using learning data from learning patients who have not received any auxiliary treatment. In the recurrence determination device or recurrence determination method of the present invention, by using a third model set including such two types of third models, it is possible to determine, for example, the presence or absence of recurrence when any auxiliary treatment is performed, and the presence or absence of recurrence when any auxiliary treatment is not performed. This makes it possible to determine, for example, whether an optional auxiliary treatment is suitable for the subject from the viewpoint of recurrence. The optional auxiliary treatment is not particularly limited and can be arbitrarily selected from the treatments described above.

In this modified example, chemotherapy is used as an example of an optional auxiliary treatment, but the present invention is not limited to this. In other words, in cancer treatment, the presence or absence of recurrence may differ depending on whether chemotherapy is adopted or performed as an auxiliary treatment method for each patient. Therefore, if the two types of third models generated in this modified example, that is, the third model with chemotherapy (3M(+)) and the third model without chemotherapy (3M(-)), are used in the recurrence determination device or recurrence determination method of the present invention, two determination results, recurrence (Re) (+)/no recurrence (-), can be obtained from each third model, as shown in Table 2 below. Then, from these two determination results, a more comprehensive determination result can be obtained. Therefore, by using this modified example, it is possible to further determine whether or not the subject needs auxiliary treatment and to select the method thereof, taking into account the possibility of recurrence.

(Variation 2)
In this modification 2, for example, the third model set (3M(+), 3M(-)) of the modification 1 may be used in combination with the third model (3M) generated using the learning data derived from all the learning patients described above. In this case, three judgment results, namely, recurrence (Re) (+)/no recurrence (-), can be obtained from the third model with treatment (3M(+)), the third model without treatment (3M(-)), and the third model of all data (3M). Then, from these three judgment results, a more comprehensive judgment result can be obtained. Therefore, by using this modification, it is possible to select a supplementary treatment for the subject that further takes into account the possibility of recurrence.

In this case, a comprehensive judgment result can be obtained from the results of each third model set (3M(+), 3M(-)) in variant 1 and the results of the third model (3M) of the entire data, for example, as shown in Table 3 below.

(Variation 3)
As shown in Table 1, the clusters are classified into a plurality of cluster groups according to the combination of the type of partial image derived from the learning slide image and the presence or absence of recurrence. In this case, all the clusters in each cluster group may be adopted as constituent clusters, or only the clusters ranked high may be adopted as constituent clusters. The ranking of the clusters may be, for example, a ranking in order of the significant difference in the count number. That is, in the case of a cluster with recurrence, it is assumed that the count number with recurrence is significantly larger than the count number without recurrence, and the clusters can be ranked in order of the magnitude of the count number. Then, only the clusters showing a relatively larger count number than the count number without recurrence can be selected as clusters constituting the cluster group with recurrence. The same is true for the cluster group without recurrence.

[Embodiment 3]
The program of the present invention is a program capable of executing the recurrence determination method of the present invention on a computer. The program of the present invention can cause a processor of a computer to execute the recurrence determination method of the present invention. Alternatively, the program of this embodiment may be recorded, for example, in a computer-readable recording medium. The recording medium is not particularly limited, and examples thereof include the storage media described above.

[Example 1]
The model set of the present invention was generated under the following conditions using slide images of colon cancer tissues surgically removed from colon cancer patients.

Total number of patients: 400 Number of patients who relapsed: 140 Number of patients who did not relapse: 260 Number of clusters for each type of partial image: 1000 Cluster group composition Cluster: Clusters that fall within the top 20% of the ranking Kernel of learning model (SVM): rbf

The model set of the comparative example and the model set of the example were generated using the following partial images obtained from slide images of cancer patients as learning data.
Comparative Example 1A: Partial image of the infiltrated area of cancer Comparative Example 1B: Partial image of the main area of cancer Comparative Example 1C: Partial image of the infiltrated area of cancer and partial image of the main area Comparative Example 2A: Partial image of the mature area of stroma Comparative Example 2B: Partial image of the immature area of stroma Comparative Example 2C: Partial image of the mature area of stroma and partial image of the immature area of stroma Example 1: Partial image of the infiltrated area of cancer, partial image of the main area of cancer, partial image of the mature area of stroma

Using each generated model set, the slide images of the patients used in the learning data were re-evaluated by computer to determine the accuracy rate for whether or not there was a recurrence. Additionally, using each of the same model sets, slide images from before adjuvant treatment of new subjects who had been diagnosed with the presence or absence of recurrence after adjuvant treatment were evaluated to determine the accuracy rate for whether or not there was a recurrence. The closer the accuracy rate (Accuracy) is to 1, the higher the accuracy. These results are shown in Table 4 below.

As shown in Table 4 below, according to the embodiment, the training data showed a high accuracy rate, and the subjects also showed a similarly high accuracy rate.

The present invention has been described above with reference to the embodiments, but the present invention is not limited to the above embodiments. Various modifications that can be understood by a person skilled in the art can be made to the configuration and details of the present invention within the scope of the present invention.

As described above, according to the present invention, for example, by using a model generated by learning using partial images of a cancer cell region and a partial image of a stromal region derived from a slide image of a cancer patient who has been given adjuvant treatment after surgery and has been diagnosed with the presence or absence of recurrence, it is possible to more easily determine the possibility of recurrence of a subject after adjuvant treatment. Therefore, according to the present invention, it is possible to determine whether further adjuvant treatment is necessary after surgery and to select an adjuvant treatment, taking into account the possibility of recurrence of a cancer patient after surgery.

Claims (14)

The apparatus includes an image acquisition unit, an extraction unit, a generation unit, a clustering unit, a combination pattern determination unit, a classification unit, a judgment unit, and an output unit,
The image acquisition unit acquires a slide image of a cancer tissue of a subject,
The extraction unit extracts a plurality of partial images from the slide image,
the generating unit uses a first model to determine whether each of the partial images corresponds to a cancer cell region or a stromal region, and generates a feature vector;
the clustering unit clusters each of the partial images into a corresponding cluster based on the type and the feature vector using a second model;
the combination pattern determination unit determines a combination pattern of cluster groups that indicates what combination each of the clusters of the plurality of partial images extracted from the slide image corresponds to for a plurality of cluster groups obtained by grouping clusters according to a combination of the type and the presence or absence of recurrence;
The determination unit determines a possibility of recurrence of the cancer tissue in the slide image based on a combination pattern of the cluster groups of the slide image by a third model;
The output unit outputs a result of a determination of a possibility of recurrence of the cancer tissue in the slide image,
The first model is
The model uses, as learning data, partial images extracted from learning slide images of cancer tissues of a plurality of cancer patients, and type information indicating whether the partial images correspond to a cancer cell region or a stromal region, and generates, through learning, a model that determines whether a region corresponds to a cancer cell region or a stromal region, and generates a feature vector;
The second model is
a clustering model that classifies the cancer cell regions into a plurality of clusters and the stromal region into a plurality of clusters, the clustering model being generated by learning using type information of the partial images and feature vectors of the partial images as learning data;
The third model is
a model for determining the presence or absence of recurrence, the model being generated by learning using recurrence information regarding the presence or absence of recurrence after adjuvant treatment following surgery and combination patterns of cluster groups for the plurality of cancer patients as learning data;
The combination pattern of the cluster group of the cancer patients is
For each cluster obtained by the second model, the partial images derived from cancer patients who have recurred after adjuvant treatment following surgery and the partial images derived from cancer patients who have not recurred after adjuvant treatment following surgery are counted, and if the count number of cases with recurrence is significantly large, the cluster is determined as a cluster with recurrence, and if the count number of cases without recurrence is significantly large, the cluster is determined as a cluster without recurrence;
A pattern is determined that indicates in what combination each cluster of the plurality of partial images included in each learning slide image of the plurality of cancer patients corresponds to a cluster group of recurrence in the cancer cell region, a cluster group of no recurrence in the cancer cell region, a cluster group of recurrence in the stromal region, and a cluster group of no recurrence in the stromal region ;
A recurrence determination device for determining the possibility of cancer recurrence, wherein the type information indicates that the types of the cancer cell regions are 2 to 20 types and the types of the stromal regions are 2 to 10 types . The recurrence determination device according to claim 1 , wherein in the type information, the types of the cancer cell regions are a cancer infiltration region and a cancer main region, and the types of the stromal regions are a stromal mature region and a stromal immature region. The recurrence determination device of claim 2, wherein the cluster groups include a cluster group with recurrence in invasive cancer regions, a cluster group with no recurrence in invasive cancer regions, a cluster group with recurrence in main cancer regions, a cluster group with no recurrence in main cancer regions, a cluster group with recurrence in mature interstitial regions, a cluster group with no recurrence in mature interstitial regions, a cluster group with recurrence in immature interstitial regions, and a cluster group with no recurrence in immature interstitial regions . 4. The recurrence determination device according to claim 1, wherein the number of clusters in the clustering section is 100 to 1000. The third model is a model set including a model with adjuvant treatment, which is generated by learning using, as the learning data, recurrence information on the presence or absence of recurrence after adjuvant treatment and a combination pattern of cluster groups for cancer patients who have received any adjuvant treatment after surgery among the plurality of cancer patients, and a model without adjuvant treatment, which is generated by learning using, as the learning data, recurrence information on the presence or absence of recurrence after adjuvant treatment and a combination pattern of cluster groups for cancer patients who have not received any adjuvant treatment among the plurality of cancer patients,
The determination unit is
using the model set as the third model;
The recurrence determination device according to claim 1 , further comprising: a determination of the possibility of recurrence based on the model with the auxiliary treatment; and a determination of the possibility of recurrence based on the model without the auxiliary treatment. The recurrence determination device according to claim 5 , wherein the determination unit performs a comprehensive determination based on a result of the determination of the possibility of recurrence based on the model with the auxiliary treatment and a result of the determination of the possibility of recurrence based on the model without the auxiliary treatment. The method includes an image acquisition step, an extraction step, a generation step, a clustering step, a combination pattern determination step, a classification step, a judgment step, and an output step, and these steps are executed by a computer;
The image acquisition step includes acquiring a slide image of a cancer tissue of a subject,
The extracting step extracts the slide image into a plurality of partial images,
the generating step includes using a first model to determine whether each of the partial images corresponds to a cancer cell region or a stromal region, and to generate a feature vector;
the clustering step includes clustering each of the partial images into a corresponding cluster based on the type and the feature vector using a second model;
The combination pattern determination step determines a combination pattern of cluster groups that indicates what combination each of the clusters of the plurality of partial images extracted from the slide image corresponds to for a plurality of cluster groups obtained by grouping clusters according to a combination of the type and the presence or absence of recurrence,
The determination step includes:
A third model is used to determine the possibility of recurrence of the cancer tissue in the slide image based on the combination pattern of the cluster group of the slide image;
The output step outputs a result of the assessment of the possibility of recurrence of the cancer tissue in the slide image,
The first model is
The model uses, as learning data, partial images extracted from learning slide images of cancer tissues of a plurality of cancer patients, and type information indicating whether the partial images correspond to a cancer cell region or a stromal region, and generates, through learning, a model that determines whether a region corresponds to a cancer cell region or a stromal region, and generates a feature vector;
The second model is
a clustering model that classifies the cancer cell regions into a plurality of clusters and the stromal region into a plurality of clusters, the clustering model being generated by learning using type information of the partial images and feature vectors of the partial images as learning data;
The third model is
a model for determining the presence or absence of recurrence, the model being generated by learning using recurrence information regarding the presence or absence of recurrence after adjuvant treatment following surgery and combination patterns of cluster groups for the plurality of cancer patients as learning data;
The combination pattern of the cluster group of the cancer patients is
For each cluster obtained by the second model, the partial images derived from cancer patients who have recurred after adjuvant treatment following surgery and the partial images derived from cancer patients who have not recurred after adjuvant treatment following surgery are counted, and if the count number of cases with recurrence is significantly large, the cluster is determined as a cluster with recurrence, and if the count number of cases without recurrence is significantly large, the cluster is determined as a cluster without recurrence;
A pattern is determined that indicates in what combination each cluster of the plurality of partial images included in each learning slide image of the plurality of cancer patients corresponds to a cluster group of recurrence in the cancer cell region, a cluster group of no recurrence in the cancer cell region, a cluster group of recurrence in the stromal region, and a cluster group of no recurrence in the stromal region ;
In the type information, the types of the cancer cell regions are 2 to 20 types, and the types of the stromal regions are 2 to 10 types.
A method for determining the possibility of cancer recurrence, comprising: The method of determining recurrence according to claim 7 , wherein in the type information, the types of the cancer cell regions are a cancer infiltration region and a cancer main region, and the types of the stromal regions are a stromal mature region and a stromal immature region. The method for determining recurrence according to claim 8, wherein the cluster groups include a cluster group with recurrence in an invasive region of cancer, a cluster group with no recurrence in an invasive region of cancer, a cluster group with recurrence in a main region of cancer, a cluster group with no recurrence in a main region of cancer, a cluster group with recurrence in a mature region of stroma, a cluster group with no recurrence in a mature region of stroma, a cluster group with recurrence in an immature region of stroma , and a cluster group with no recurrence in an immature region of stroma. The recurrence determination method according to any one of claims 7 to 9 , wherein the number of clusters in the clustering step is 100 to 1000. The third model is a third model set including a model with adjuvant treatment, which is generated by learning using, as the learning data, recurrence information on the presence or absence of recurrence after adjuvant treatment and a combination pattern of cluster groups for cancer patients who have received any adjuvant treatment after surgery among the plurality of cancer patients, and a model without adjuvant treatment, which is generated by learning using, as the learning data, recurrence information on the presence or absence of recurrence after adjuvant treatment and a combination pattern of cluster groups for cancer patients who have not received any adjuvant treatment among the plurality of cancer patients,
The determination step includes:
using the third model set as the third model;
The method for determining recurrence according to claim 7 , further comprising: determining the likelihood of recurrence using a model with adjuvant treatment; and determining the likelihood of recurrence using a model without adjuvant treatment. The recurrence determination method according to claim 11 , wherein the determination step comprises making a comprehensive determination based on a result of the determination of the possibility of recurrence using the model with the supplemental treatment and a result of the determination of the possibility of recurrence using the model without the supplemental treatment. A program for causing a computer to execute the method for determining the possibility of cancer recurrence according to any one of claims 7 to 12 . A computer-readable recording medium having the program according to claim 13 recorded thereon.

Images