JP7690377B2 - IMAGE PROCESSING METHOD, FERTILIZED EGG EVALUATION METHOD, COMPUTER PROGRAM, AND RECORDING MEDIUM - Google Patents

Description

This invention relates to image processing suitable for imaging fertilized eggs (embryos), particularly fertilized eggs in the cleavage stage, and evaluating their condition.

For example, in assisted reproductive technology aimed at infertility treatment, an embryo (fertilized egg) is fertilized outside the body and cultured for a certain period of time, and then returned to the body. However, the pregnancy success rate (in assisted reproductive technology) is not necessarily high, and it also places a large mental and financial burden on the patient. To solve this problem, methods are being explored to accurately determine the condition of the embryo being cultured.

For example, the Veeck classification is known as a general standard for evaluating the early stages of embryo development, from when a fertilized egg develops into a morula. This is a guideline for classifying the state of a fertilized egg into multiple stages, based on the number and uniformity of size of the blastomeres produced by cleavage, and the rate of fragmentation.

Conventionally, the condition of fertilized eggs has been observed and evaluated by periodically taking microscopic images while placing the eggs in a culture environment using a device known as a time-lapse incubator. However, because optical microscopic images are taken of the eggs from a single line of sight, this is not sufficient to accurately represent the three-dimensional structure of the fertilized egg. For this reason, although there are standards for the evaluation of fertilized eggs, such as the Veeck classification mentioned above, the evaluation is ultimately left to the subjective judgment of experts known as embryologists.

Therefore, a technology that can perform such evaluation quantitatively and objectively is desired. As a technology that is expected to meet this demand, research is being conducted on analyzing three-dimensional images of embryos (fertilized eggs) captured by non-invasive tomographic imaging technology such as optical coherence tomography (OCT) to obtain quantitative information. For example, Patent Document 1, previously disclosed by the applicant of the present application, describes a technology that divides a three-dimensional image of an embryo (fertilized egg) obtained by OCT imaging into multiple regions according to its structure. It is expected that by using this technology, it will be possible to individually extract individual blastomeres and fragmentations from a fertilized egg (embryo) at the cleavage embryo stage, for example.

JP 2019-133429 A

However, as mentioned above, evaluation techniques based on qualitative and subjective evaluation criteria have been put to practical use up until now, so it cannot be said that standards for quantitatively and objectively evaluating fertilized eggs have been fully established. In other words, when capturing fertilized eggs in their three-dimensional structure, converting them into image data, and analyzing them as with the above-mentioned technology, there are currently no clear guidelines for what quantitative information should be sought and how it should be utilized.

This invention has been made in consideration of the above problems, and aims to provide a technology that can obtain quantitative information that can be used as an index for evaluating the state of a fertilized egg from image data obtained by OCT imaging of a fertilized egg in the early developmental stage.

One aspect of the invention is an image processing method comprising the steps of: acquiring three-dimensional image data representing a three-dimensional image of a fertilized egg obtained by imaging the fertilized egg during the cleavage stage with optical coherence tomography; extracting, for each blastomere, a blastomere region that each blastomere occupies in the three-dimensional image based on the three-dimensional image data; calculating the volume of each blastomere region based on the three-dimensional image data; and calculating an index value that indicates the volume variation of the blastomere based on the volume derivation results.

Another aspect of the present invention is a method for evaluating a fertilized egg, comprising the steps of: capturing an image of a fertilized egg in the cleavage stage using optical coherence tomography to obtain three-dimensional image data representing a three-dimensional image of the fertilized egg; extracting, for each blastomere, a blastomere region that each blastomere occupies in the three-dimensional image based on the three-dimensional image data; determining the volume of each blastomere region based on the three-dimensional image data; and determining an index value that indicates the volume variation of the blastomeres based on the volume derivation results.

In these inventions, the index value is calculated based on the maximum of the differences between the volumes of the individual blastomere regions and their average or median values.

In the invention configured in this way, the volume of each blastomere is extracted from the three-dimensional image data obtained by OCT imaging, and an index value that represents the degree of variation is obtained. By using the technology described in the above-mentioned Patent Document 1, it is possible to identify the region of each blastomere contained in a fertilized egg.

Here, in the early stages of embryo development, one of the conditions for a good embryo is that the blastomeres resulting from cleavage are uniform in size. Until now, this uniformity has been evaluated mainly by visual judgment by an experienced person, but the use of OCT images has made it possible to perform more quantitative and accurate measurements based on three-dimensional structures. Specifically, it is possible to determine the volume of each blastomere individually from the size of the spatial area (here referred to as the "blastomere area") occupied by each blastomere identified in the three-dimensional image. Then, from the determined volume, an index value can be obtained that indicates the variation in volume between blastomeres. This makes it possible to establish a more quantitative and objective evaluation technique that replaces the conventional subjective judgment.

In the present invention, the index value is calculated based on the maximum difference between the volume of each blastomere region and the average or median of the volumes. In other words, in the present invention, the index value is calculated by focusing on the individual blastomere regions whose volumes deviate most from the average or median. During the cleavage process, cases may occur in which only a small number of blastomeres are clearly larger (or smaller) than the other blastomeres. In such cases, even if the uniformity of the other blastomeres is good, this does not necessarily apply to the fertilized egg as a whole. However, evaluation criteria for such cases have not yet been established.

When some blastomeres have sizes that deviate from other blastomeres as described above, it is expected that quantitative and objective evaluation of fertilized eggs will proceed more efficiently by providing an index value that indicates this fact and, more preferably, quantitatively indicates the degree of deviation. The index value of the present invention is based on the maximum degree of deviation from the standard blastomere size, represented by the average or median volume, calculated for each blastomere, and can accurately meet such demands.

As described above, the present invention provides an index value that quantitatively and objectively represents the uniformity of blastomere size in early fertilized eggs. This makes it possible to provide quantitative information that is useful for evaluating the condition of fertilized eggs.

1 is a diagram showing an example of the configuration of an image processing apparatus suitable for executing an image processing method according to the present invention; FIG. 2 is a diagram showing a schematic structure of a fertilized egg serving as a sample in this embodiment. 4 is a flowchart showing image processing in this embodiment. FIG. 1 is a diagram for explaining the principle of the blastomere number adjustment process. FIG. 13 is a diagram showing a blastomere number adjustment process. FIG. 1 shows an example of a method for dividing a blastomere into two parts. 13 is a flowchart showing a process for determining a variation index value. FIG. 11 is a diagram showing three examples of a method for calculating an index value. FIG. 1 shows an example of blastomere volume variation. 1 is a flowchart showing an example of a method for evaluating a fertilized egg based on an index value.

Figure 1 shows an example of the configuration of an image processing device suitable for executing the image processing method according to the present invention. This image processing device 1 takes tomographic images of a sample held in a liquid, for example a fertilized egg (embryo) cultured in a culture medium, and processes the resulting tomographic images to create a cross-sectional image showing the structure of one cross section of the sample. It also creates a three-dimensional image of the sample from multiple cross-sectional images. In order to unify the directions in each of the following figures, XYZ orthogonal coordinate axes are set as shown in Figure 1. Here, the XY plane represents the horizontal plane. The Z axis represents the vertical axis, and more specifically, the (-Z) direction represents the vertical downward direction.

The image processing device 1 includes a holding unit 10. The holding unit 10 holds a sample container 11 in a horizontal position, which contains a sample S to be imaged. The sample container 11 is, for example, a flat-bottomed shallow dish-shaped container called a dish, which is open at the top and has a recess capable of holding liquid. A medium M such as a culture fluid is poured into the sample container 11, and a fertilized egg is held inside the sample container 11 as the sample S. In this example, a plurality of samples S are held in the dish as the sample container 11, but this is not limiting.

In this example, multiple samples S are held in a sample container 11 with a single recess, but this is not limiting. For example, the sample container 11 may be a well plate in which multiple recesses called wells are arranged in a single plate-like member. In this case, multiple samples S can be held in multiple wells, one by one. Also, for example, multiple dishes, each holding a sample S, may be held in the holding unit 10 in a horizontally aligned state and used for imaging.

The imaging unit 20 is disposed below the sample container 11 held by the holder 10. The imaging unit 20 uses an OCT (optical coherence tomography) device capable of capturing tomographic images of the object to be imaged non-contact and non-destructive (non-invasive). As will be described in detail later, the imaging unit 20, which is an OCT device, includes a light source 21 that generates illumination light for the object to be imaged, an optical fiber coupler 22, an object optical system 23, a reference optical system 24, a spectrometer 25, and a photodetector 26.

The image processing device 1 further includes a control unit 30 that controls the operation of the device, and a drive section 40 that drives the movable section of the imaging unit 20. The control unit 30 includes a CPU (Central Processing Unit) 31, an A/D converter 32, a signal processing section 33, a 3D restoration section 34, an interface (IF) section 35, an image memory 36, and a memory 37.

The CPU 31 controls the operation of the entire device by executing a predetermined control program, and the control program executed by the CPU 31 and data generated during processing are stored in the memory 37. The A/D converter 32 converts the signal output from the photodetector 26 of the imaging unit 20 according to the amount of received light into digital data. The signal processing unit 33 performs signal processing, described below, based on the digital data output from the A/D converter 32 to create a tomographic image of the object to be imaged. The 3D reconstruction unit 34 has the function of creating a three-dimensional image (3D image) of the captured cell clump based on image data of the captured multiple tomographic images. The image data of the tomographic images created by the signal processing unit 33 and the image data of the three-dimensional image created by the 3D reconstruction unit 34 are appropriately stored and saved in the image memory 36.

The interface unit 35 is responsible for communication between the image processing device 1 and the outside. Specifically, the interface unit 35 has a communication function for communicating with external devices, and a user interface function for accepting operation input from the user and notifying the user of various information. For this purpose, the interface unit 35 is connected to an input device 351, such as a keyboard, mouse, or touch panel, capable of accepting operation input related to device function selection and operating condition setting, and a display unit 352, such as a liquid crystal display, for displaying various processing results such as the tomographic images created by the signal processing unit 33 and the stereoscopic images created by the 3D reconstruction unit 34.

In the imaging unit 20, a low-coherence light beam containing a wide band of wavelength components is emitted from a light source 21 having a light-emitting element such as a light-emitting diode or a superluminescent diode (SLD). For the purpose of imaging a sample such as a cell, it is preferable to use, for example, near-infrared light in order to allow the incident light to reach the inside of the sample.

The light source 21 is connected to an optical fiber 221, which is one of the optical fibers that make up the optical fiber coupler 22. The low-coherence light emitted from the light source 21 is branched by the optical fiber coupler 22 into light for the two optical fibers 222 and 224. The optical fiber 222 constitutes an object-system optical path. More specifically, the light emitted from the end of the optical fiber 222 enters the object optical system 23.

The object optical system 23 includes a collimator lens 231 and an objective lens 232. The light emitted from the end of the optical fiber 222 enters the objective lens 232 via the collimator lens 231. The objective lens 232 has the function of converging the light (observation light) from the light source 21 onto the sample as a light beam, and the function of collecting the reflected light emitted from the sample and directing it toward the optical fiber coupler 22. Although a single objective lens 232 is shown in the figure, multiple optical elements may be combined. The reflected light from the object to be imaged enters the optical fiber 222 as signal light via the objective lens 232 and the collimator lens 231. The optical axis of the objective lens 232 is perpendicular to the bottom surface 111 of the sample container 11, and in this example, the optical axis direction coincides with the vertical axis direction.

The drive unit 40 is controlled by the CPU 31. That is, the CPU 31 gives a control command to the drive unit 40, and in response to this, the drive unit 40 moves the imaging unit 20 in a predetermined direction. More specifically, the drive unit 40 moves the imaging unit 20 in the horizontal direction (XY direction) and the vertical direction (Z direction). The horizontal movement of the imaging unit 20 changes the imaging range in the horizontal direction. In addition, the vertical movement of the imaging unit 20 changes the focal position in the optical axis direction of the objective lens 232 with respect to the sample S, which is the object to be imaged.

A portion of the light incident on the optical fiber coupler 22 from the light source 21 enters the reference optical system 24 via the optical fiber 224. The reference optical system 24 includes a collimator lens 241 and a reference mirror 243, which together with the optical fiber 224 constitute the reference system optical path. Specifically, light emitted from the end of the optical fiber 224 enters the reference mirror 243 via the collimator lens 241. The light reflected by the reference mirror 243 enters the optical fiber 224 as reference light.

The reflected light (signal light) reflected by the surface or internal reflection surface of the sample and the reference light reflected by the reference mirror 243 are mixed in the optical fiber coupler 22 and enter the photodetector 26 via the optical fiber 226. At this time, interference occurs between the signal light and the reference light due to the phase difference, but the spectrum of the interference light differs depending on the depth of the reflection surface. In other words, the spectrum of the interference light contains information about the depth direction of the imaged object. Therefore, the reflected light intensity distribution in the depth direction of the imaged object can be obtained by detecting the amount of light by dispersing the interference light by wavelength and performing a Fourier transform on the detected interference signal. An OCT imaging technology based on this principle is called Fourier Domain OCT (FD-OCT).

In this embodiment, the imaging unit 20 has a spectroscope 25 on the optical path of the interference light from the optical fiber 226 to the photodetector 26. The spectroscope 25 may be, for example, one that uses a prism or one that uses a diffraction grating. The interference light is split into wavelength components by the spectroscope 25 and received by the photodetector 26.

By performing a Fourier transform on the interference signal output from the photodetector 26 in response to the interference light detected by the photodetector 26, the reflected light intensity distribution in the depth direction at the incident position of the illumination light in the sample, i.e., in the Z direction, can be obtained. By scanning the light beam incident on the sample container 11 in the X direction, the reflected light intensity distribution in a plane parallel to the XZ plane can be obtained, and from the result, a tomographic image of the sample S with the plane as a cross section can be created. Since the principle is well known, a detailed explanation will be omitted.

Furthermore, by changing the beam incidence position in the Y direction in multiple stages and capturing a tomographic image each time, it is possible to obtain a large number of tomographic images of the sample in cross sections parallel to the XZ plane. By reducing the scanning pitch in the Y direction, it is possible to obtain image data with a resolution sufficient to grasp the three-dimensional structure of the sample. From these tomographic image data, three-dimensional image data (so-called voxel data) corresponding to a three-dimensional image of the sample can be created. In this way, the image processing device 1 has the function of creating a tomographic image of the sample S, which is the object to be imaged, in any cross section parallel to the XZ plane, and the function of creating a three-dimensional image of the sample S from multiple different tomographic images.

In this embodiment, the image processing device 1 configured as described above is used to capture images of fertilized eggs (embryos) and perform quantitative evaluation of the images. Specifically, a fertilized egg in the cleavage stage is imaged using OCT to obtain a three-dimensional image, which is then analyzed using image processing to calculate an index value that indicates the degree of variation in size of the blastomeres that make up the fertilized egg. The specific details of this process are described below.

Figure 2 is a diagram showing a schematic structure of a fertilized egg (embryo) that serves as a sample in this embodiment. As is already known, when an egg is fertilized, cleavage begins, and a blastocyst is formed after passing through a stage called a morula. The image processing method of this embodiment is suitable for analyzing and evaluating a fertilized egg in the cleavage stage, for example, from immediately after fertilization to the morula stage.

Figure 2(a) shows a schematic diagram of the structure of a fertilized egg at an early stage (for example, from the four-cell stage to the morula stage). The external shape of the fertilized egg (embryo) E is roughly spherical. Its surface is covered with a layer Z of a jelly-like glycoprotein called the zona pellucida, and inside it are multiple cells C that are produced by cell division of the fertilized egg. These cells at this stage are also called "blastomeres."

As cleavage progresses, the number of blastomeres C increases. When culture is progressing well, the inside of the zona pellucida Z is occupied by multiple blastomeres C that are relatively large and uniform in size. In other words, in a fertilized egg in the cleavage stage, whether the blastomeres C that make up the fertilized egg are uniform in size is one indicator of whether the culture condition is good or not.

The Veeck classification also uses the variation in blastomere size as an index for evaluating fertilized eggs. However, as mentioned above, the evaluation of the degree of variation is left to the observer's visual judgment. In this embodiment, an index value that quantitatively represents the degree of this variation is automatically calculated by analyzing images captured by OCT.

Although details will be described later, for the above purpose, the image processing of this embodiment includes processing for extracting the areas occupied by each blastomere C from the three-dimensional image of the fertilized egg E obtained by OCT imaging. As the image processing for this purpose, for example, the technology described in Patent Document 1 can be suitably applied. Patent Document 1 discloses a technology for performing image processing on the three-dimensional image of the fertilized egg, separating and extracting the areas corresponding to the zona pellucida Z, the blastomere C, and minute objects called fragments, and further dividing the areas corresponding to each blastomere C individually. A local thickness calculation and a watershed algorithm are used to divide the areas.

Figure 2(b) is a diagram showing an example of a blastomere extracted from an OCT image. More specifically, this figure shows the application of the image processing method described in Patent Document 1 to an OCT image of a fertilized egg at the cleavage stage. Here, the area corresponding to the zona pellucida extracted from the three-dimensional image of the fertilized egg is erased, and the areas corresponding to each blastomere C are displayed with different brightness. Although multiple blastomeres C are integrated in a state of close contact with other blastomeres, they can be separated by image processing. This method can also be applied to the extraction of blastomere C in this embodiment. Therefore, in the following explanation, a detailed explanation of the blastomere extraction process will be omitted.

Figure 3 is a flow chart showing image processing in this embodiment. This processing is realized by the CPU 31 of the control unit 30 executing a prepared control program and causing each part of the device to perform a predetermined operation. First, three-dimensional image data corresponding to a three-dimensional image of the cleavage stage fertilized egg to be evaluated is acquired (step S101). The image data may be acquired by newly performing OCT imaging of the fertilized egg as the sample S, or image data acquired in a previous imaging may be read from the image memory 36. Furthermore, image data stored in an external storage device or the like may be acquired via an electric communication line.

Based on the three-dimensional image data of the fertilized egg thus obtained, blastomere regions corresponding to each blastomere are extracted from the three-dimensional image of the fertilized egg (step S102). More specifically, the image processing described in Patent Document 1 is applied to the three-dimensional image data to divide the three-dimensional image into regions corresponding to the zona pellucida and blastomeres (and fragments, if necessary). For blastomeres, regions are divided on a cell-by-cell basis to identify the blastomere regions corresponding to each individual blastomere. Then, by erasing objects other than those corresponding to the blastomere regions, the blastomere regions corresponding to each individual blastomere are extracted.

Once the regions corresponding to the individual blastomeres have been identified in this way in the three-dimensional image, the number and volume of the blastomeres are measured (step S103). In the following, the number of blastomeres thus determined is represented by the symbol X. The volume of each blastomere can be represented, for example, by the number of voxels that make up the blastomere region. In addition, the volume of the blastomere in real space can be determined by multiplying the number of voxels by the volume per voxel in real space.

Here, it is determined whether the counted number of blastomeres X is a number that corresponds to a power of 2 (step S104). If this is the case (YES in step S104), then a blastomere number adjustment process (step S105) is executed. On the other hand, if the number of blastomeres X is not a power of 2 (NO in step S104), step S105 is skipped. The blastomere number adjustment process is a process for suppressing erroneous evaluation caused by variations in the timing of cell division during the cleavage process in the blastomere variation evaluation described below. The contents of this process are explained below.

Figure 4 is a diagram explaining the principle of the blastomere number adjustment process. In the "Pattern" column of the figure, a circle represents one blastomere. As shown as case (a) in Figure 4, cleavage generally progresses by repeating a cycle in which one cell divides into two for each generation, and the number of blastomeres doubles with each generation: 1, 2, 4, ... In other words, the number of blastomeres X for each generation can be expressed by a geometric progression with 1 as the first term and 2 as the common ratio.

However, the division of each cell does not occur strictly at the same time. For this reason, in an image captured as a still image at a certain time, cells before and after division may be mixed. In this case, the number of blastomeres X may not follow the geometric progression described above. Even in this embodiment, the number of blastomeres X found in step S103 may not follow the regularity expressed by the geometric progression described above.

Even if the number of blastomeres does not match the pattern, as long as it is due to the relationship between the timing of cell division and the timing of imaging, it does not necessarily indicate a problem with the quality of the fertilized egg. However, it can lead to misassessment when trying to evaluate the condition of the fertilized egg based on, for example, the variation in the size of the blastomeres.

In other words, in a fertilized egg, it is considered that the size of each blastomere is uniform, that is, there is little variation in size, which is considered to be in a good state. Generally, pre-division blastomeres are larger than post-division blastomeres. Therefore, when pre-division blastomeres and post-division blastomeres are mixed as described above, the variation found between them will naturally be large. However, because this variation is a temporary occurrence due to differences in the timing of cell division, the evaluation results will be inappropriate if this result is used as is for evaluation.

Therefore, in this embodiment, if the number of blastomeres X obtained as a result of the region division does not conform to the above regularity, it is assumed that there is a mixture of pre-division and post-division blastomeres. Based on the general knowledge that pre-division blastomeres are larger than post-division blastomeres, it is presumed that the blastomere with the largest volume is the pre-division blastomere. When such a blastomere divides into two, each blastomere will be approximately half the size of the original blastomere.

Therefore, it is believed that by dividing the original blastomere region so that its volume is divided in half, it is possible to simulate the size of each blastomere after division. By dividing one blastomere region presumed to be before division into two pseudo blastomere regions in this way, it is possible to treat them on the same level as other blastomeres after division, that is, to evaluate them as cells of the same generation. Although the blastomere regions created by division do not necessarily represent the actual shape after division, they can be suitably used for the purpose of deriving quantitative information such as size.

As mentioned above, the number of blastomeres X obtained from one still image can take various values depending on the timing of the image capture. For example, as shown as case (b) in FIG. 4, a case in which the number of blastomeres X is 3 can be a case in which there are two blastomeres after division from a state in which the number of blastomeres is 2, and one blastomere before division, as shown by the dashed line in the figure. Also, a case in which the number of blastomeres X is 5 can be a case in which there are two blastomeres after division from a state in which the number of blastomeres is 4, and three blastomeres before division.

Similarly, the case (d) where the number of blastomeres X is 6 can be considered as a combination of four blastomeres formed by division of two blastomeres from the state where the number of blastomeres is four, and two blastomeres before division, and the case (e) where the number of blastomeres X is 7 can be considered as a combination of six blastomeres formed by division of three blastomeres from the state where the number of blastomeres is four, and one blastomere before division. More generally, the number of blastomeres X can be expressed by the following formula using a natural number N:
2 ^N <X<2 ^N+1 ... (Formula 1)
Of these, (2 ^N+1 −X) are presumed to be pre-division blastomeres.

Therefore, by dividing each of the (2 ^N+1 −X) blastomeres estimated to be undivided into two and rounding the apparent number of blastomeres X to any number in the geometric progression (4, 8, ...), it is possible to reduce the risk of misassessment in the quantitative evaluation of the fertilized egg. The process for this purpose is the blastomere number adjustment process (step S105).

In addition to the above-mentioned cases where blastomere numbers do not follow the regularity of a geometric progression, there may also be cases where there is an abnormality in the cleavage process, such as the inactivation of some blastomeres. In such cases, the above-mentioned blastomere number adjustment process does not necessarily return appropriate results. However, in such cases, large variations will also appear in the process of evaluating the variation in blastomeres, which will be described later, and it is possible to avoid mistakenly evaluating the fertilized egg as being in good condition.

Fig. 5 shows the process of adjusting the number of blastomeres. Fig. 5(a) is a flow chart showing the process of adjusting the number of blastomeres. When the extracted number of blastomeres X cannot be expressed as ^2N , (2N ⁺¹ -X) blastomere regions with the largest volumes are selected based on the above-mentioned principle (step S201). These blastomere regions are presumed to correspond to blastomeres before division. Therefore, these blastomere regions are divided into two equal parts. There are various methods for dividing a blastomere region into two equal parts.

For example, there is a method of dividing a blastomere region into two equal parts by dividing the region along a plane perpendicular to its long axis. Specifically, an axis corresponding to the long axis of each blastomere region is identified (step S202), and a cutting plane for dividing the blastomere region is provisionally set (step S203). The cutting plane is a plane perpendicular to the long axis. The ratio of the volumes of the two regions obtained by dividing the blastomere region along this cutting plane is calculated (step S204), and it is determined whether or not the ratio is within an allowable range (step S205).

For the purpose of dividing one blastomere region into two equal parts, it is ideal for the volume ratio of the two divided blastomere regions to be 1:1. However, in reality, it is sufficient if the two blastomere regions are roughly equal, and the above ratio can be left with a certain degree of margin based on 1:1 as the standard, and this can be used as the acceptable range, and it can be judged whether the volume ratio falls within this acceptable range.

If the volume ratio is within the allowable range (YES in step S204), the region division result at the cutting plane set at that time can be adopted. On the other hand, if the volume ratio is outside the allowable range (NO in step S205), the cutting plane is set again, and steps S203 to S205 are repeated until the volume ratio falls within the allowable range. In this way, one blastomere region can be divided into two blastomere regions of approximately equal volume. The above division is performed for all blastomere regions selected in step S201. As a result, the number of blastomeres X after division can be expressed as ^2N using the natural number N.

FIG. 5B is a diagram showing the relationship between the number of blastomeres X obtained in step S103 and the number of blastomeres (2 ^N+1 −X) to be divided at that time. For example, when the number of blastomeres X obtained in step S103 is 3, the one blastomere with the largest volume is divided into two equal parts, resulting in a final number of blastomeres of 4. When the number of blastomeres X is 5, the three blastomeres with the largest volume are each divided into two equal parts, resulting in a final number of blastomeres of 8. The same can be considered for any number of blastomeres X of 6 or more. In this way, by dividing some blastomeres according to the value of the number of blastomeres X, the final number of blastomeres can be rounded to 2 ^N . This makes it possible to avoid erroneous evaluation due to size variations caused by a difference in timing between cell division and imaging.

Figure 6 shows an example of a method for dividing a blastomere into two. In the above-mentioned method, as shown in Figure 6(a), a cut surface Sc perpendicular to the long axis Lx of the blastomere B1 selected in step S201 is provisionally set. Then, the cut surface Sc is moved along the long axis Lx so that the two regions divided by this cut surface Sc have approximately the same volume, thereby dividing the blastomere region B1 into two regions having approximately the same volume.

In addition to the above, as shown in FIG. 6B, for example, by performing further region division processing, such as a division processing using the Watershed algorithm, on the blastomere region B2 having an indefinite shape, it is possible to divide the blastomere region B2 into two regions based on its shape. However, in order to quantitatively express the variation in size of the blastomeres in this embodiment, the focus is on making the volumes after division approximately the same, rather than on division based on shape. Therefore, even in such cases, if a long axis Lx is set for the blastomere region B2, and if the volumes of the two divided regions differ significantly, the cutting plane Sc set by the Watershed algorithm is moved along the long axis Lx as necessary, it is possible to bring the volume ratio of the two regions closer to 1:1.

Returning to Fig. 3, the image processing in this embodiment will be described further. As described above, by adjusting the number of blastomeres as necessary, at the time of step S106, a blastomere before division is also treated as two blastomeres that have already divided, and the number of blastomeres X is a value expressed as ^2N . An index value that quantitatively represents the size, specifically the volume variation, of these blastomeres is calculated (step S106).

As mentioned above, in fertilized eggs in the cleavage stage, uniformity in the size of the blastomeres is considered to indicate a good condition. Until now, trained users such as embryologists have visually evaluated the degree of variation in blastomere size. Therefore, if an index value that quantitatively represents the degree of variation in blastomere size could be automatically calculated and presented from images of fertilized eggs, it is expected that the workload of users evaluating the condition of fertilized eggs could be greatly reduced.

Therefore, one possible use of the obtained index value is to display and output the index value on the display unit 352 and present it to the user (step S107), thereby assisting the user in evaluating the fertilized egg. In addition, as described below, another possible use is to compare the obtained index value with an appropriate threshold value to automatically determine whether the condition of the fertilized egg is good or bad.

One way to quantify the variation in the volumes found for each blastomere region is to statistically process the volume values and, for example, express them quantitatively using a variance value. Although this method may be applied, when the population is small, as in this case, it is also possible to quantify the variation using, for example, the following method.

7 is a flow chart showing the process of calculating the variation index value in this embodiment. The volume values of X (=2 ^N ) blastomeres extracted in the process up to step S105 (FIG. 3) are sorted according to their size, for example, in ascending order (step S301). The average value of these volumes is calculated (step S302). The volume of each blastomere is then normalized by the calculated average volume value (step S303), and the normalized volume value is applied to a predetermined formula to calculate an index value that quantitatively represents the volume variation of the blastomeres (step S304).

The index value may be one that increases as the variation in volume between blastomeres increases, and various formulas for calculating it are possible. From the viewpoint of allowing the user to easily grasp the degree of variation intuitively, it is desirable for the index value to be 0 or 1 when there is no variation, and to increase as the variation increases. Below, three calculation methods are proposed as specific examples of methods for calculating such index values.

Figure 8 shows three examples of the index value calculation method. Here, as a simple example for ease of understanding, we will take up the case where the number of blastomeres X after the blastomere number adjustment process is 4. In the following explanation, the four blastomeres will be distinguished by the numbers 1, 2, 3, and 4 when their volumes are sorted in ascending order, and their respective volumes will be represented by the symbols V1, V2, V3, and V4. The average value of these volumes will be represented by the symbol Vm. In each figure, the volume of each blastomere is represented as a normalized volume value after normalization by the volume average value Vm.

In the first example shown in Fig. 8(a), the index value is the increase rate of volume with respect to the blastomere number when the blastomeres are arranged in ascending order of volume, that is, the slope of the line drawn in Fig. 8(a). For example, the index value Vi is calculated using the minimum (V1/Vm) and maximum (V4/Vm) of the normalized volume as follows:
Vi=(V4/Vm-V1/Vm)/(4-1)... (Formula 2)
When there is absolutely no variation in volume between blastomeres, the volumes V1 and V4 are equal, and the index value Vi expressed by (Equation 2) is zero. On the other hand, the greater the variation, the greater the difference between the volumes V1 and V4, and the greater the index value Vi expressed by (Equation 2). If (Equation 2) is generalized further, the following equation can be used for the number of blastomeres X:
Vi=(Vx/Vm-V1/Vm)/(X-1)... (Formula 3)
It becomes.

The significance of normalizing the volume of each blastomere by its average value here is to eliminate the influence of the size of each blastomere due to the progress of cleavage or due to individual differences in fertilized eggs. For example, since each blastomere becomes smaller as cleavage progresses, it is effective to use normalized volumes to ensure that the index value of variation is not influenced by the absolute value of the volume. Note that here, the volumes V1 to Vx of each blastomere are normalized by the average volume Vm before being substituted into the formula, but the technical significance will not change even if the original volumes V1 to Vx are substituted into the formula as they are and the calculated values are normalized by the average volume Vm. These points are also true in the following examples.

により表すことができる。 In the second example shown in FIG. 8(b), the index value Vi is calculated based on the cumulative value of the normalized volume of each blastomere. Specifically, for the blastomere number 2, the cumulative value of the volumes of the two blastomeres 1 and 2 with the smallest volumes is plotted. Similarly, for the blastomere number 3, the cumulative value of the volumes of the three blastomeres 1 to 3 in ascending order of volume is plotted. Furthermore, for the largest blastomere 4, the cumulative value of the volumes of all blastomeres is plotted. The slope of the straight line in the graph thus created is taken as the index value Vi. Generalizing this, the index value Vi can be calculated using the following equation:

It can be expressed as follows:

により表される。 In the third example shown in Fig. 8(c), the index value Vi is calculated based on the rate of variation from the average value of the normalized volume of each blastomere. Specifically, the difference between the normalized volume value and the average normalized volume (i.e., 1) is calculated for each blastomere, and the sum of the absolute values is divided by the number of blastomeres to determine the index value Vi. In general, the index value Vi is calculated by the following formula:

It is expressed by:

With any of these calculation methods, it is possible to obtain an index value Vi that is minimum when there is no variation in volume between blastomeres and increases as the variation increases. In other words, the size variation between blastomeres can be quantitatively expressed by the magnitude of the index value Vi. Note that instead of normalizing by the average volume, normalization by the median, minimum, or maximum value, etc., can also be considered. In particular, when the volume of one blastomere is significantly larger (or smaller) than the others, the average value may be significantly biased due to its influence. For example, by using the median, this problem can be avoided.

In order to deal with cases in which some blastomeres are significantly larger (or smaller) than the others, the following method of calculating index values can also be applied. Here again, the specific content will be explained by taking the case where the number of blastomeres X after the blastomere number adjustment process is 4. Note that the processing other than the calculation of the index value can be the same as in the above embodiment.

Figure 9 shows examples of blastomere volume variation. In the example shown in Figure 9(a), the normalized volumes of the blastomeres represented by blastomere numbers 1 to 4 are 0.9, 0.9, 1.1, and 1.1, respectively. In this case, if the index value Vi is calculated using, for example, (Equation 5), Vi = 0.1 is obtained.

On the other hand, in the example shown in Figure 9(b), the normalized volumes of the blastomeres represented by blastomere numbers 1 to 4 are 0.9, 0.9, 1.0, and 1.2, respectively, and only the volume of the blastomere represented by number 4 is significantly larger than the others. In the example shown in Figure 9(c), the normalized volumes of the blastomeres represented by blastomere numbers 1 to 4 are 0.8, 1.0, 1.1, and 1.1, respectively, and only the volume of the blastomere represented by number 1 is significantly smaller than the others. In these examples as well, the index value Vi calculated by (Equation 5) is 0.1 in all cases.

で表される指標値Ｖｉを導入することを考える。ここで、（式６）および後述の（式９）における演算子Ｍａｘ（ｐ1，…，ｐn）は、複数の数値ｐ1，ｐ2，…，ｐnのうちの最大値を返す関数を表すものとする。 In this way, in a plurality of cases in which there is a clear difference in terms of uniformity of the blastomeres, there may be cases in which there is no significant difference in the index value Vi calculated by a certain calculation method.

Here, the operator Max(p, ..., p) in (Formula 6) and (Formula 9) described later represents a function that returns the maximum value among multiple numerical values p, p, ..., p.

In other words, the index value Vi according to this definition is calculated as the maximum value among the differences between the volume calculated for each blastomere and the average volume normalized by the average volume. According to this definition, the index value Vi in the example of FIG. 9(a) is 0.1, while the index value Vi in the examples of FIG. 9(b) and FIG. 9(c) is both 0.2, which is larger than the value in the example of FIG. 9(a).

The index value Vi according to this definition decreases as the variation in blastomere volume decreases, but increases when there is a blastomere that is significantly larger than the others. The index value Vi also increases when there is a blastomere that is significantly smaller than the others, and the numerical values are approximately the same if the degree of deviation from the average value is the same. This shows that when some blastomeres are significantly larger (or smaller) than the others, the index value Vi calculated by (Equation 6) quantitatively represents the degree of deviation. It is also possible to quantitatively represent the uniformity of blastomere volume using such index values.

The index value Vi calculated using these calculation methods can be used as a reference by the user when determining the state of the fertilized egg. It is also possible to automatically determine the state of the fertilized egg from the index value, as shown below.

Figure 10 is a flow chart showing an example of a method for evaluating a fertilized egg based on an index value. This evaluation method can be realized by adding several processing steps after the image processing shown in Figure 3. Therefore, illustration and description of steps S101 to S106, in which similar processing can be adopted, will be omitted. As in the above processing, in step S107, the calculated index value Vi is displayed. Then, the index value Vi is compared with a threshold value Vth preset for the index value (step S108). Here, if the index value Vi is equal to or less than the threshold value (YES in step S108), the variation in the volume of the blastomeres is relatively small, and therefore the condition of the fertilized egg is judged to be good (step S109). On the other hand, if the index value Vi exceeds the threshold value (NO in step S108), the variation in the volume of the blastomeres is large, and the condition of the fertilized egg is judged to be poor (step S110).

In this way, it is in principle possible to automatically judge the quality of a fertilized egg from the calculated index value Vi. However, there is not enough knowledge about how to set a threshold value for the index value, so it is currently considered difficult to make a definitive automatic judgment based on the calculation results alone. Therefore, even if a tentative judgment result is shown as reference information, the index value Vi itself can be presented to the user, leaving the final judgment to the user.

In addition, in order to assist the user in comprehensively evaluating the fertilized egg from various viewpoints, it is also effective to present a combination of index values calculated by a plurality of calculation methods. For example, the method may be such that some of the index values calculated by the calculation methods (Formula 1) to (Formula 6) are displayed side by side, or the total value of these may be presented as one index value. In each of the calculation methods of the above formulas, the better the uniformity of the blastomeres, the smaller the resulting numerical value. Therefore, the total value of several index values will also be small if the uniformity of the blastomeres is good. On the other hand, when any of the index values indicates low uniformity, that is, the value is large, the total value will also be large. In other words, the total value will be small only when each of the individual index values calculated by different methods indicates high uniformity. Therefore, it is possible to perform a comprehensive evaluation of the fertilized egg using this total value.

でそれぞれ表される各種のものを用いることが可能である。 It is also possible to perform a comprehensive evaluation by multivariate analysis using multiple index values. For this purpose, it is preferable to present various index values obtained by statistically processing the measured volumes of each blastomere. Indices that can be used for such purposes include those calculated by the above-mentioned calculation methods, as well as the following (Equation 7) to (Equation 10):

It is possible to use various types of the above-mentioned.

Here, the operator Min(p1, ..., pn) in (Equation 8) and (Equation 9) represents a function that returns the minimum value among multiple numerical values p1, p2, ..., pn. Of these, (Equation 7) represents the sum of the differences between the volume of each blastomere and the volume average value, and compared to the index value obtained by (Equation 6), the calculation of division by the number of blastomeres X is omitted, but it is equivalent in terms of technical significance. Since (Equation 6) normalizes by the number of blastomeres, the difference is that the index value can be evaluated on the same scale regardless of the number of blastomeres, such as the 4-cell stage or the 8-cell stage, but these can be used according to the purpose.

Furthermore, (Equation 8) returns the minimum value of the difference between the volume of each blastomere and the volume average, and (Equation 9) returns the range of the difference between the volume of each blastomere and the volume average. Furthermore, (Equation 10) returns the standard deviation, which indicates the distribution of the difference between the volume of each blastomere and the volume average. By appropriately combining the index values obtained by these various calculation methods, it is possible to quantitatively and objectively express the uniformity of the blastomeres.

As described above, in the image processing method of this embodiment, the blastomere regions occupied by individual blastomeres are identified from a three-dimensional image of a fertilized egg at the cleavage stage obtained by OCT imaging, and their number and volume are measured. An index value that quantitatively indicates the variation in volume between blastomeres is then calculated using a predetermined formula. This index value can effectively assist the user in evaluating the fertilized egg, and can also be important quantitative information in automating the evaluation process.

In addition, when the extracted number of blastomeres does not follow a pattern expressed by a power of 2, the number of blastomeres is adjusted by dividing some of the blastomeres with the largest volume in half. This eliminates the effects of temporary size variations between blastomeres caused by variations in the timing of cell division, and provides quantitative information useful for evaluating fertilized eggs.

According to this embodiment, when a user performs an evaluation of a fertilized egg based on the Veeck classification, the user's work can be effectively supported by quantitatively indicating the variation in blastomere size, which is one of the evaluation indexes. For example, when there are multiple fertilized eggs, the degree of variation in the blastomeres among them can be quantitatively indicated on a unified scale, making it easier to compare the states of the fertilized eggs.

The present invention is not limited to the above-described embodiment, and various modifications other than those described above can be made without departing from the spirit of the present invention. For example, the image processing device 1 of the above embodiment has a function of OCT imaging of the sample S and a function of executing image processing based on the imaging data. However, the image processing method of the present invention can also be executed by a computer device that does not have an imaging function itself, but acquires imaging data obtained by imaging with another device that has an imaging function. To make this possible, the present invention may be implemented as a software program that causes a computer device to execute each of the processing steps in FIG. 3.

Such a program can be distributed, for example, by downloading it via a telecommunications line such as the Internet, or by distributing a computer-readable recording medium on which the program is non-temporarily recorded. In addition, by having an existing OCT imaging device load the program via an interface, the present invention can be implemented by the device.

In addition, in the above embodiment, the technology described in Patent Document 1 is used to extract blastomere regions from a three-dimensional image of a fertilized egg. However, the method is not limited to the above and other methods may be used as long as they can properly separate the individual blastomeres that make up the fertilized egg.

As described above by way of example of specific embodiments, in the image processing method according to the present invention, the number X of extracted blastomere regions is calculated based on the following formula:
^2N < X < ^2N+1 (N is a natural number)
When the index value is expressed as: (2 ^N+1 −X) of the blastomere regions with the largest volume may be regarded as two blastomere regions with the volume divided in half to obtain the index value. In the cleavage process, the number of cells (blastomeres) generally doubles each time, but there is some variation in the timing of cell division. For this reason, depending on the timing of imaging, a still image may contain a mixture of blastomeres before and after division, and the number of blastomeres may not be a power of two. In such a case, the size of the blastomere after division can be expressed in a pseudo manner by dividing the blastomere with the largest volume into two on the assumption that it is the one before division. In this way, it is possible to avoid the variation in size caused by the mixture of blastomeres before and after division causing erroneous evaluation.

For example, an index value based on the maximum difference between the volume of each blastomere region and its average or median, and an index value based on the average or median volume may be calculated. The size and variation of each blastomere varies from one fertilized egg to another, and there may be cases where a single index value alone cannot accurately represent the state of the fertilized egg. By combining multiple types of index values, it is possible to deal with such situations. For example, a combination of an index value based on the maximum difference and an index value based on the average or median is suitable for such purposes.

For example, a single index value may be calculated based on the maximum value of the differences and the average or median value. For example, the sum of the index values calculated based on these two types of values may be used as the final index value. With this configuration, the values calculated using multiple calculation methods that quantitatively represent the state of the fertilized egg can be ultimately consolidated into a single index value to obtain a numerical value that comprehensively represents the state of the fertilized egg.

For example, the volume of each blastomere region may be normalized by the average or median of the volumes of all blastomere regions, and the index value may be calculated based on the normalized volume. With this configuration, it is possible to calculate an index value that changes as cleavage progresses or is not affected by differences in the size of each blastomere due to individual differences in fertilized eggs.

For example, the Watershed algorithm can be suitably applied to extract blastomere regions. Inside a fertilized egg, each blastomere is in close contact with one another, and it is not easy to identify their boundaries in an OCT image. The Watershed algorithm, which uses the unevenness of the object surface as a clue to divide the object into multiple regions, is extremely useful as a region segmentation process for a fertilized egg in the division stage, where multiple cells are densely packed while maintaining their individual shapes to some extent.

Furthermore, the image processing method and fertilized egg evaluation method according to the present invention may further include a step of comparing the determined index value with a predetermined threshold value to evaluate the state of the fertilized egg. Evaluating the variation in size between blastomeres as an index for evaluating the state of a fertilized egg has been done for some time, but it has been mainly left to the subjective judgment of the observer. If this could be done automatically based on quantitative information, it would be possible to greatly reduce the workload of the observer and effectively suppress variation in the evaluation results.

This invention is suitable for supporting the observation and evaluation of fertilized eggs (embryos) in the cleavage stage, and can be used, for example, to support the evaluation of the condition of cultured embryos, for the purpose of selecting good fertilized eggs with a higher pregnancy success rate in assisted reproductive technology.

REFERENCE SIGNS LIST 1 Image processing device 10 Holding unit 11 Sample container 20 Imaging unit 30 Control unit 31 CPU
33 Signal processing section 352 Display section B1, B2 Blastomere area C Cell (blastomere)
E Fertilized egg (embryo)
S sample

Claims (11)

acquiring three-dimensional image data representing a three-dimensional image of a fertilized egg during the cleavage stage, the three-dimensional image data being obtained by optical coherence tomography imaging of the fertilized egg;
extracting, for each blastomere, a blastomere area occupied by each blastomere in the three-dimensional image based on the three-dimensional image data;
determining a volume of each of the blastomere regions based on the three-dimensional image data;
and calculating an index value indicating a volume variation of the blastomere based on the volume derivation result, the index value being:
An image processing method, in which the volume is determined based on the maximum value among the differences between the volumes of the individual blastomere regions and their average value or median value. The number X of the extracted blastomere regions is calculated by the following formula:
^2N < X < ^2N+1 (N is a natural number)
The image processing method according to claim 1 , wherein, when expressed as ^: The image processing method according to claim 1 or 2, which calculates the index value based on the maximum value and the index value based on the average or median value of the volumes of the individual blastomere regions. The image processing method according to any one of claims 1 to 3, wherein the index value is calculated based on the maximum value and the average or median value of the volumes of the individual blastomere regions. An image processing method according to any one of claims 1 to 4, in which the volume of each of the blastomere regions is normalized by the average or median of the volumes of all of the blastomere regions, and the index value is calculated based on the normalized volume. The image processing method according to any one of claims 1 to 5, further comprising a step of evaluating the state of the fertilized egg based on a comparison between the obtained index value and a predetermined threshold value. The image processing method according to any one of claims 1 to 6, wherein a Watershed algorithm is used to extract the blastomere region. A computer program for causing a computer to execute each step of the image processing method according to any one of claims 1 to 7. A computer-readable recording medium on which the computer program according to claim 8 is non-temporarily recorded. A step of capturing an optical coherence tomography image of a fertilized egg at the cleavage stage to obtain three-dimensional image data representing a three-dimensional image of the fertilized egg;
extracting, for each blastomere, a blastomere area occupied by each blastomere in the three-dimensional image based on the three-dimensional image data;
determining a volume of each of the blastomere regions based on the three-dimensional image data;
and calculating an index value indicating a volume variation of the blastomere based on the volume derivation result, the index value being:
A method for evaluating a fertilized egg, comprising determining the volume of each blastomere region based on the maximum value among the differences between the volume of each blastomere region and the average or median of the volumes. The method for evaluating a fertilized egg according to claim 10, further comprising a step of comparing the determined index value with a predetermined threshold value to evaluate the state of the fertilized egg.

Images