JP7385828B2 - Malignant melanoma diagnostic device - Google Patents

Description

Application of Article 30, Paragraph 2 of the Patent Act January 8, 2019, 24th National College of Technology Symposium in Oyama Lecture Abstract F-22, Presenters: Momoko Araki, Umio Fujii, Ryoaki Keida, Shunsuke Akiguchi

Application of Article 30, Paragraph 2 of the Patent Act May 11, 2019, 2019 Japan Heat Transfer Hokuriku-Shinetsu Branch General Meeting/Spring Seminar, Presenters: Momoko Araki, Shunsuke Akiguchi, Ryoaki Keida, Noboru Momose, Tajiri Tomoki, Masashi Yaga, Tsugunobu Ando

Application of Article 30, Paragraph 2 of the Patent Act August 31, 2019, 2019 Hokuriku Branch Federation Conference of Electrical and Information Related Societies, Presenters: Momoko Araki, Shunsuke Akiguchi, Ryōaki Keida, Tomoki Tajiri, Tsugunobu Ando, Masashi Yaga

The present invention relates to a diagnostic device that measures the state of blood flow in a living body using a laser Doppler method and uses artificial intelligence based on deep learning to clearly distinguish malignant melanoma.

Conventional malignant melanoma diagnosis uses tumor markers, as disclosed in Patent Documents 1, 2, and 3, and requires collecting blood from the patient. Therefore, tests and diagnoses are not performed non-invasively, and regular tests and follow-up observations place a heavy burden on patients.

On the other hand, data regarding human skin blood flow is related to the metabolic activity state of skin tissue, metabolic state, degree of aging, skin cancer, and various other skin conditions. , which serves as a reference in determining the condition of the skin. As a device for measuring blood flow in human skin, a device that non-invasively measures blood flow using a laser Doppler method (Laser Doppler Velocimetry (LDV)) has been proposed, as disclosed in Patent Documents 4 and 5. The laser Doppler measurement method is a blood flow measurement method that does not invade the skin and can continuously measure instantaneous values of blood flow velocity and blood flow volume. The laser Doppler method for measuring blood flow utilizes the monochromaticity and coherence of laser light.When a laser beam is irradiated onto a blood vessel in the skin tissue, light scattered by blood cells moving within the blood vessel is detected. Since a frequency shift occurs due to the Doppler effect, the blood flow velocity is determined based on this frequency shift amount. Furthermore, Multipoint LDV, which uses the laser Doppler method as a basic technology and has multiple measurement points, is also being developed.

International publication WO2005/039380 International publication WO2006/043362 International publication WO2013/191146 International publication WO2009/081883 JP2015-59856A

Conventional methods for diagnosing malignant melanoma involve collecting blood from the patient, rather than performing non-invasive testing and diagnosis, which places a heavy burden on patients with frequent tests and follow-up observations. Furthermore, the laser Doppler blood flow measurement methods disclosed in Patent Documents 4 and 5 are limited to observing the state of blood flow, and do not enable diagnosis of malignant melanoma. Diagnosis of malignant melanoma by a non-invasive method generally relies on a doctor's visual judgment using dermoscopy, and there is no technology for diagnosing skin cancer completely non-invasively.

This invention was made in view of the above-mentioned conventional background technology, and uses a laser Doppler blood flow measurement method using a simple optical system to reduce the burden on patients and to detect malignant melanoma non-invasively and with high precision. An object of the present invention is to provide a malignant melanoma diagnostic device that can accurately diagnose malignant melanoma.

This invention diagnoses the presence or absence of skin cancer based on image judgment technology using deep learning. In diagnosis, first, several thousand images of skin cancer, which is malignant melanoma, and lentigines, which are not malignant, are prepared, and data labeled as skin cancer or lentigo is accumulated. Next, features are extracted from each image using an image diagnosis program using deep learning. Normally, skin cancer diagnosis is performed using images of the skin surface, but in the present invention, blood flow measurement results are also used. Skin cancer induces internal blood flow abnormalities, but lentigines do not. The basis for highly accurate malignant melanoma diagnosis can be obtained by using both external information on the skin surface and internal information on the blood flow state.

The malignant melanoma diagnostic device of the present invention includes: an imaging device that takes an image of a living tissue; a laser light source; and an acousto-optic element that branches the laser light from the laser light source into two laser lights with slightly different frequencies. a reflecting mirror that reflects each of the laser beams branched by the acousto-optic element in a predetermined direction; an extended optical system that converts the laser beam reflected by the reflecting mirror into planar thin sheet light; and each of the branched laser beams. a condensing optical system that causes the sheet lights to intersect with each other at a predetermined position within the living body; an imaging optical system that forms an image of the scattered light at the site where the sheet lights intersect; and an imaging optical system that is arranged at the imaging position of the imaging optical system. a photoelectric conversion element that converts the scattered light incident on the light receiving member into an electrical signal; and a blood flow in which the laser beam is shifted in Doppler frequency by the blood flow of the blood vessel at the predetermined position in the living tissue. The laser Doppler blood flow measurement device includes a blood flow calculation means for calculating the speed based on the electrical signal of each photoelectric conversion element. Furthermore, a photographed image of a known malignant melanoma, a blood flow image obtained by measuring the region of the photographed image with the laser Doppler blood flow measuring device, a photographic image of a lentigine that is not a tumor, and a photographed image region of the lentigine are A deep learning means is capable of performing deep learning in advance to distinguish between a blood flow image measured by the laser Doppler blood flow measuring device, and a captured image of skin suspected of having malignant melanoma captured by the imaging device. , a malignant melanoma diagnostic device that acquires a blood flow image obtained by measuring a region of the photographed image with the laser Doppler blood flow measuring device, and uses the deep learning means to determine whether or not it is malignant melanoma. .

Further, the two laser beams are each modulated by the acousto-optic element, and the laser Doppler blood flow measuring device measures the blood flow based on the moving direction of interference fringes due to the phase difference between the incident wavefronts of the two laser beams. It is provided to be able to detect the direction and determine the state of blood supply to the malignant melanoma.

The condensing optical system may make the surfaces of the two sheets of light included in the same plane intersect with each other within the plane .

According to the malignant melanoma diagnostic device of the present invention, the blood flow velocity within the human body can be directly measured in a completely non-invasive manner, and malignant melanoma can be diagnosed with high accuracy. This reduces the burden on patients, enables more frequent examinations and follow-up observations, and improves treatment accuracy.

1 is a schematic diagram showing an optical system of a laser Doppler blood flow measuring device according to a first embodiment of a malignant melanoma diagnostic device of the present invention. FIG. 1 is a schematic block diagram showing functions from a photoelectric conversion element to a computer in a first embodiment of the malignant melanoma diagnostic device of the present invention. FIG. 2 is a schematic diagram showing a computer-based deep learning means of the malignant melanoma diagnostic device of the present invention. These are a photographic image of a blood vessel in the ear of a mouse and a blood flow image before cancer cell transplantation of malignant melanoma determined according to the present invention. These are a photographic image of a blood vessel in the ear of a mouse and a blood flow image on the second day after cancer cell transplantation of malignant melanoma determined according to the present invention. These are a photographic image of a blood vessel in the ear of a mouse and a blood flow image on the fifth day after cancer cell transplantation of malignant melanoma determined according to the present invention. These are a photographic image of a blood vessel in the ear of a mouse and a blood flow image on the 9th day after cancer cell transplantation of malignant melanoma determined according to the present invention. These are a photographic image of a blood vessel in the ear of a mouse and a blood flow image on the 12th day after cancer cell transplantation of malignant melanoma determined according to the present invention. This is a photographed image of a blood vessel in the ear of a mouse on the 16th day after cancer cell transplantation of malignant melanoma determined according to the present invention. FIG. 2 is a schematic diagram showing an optical system of a laser Doppler blood flow measuring device according to a second embodiment of the malignant melanoma diagnostic device of the present invention.

Embodiments of the present invention will be described below based on the drawings. 1 and 2 show a laser Doppler blood flow measuring device 10 used in a malignant melanoma diagnostic device according to a first embodiment of the present invention. As shown, it includes a laser light source 12 such as a laser diode, and an acousto-optic element 14 that branches continuous wave laser light L from the laser light source 12 into two laser lights L1 and L2 having slightly different frequencies. The wavelength of the laser beam L is preferably about 750 to 1500 nm, and for example, a near-infrared laser with a wavelength of 785 nm is used. This is because near-infrared light has a high transmittance into living tissues such as blood vessels.

The acousto-optic element 14 has a structure in which a piezoelectric element is bonded to an acoustic engineering medium, and utilizes the fact that when ultrasonic waves are generated in the medium, a periodic refractive index change occurs due to the photoelastic effect. When the laser beam L is incident on the acousto-optic element 14, Bragg diffraction occurs, and first-order, second-order, third-order, etc. diffracted light is obtained from the zero-order light that is not subjected to diffraction. In the acousto-optic element 14, an internal piezoelectric element is driven at a frequency fa=80 MHz, for example, and zero-order light (L1) and first-order light (L2) with high light intensity are used. The first-order light (L2) has a frequency that is the frequency of the zero-order light (L1) plus the ultrasonic frequency fa, and the frequency difference between the zero-order light (L1) and the first-order light (L2) is determined by the acousto-optic element. This is the ultrasonic frequency that drives 14.

Further, on the light output side of the acousto-optic element 15, there is a reflecting prism 16 that reflects the split laser beams L1 and L2 in two directions, and a reflecting prism 16 that reflects the laser beams L1 and L2 reflected by the reflecting prism 16 in parallel to each other. It is equipped with reflecting mirrors 18 and 19. Furthermore, two achromatic lenses 20, which are condensing optical systems into which the two laser beams L1 and L2 reflected by the reflecting mirrors 18 and 19 enter, and an extended optical system that spreads the laser beams L1 and L2 into a sheet shape. Rod lenses 22 and 23 are arranged. The sheet lights of the laser beams L1 and L2 that have entered the respective rod lenses 22 and 23 intersect at the focal position of the achromatic lens 20. The laser beams L1 and L2, which are sheet lights, intersect with each other, and the intersection portion is formed as a linear irradiation region Lx with high light intensity, and is irradiated by intersecting blood vessels of a living tissue, which will be described later. Ru.

As shown in FIG. 1, the scattered light at the linear irradiation site Lx is focused and imaged by an achromatic lens 20 that also serves as an imaging optical system for the scattered light. The end surface portion 24a of the optical fiber array 24 of the light receiving member 26 is located at the image formation position of the scattered light of the linear irradiation area Lx by the achromatic lens 20, and the light is formed linearly along the linear irradiation area Lx. It is arranged so that an image of the shaped irradiation site Lx can be formed.

The optical fiber array 24 is guided to the other end of each optical fiber 28 through an optical fiber bundle 28a in which each optical fiber 28 is bundled, and each optical fiber 28 is connected to an optical plug 30. The optical plugs 30 are provided so as to be connectable to respective optical receptacles 32 which are photoelectric conversion elements. Each optical receptacle 32 is provided with avalanche photodiodes equal to the number of optical fibers 28. The avalanche photodiodes each convert the light guided by the optical fiber 28 into electrical signals with high sensitivity.

As shown in FIG. 2, each output of the avalanche photodiode of each optical receptacle 32 is inputted to a frequency filter 36 via each amplifier 34 provided in the number of optical fibers 28, and a signal of a predetermined frequency is output. The signals are A/D converted and recorded on data recorders 38 provided as many times as there are optical fibers 28. The digital signals recorded in each data recorder 38 are input to a computer 40 which is blood flow calculation means. The computer 40 is equipped with deep learning means that calculates the blood flow velocity, direction, blood flow volume, etc. of the linear irradiation site Lx, and performs deep learning to be described later.

Next, a laser Doppler blood flow measurement method using the laser Doppler blood flow measurement device 10 of this embodiment will be described below. This laser Doppler blood flow measurement method modulates a laser beam L emitted from a laser light source 12 with an acousto-optic element 14, and splits the laser beams L1 and L2 with a reflection prism 16 into a cylindrical beam through an achromatic lens 20. The lenses 22 and 23 convert the laser beams into sheet-like laser beams L1 and L2. Further, the sheet light laser beams L1 and L2 are caused to intersect with each other at the focal position of the achromatic 20 by the cylindrical lenses 22 and 23 to form a linear irradiation site Lx at a predetermined position in the living body where a blood vessel exists. In this state, the blood flow velocity and the direction of blood flow in the blood vessel can be determined using the optical fiber 28 as a unit by the computer 40 with a spatial resolution determined by the magnification of the imaging optical system.

The linear irradiation area Lx, which is the intersection of the sheet-like laser beams L1 and L2, is a single linear portion with high light intensity, and interference fringes are formed due to the phase difference between the incident wavefronts of the laser beams L1 and L2. When red blood cells pass through the linear irradiation site Lx at a certain speed, the Doppler frequency of the scattered light changes. In this linear irradiation site Lx, the Doppler frequency of the laser light scattered by blood cells in the bloodstream irradiated with the sheet light Ls changes due to the Doppler effect due to the flow velocity of the bloodstream. Note that in a stationary tissue, there is no change in the Doppler frequency of the scattered light. Each of the scattered lights at the linear irradiation site Lx is imaged by the achromatic lens 20 at a corresponding point on the end face portion 24a of the optical fiber array 24. This scattered light is input to the avalanche photodiode of the optical receptacle 32 via the optical fiber 28 and detected as an electrical beat signal, and the detected frequency change and intensity are values corresponding to the speed and number of blood cells. .

The blood flow calculation processing here is performed by A/D converting the input signal obtained by the avalanche photodiode, inputting it to the computer 40, fast Fourier transforming the digital signal, performing noise removal processing, etc., and converting it to Doppler. Calculate the frequency. The Doppler frequency is determined by the appearance of a peak at the Doppler frequency in the frequency spectrum of the detected signal. Once the Doppler frequency is determined, the velocity of the blood cells is calculated from the frequency, and since the velocity of the blood cells is the flow velocity of the blood flow, the blood flow velocity is calculated. This is determined for each optical fiber 28 that is a light receiving member, and the computer 40 calculates the velocity of blood flow in the corresponding region.Furthermore, the linear irradiation region Lx is optically scanned, and the blood vessel cross section of the target region is Find the blood flow velocity at In this embodiment, since the laser beams L1 and L2 are modulated by the acousto-optic element 14, the direction of blood flow can also be identified from the direction of movement of the interference fringes.

Next, malignant melanoma diagnosis using the malignant melanoma diagnostic apparatus of this embodiment will be described below. Malignant melanoma diagnosis in this embodiment is performed using a computer 40 based on laser Doppler blood flow measurement by the laser Doppler blood flow measuring device 10 and photographed images of blood vessels of biological tissue acquired by a general photographing device. It utilizes deep learning. As shown in FIG. 3, the malignant melanoma diagnosis program 42 by the computer 40, which is a deep learning means, first takes a photographed image a1 of a known malignant melanoma and a region of the photographed image a1 using the laser Doppler blood flow measuring device 10. Deep learning is performed by capturing a large number of measured blood flow images b1. Furthermore, a large number of captured images a of moles that are not tumors and blood flow images b obtained by measuring the area using the laser Doppler blood flow measuring device 10 are also taken in for deep learning. In this embodiment, several thousand images of malignant melanoma, which is skin cancer, and lentigines, which are not malignant, are prepared, and data is prepared in which they are labeled as skin cancer or lentigo. Next, using the computer 40, features are extracted from each image using an image diagnosis program using deep learning.

Here, regarding melanoma, which is malignant melanoma, FIGS. 4 to 9 show examples in which tumor cells are transplanted into the ears of mice, for example. Photographed image a is a photograph of a portion of a mouse ear where tumor cells were transplanted. The blood flow image b is an image obtained by measuring the region of the photographed image a using the laser Doppler blood flow measuring device 10. The speed of blood flow is shown by color and density, but this diagram does not show color, so it only shows that the speed of blood flow is changing due to the proliferation of cancer cells. As shown in Figures 4 to 8, on the 12th day, cancer cell proliferation was confirmed and the blood flow rate changed significantly based on the images of the ear before transplantation and the blood flow rate. . Furthermore, as shown in FIG. 9, on the 16th day after transplantation, it can be seen that the cancer cells have become black and enlarged.

Each of the captured photographic images a and each blood flow image b undergoes deep learning from the input layer 44 of the malignant melanoma diagnosis program 42 of the computer 40, which is a deep learning means, through the intermediate layer 46, and the output layer 48 makes a judgment. Participants will learn about the distinction between lentigo, which is not a tumor, and skin cancer, which is malignant melanoma. As for the deep learning method, a well-known method is used, so a detailed explanation will be omitted, but the more captured images a and blood flow images b are input to the input layer 44, the more accurately feature extraction will be performed. To enable more accurate diagnosis, it is necessary to input many images in advance and perform deep learning.

According to the malignant melanoma diagnostic device of this embodiment, by preparing several thousand images each of malignant melanoma (melanoma) and non-malignant lentigines, artificial A malignant melanoma diagnostic device using a computer that is intelligent (AI) can be provided. By using deep learning to determine whether a black area on the skin surface is a melanoma or a lentigo in combination with blood flow measurement images, extremely accurate diagnosis is possible. In particular, the growth of cancer can be determined by the amount of blood supplied, and the direction of blood flow can be used to distinguish between the discharge and supply of blood to cancer cells, so deep learning can be used to better understand the growth and progress of cancer cells. Become more predictable with precision. Furthermore, the number of images required for input in deep learning can be reduced, improving learning efficiency.

Next, a second embodiment of the laser Doppler blood flow measuring method and apparatus of the present invention will be described based on FIG. 10. Here, the same configurations as those in the above-described embodiment are given the same reference numerals, and the description thereof will be omitted. In the laser Doppler blood flow measuring device 50 of this embodiment, two laser beams L1 and L2 reflected by reflecting mirrors 18 and 19 are first converted into sheet-like laser beams L1 and L2 by rod lenses 22 and 23 of an extended optical system. L2, and is further converted into a sheet of parallel light by cylindrical lenses 52 and 53, and is provided so as to be incident on an achromatic lens 20, which is a condensing optical system. An optical system is arranged so that the laser beams L1 and L2, which are sheet lights, are located on the same plane. The sheet-shaped laser beams L1 and L2 are located on the same plane and intersect at a predetermined angle. The tolerance angle is the angle at which each of the laser beams L1 and L2 is refracted and emitted with respect to the optical axis of the achromatic lens 20.

As shown in FIG. 10, the intersection area D of the laser beams L1 and L2, which are sheet lights, is such that the planes of the laser beams L1 and L2, which are parallel lights, coincide with each other and overlap each other, and the intersection area D is formed in a diamond shape. The intersecting region D becomes a flow velocity measurement region that intersects with the blood vessel T.

An imaging lens 54, which is an imaging optical system including a pair of achromatic lenses, is arranged at a predetermined distance apart in a direction perpendicular to the intersection area D of the sheet-shaped laser beams L1 and L2. A light receiving member 60 in which an optical fiber array is two-dimensionally arranged is located at the imaging position of the imaging lens 54, facing the outgoing light side of the imaging lens 54. The light receiving member 60 is a two-dimensional array of optical fibers, and is composed of optical fibers, optical plugs, optical receptacles, etc. similar to those in the above embodiments. The other configurations are similar to the laser Doppler blood flow measuring device 10 of the above embodiment. Furthermore, an image sensor may be used as the light receiving member 60 in which the optical fiber array is two-dimensionally arranged.

In the case of this embodiment, in addition to the same effect as the above-described embodiment, the sheet-shaped laser beams L1 and L form a two-dimensional intersection area D, thereby obtaining blood flow information of more areas at the same time. This enables efficient measurement.

Note that the malignant melanoma diagnostic apparatus of the present invention is not limited to the above embodiment, and the arrangement, type, and number of optical systems can be set as appropriate. The light receiving member and the photoelectric conversion element may be a line sensor of a semiconductor element, or a solid-state imaging element of an image sensor such as a CCD or C-MOS. The optical fiber can also be selected as appropriate, such as glass fiber or plastic fiber.

10 Laser Doppler blood flow measuring device 12 Laser light source 14 Acousto-optic element 16 Reflection prism 18, 19 Reflector 20 Achromatic lens 22, 23 Rod lens 24 Fiber array 26, 60 Light receiving member 28 Optical fiber 30 Optical plug 32 Optical receptacle 40 Computer

Claims (3)

an imaging device that takes an image of a living tissue, a laser light source, an acousto-optic element that splits the laser light from the laser light source into two laser lights with slightly different frequencies, and the laser light that is split by the acousto-optic element. a reflecting mirror that reflects each of the laser beams in a predetermined direction, an expansion optical system that converts the laser beam reflected by the reflecting mirror into a planar thin sheet light, and a mirror that converts each of the branched sheet lights to each other at a predetermined position in the living body. a light-condensing optical system that intersects, an imaging optical system that forms an image of scattered light at a portion where the sheet light intersects, a light-receiving member disposed at an image-forming position of the imaging optical system, and a light-receiving member that A photoelectric conversion element that converts the incident scattered light into an electrical signal, and a blood flow velocity whose Doppler frequency is shifted by the laser beam due to blood flow in the blood vessel at the predetermined position in the living tissue, is determined for each photoelectric conversion element. Equipped with a laser Doppler blood flow measurement device having a blood flow calculation means that calculates based on electrical signals,
A photographed image of a known malignant melanoma, a blood flow image in which the region of the photographed image was measured by the laser Doppler blood flow measurement device, a photographic image of a lentigine that is not a tumor, and a region of the photographed image of the lentigine were measured using the laser Doppler blood flow measuring device. Equipped with deep learning means that can distinguish blood flow images measured by a Doppler blood flow measuring device through deep learning in advance,
Obtaining a photographed image of skin suspected of having malignant melanoma taken by the photographing device and a blood flow image obtained by measuring the area of the photographed image with the laser Doppler blood flow measurement device, and using the deep learning means, A malignant melanoma diagnostic device characterized by determining whether or not it is malignant melanoma. The two laser beams are each modulated by the acousto-optic element, and the laser Doppler blood flow measurement device determines the direction of blood flow based on the moving direction of interference fringes due to the phase difference between the incident wavefronts of the two laser beams. 2. The malignant melanoma diagnostic device according to claim 1, wherein the malignant melanoma diagnostic device is capable of detecting and determining the state of blood supply to the malignant melanoma. The malignant melanoma diagnostic apparatus according to claim 1, wherein the condensing optical system causes the surfaces of the two sheets of light included in the same plane to intersect with each other within the plane .

Images