JP7607248B2 - Information processing device and program - Google Patents

Description

This disclosure relates to an information processing device and a program.

In recent years, due to the spread of coronavirus, test kits that combine a cotton swab, squeeze tube, dropping tip, etc. with a reaction cassette that improves the convenience of antigen-antibody testing by immunochromatography based on the principle of immunoassay have become widespread. As shown in FIG. 11, a sample collected from a subject is dropped into a predetermined dropping part 32 of a reaction cassette 31, and after a predetermined testing time has elapsed, the presence or absence of the test target, i.e., positive or negative, can be determined by visually confirming the reference line that appears in the reference area on the test window 33 and the judgment line that appears in the judgment area.

Specifically, if a reference line appears in the reference area, for example in a bluish color, and no judgment line appears in the judgment area a specified distance below the reference area, the result can be determined as negative, and if both the reference line and the judgment line appear in a bluish color, the result can be determined as positive.

Such test kits are difficult for individuals to use. The main reasons for this are that sufficient experience is required to judge the presence or absence of the test line, particularly based on the degree of color development, and the variety of lighting conditions makes it difficult to make a uniform judgment.

The objective is to provide an information processing device and a program that can improve the accuracy of determining the presence or absence of a test subject using a test kit based on an antigen-antibody test method using the immunochromatography method.

The information processing device according to this embodiment includes a storage unit that stores data on an original image of a reaction cassette for reacting a reagent with a sample and detecting the presence or absence of a substance to be tested based on the color development state of the reagent caused by the reaction, a program for processing the original image to determine the presence or absence of a substance to be tested, a processor that executes the program, and a display unit that displays the result of the determination. The processor executes the program to realize a means for identifying, from an examination window image in the original image, a color space value for a pixel in a reference line that appears in a reference region and a color space value for a pixel in a region between the reference region and a determination region where the determination line appears, a means for normalizing the color space value for the pixel in the determination line between the color space value for the pixel in the reference line and the color space value for the pixel in the region between the reference region and the determination region, and a means for determining the presence or absence of a substance to be tested based on the color space value for the pixel in the normalized determination line.

FIG. 1 is a diagram showing a portable information processing terminal as an example of an information processing device according to this embodiment. FIG. 2 is a diagram showing the configuration of the portable information processing terminal of FIG. FIG. 3 is a diagram showing a server device as another example of the information processing device according to the present embodiment. FIG. 4 is a flowchart showing the procedure of the determination process by the information processing device of FIG. FIG. 5 is a flow chart showing a procedure following the determination process of FIG. FIG. 6 is a diagram showing an example of an inspection window image extracted in step S12 of FIG. FIG. 7 is a supplementary diagram of the steps shown in FIGS. FIG. 8 is a diagram showing the reduced distance image generated by step S20 of FIG. FIG. 9 is a supplementary diagram of the determination processes S24 and S28 in the step S of FIG. FIG. 10 is a diagram showing an example of a display screen showing the determination result in step S29 of FIG. FIG. 11 is a plan view of an example of a reaction cassette.

Hereinafter, an embodiment of the present invention will be described with reference to the drawings.
The embodiment of the present invention is characterized in that a reaction cassette for detecting the presence or absence of a test target substance by reacting a reagent with a sample and detecting the presence or absence of a test target substance based on the color development state of the reagent due to the reaction is photographed, and from the test window image in the original image generated by this, the color space values for the pixels in the judgment line are normalized (scaled) between the color space values for the pixels in the reference line appearing in the reference area and the color space values for the pixels in the non-colored area between the reference area and the judgment area where the judgment line appears, to determine the presence or absence of the test target substance. This suppresses a decrease in judgment accuracy due to disturbance factors such as changes in camera characteristics and lighting environment, and realizes stabilization of the judgment process. This will be described in detail below.

As shown in FIG. 1, the information processing device according to this embodiment is typically a portable information processing terminal such as a smartphone. As shown in FIG. 2, the portable information processing terminal is configured such that a RAM 12, a ROM 13, a memory unit 14, a camera 15, a touch panel 17, an input controller 16 for the touch panel 17, a display 19 such as an LCD (Liquid Crystal Display), a video controller 18 for the display 19, and a communication module 20 are connected to a processor 11 via a data/control bus 10. The processor 11 is configured with a CPU (Central Processing Unit) and a GPU (Graphics Processing Unit), and by executing a program stored in the memory unit 14 and loaded into the RAM, various means are realized for determining the presence or absence of a test target substance such as a coronavirus or influenza virus in an image (original image) of a reaction cassette captured by the camera 15.

The information processing device according to this embodiment may be a server device 3 connected to the portable information processing terminal 1 via an Internet line 5, as shown in FIG. 3. The server device 3 receives image (original image) data relating to the reaction cassette taken by the camera 15 of the portable information processing terminal 1 from the portable information processing terminal 1, processes the original image data to determine the presence or absence of the test target substance, and returns the determination result to the portable information processing terminal 1. Here, the information processing device according to this embodiment will be described as being the portable information processing terminal 1.

Figures 4 and 5 show the procedure for determining the presence or absence of the test target substance. As a premise, a sample collected from a subject is dropped into a predetermined dropping portion 32 of the reaction cassette 31 shown in Figure 11, and after a predetermined test time has elapsed, if a reference line appears in a blue-based color properly in the reference area on the test window 33, and a judgment line appears in a blue-based color in the judgment area a predetermined distance below the reference area, it can be determined that the sample contains the test target substance (positive judgment). On the other hand, if the reference line appears properly in the reference area but no judgment line appears in the judgment area, it can be determined that the sample contains the test target substance (positive judgment). The reference line is a line to confirm that the test kit is functioning properly, and it develops color regardless of the appearance of the judgment line (positive or negative). The color that develops depends on the color development principle adopted by the test kit, and the colors of the reference line and judgment line include red, blue, green, purple, black, etc. Here, we will assume that both the reference line and the judgment line are colored blue.

For ease of explanation, the up-down direction for the original image I0 and other images is defined as the downside of the drip part 32 and reaction start button 34 when viewed from the test window 33 in the reaction cassette 31 shown in FIG. 11, and the upside of the drip part 32 and test window 33 when viewed from the reaction start button 34.

First, in step S11, the entire reaction cassette 31 is photographed by the camera 15. As a result, data of the original image I0 is generated and stored in the memory unit 14, as shown in FIG. 6. Preferably, the entire reaction cassette 31 is photographed so that the top and bottom are aligned and fit within the entire range of the vertical angle of view. In this description, the values of the pixels constituting the original image I0 are expressed in RGB format. In other words, the pixel values are given by color space values (R value, G value, B value) as coordinates on a coordinate space (color space) consisting of three orthogonal axes, the R axis, the G axis, and the B axis. As is well known, when the R value, the G value, and the B value are each expressed in 256 gradations, blue is expressed as RGB (0, 0, 255) and white is expressed as RGB (255, 255, 255).

Next, in step S12, as shown in FIG. 6, an inspection window image I1 relating to the inspection window 33 on the reaction cassette 31 is extracted from the original image I0, for example, by contour extraction processing. This extraction method is not limited to contour extraction processing, and any method can be used. Next, in step S13, the inspection window image I1 is resized to a predetermined matrix size (number of pixels vertically and horizontally) by interpolation processing or thinning processing. This generates a resized inspection window image I2, which is stored in the memory unit 14. Since the resolution of the camera 15 equipped in the portable information processing terminal 1 varies and varies depending on the settings of the shooting conditions, the resizing processing keeps the matrix size constant to realize stable execution of the processing described below.

In step S14, a search frame F1 is set to search for a reference line RL indicating the lower edge of the reference line shown in FIG. 7. The reference line appears somewhere in the reference area in the inspection window 33. The position and range of the reference area in the inspection window 33 are predetermined. The positions and ranges of the judgment area in which the influenza virus judgment line appears and the judgment area in which the coronavirus judgment line appears are also predetermined. The search frame F1 is set to surround the reference area. The center of the search frame F1 is set at a predetermined distance from the top end of the inspection window image I2, that is, a predetermined number of pixels P1 below, the width is set to the same as the width of the inspection window image I2, and the height is set to the same as or slightly larger than the reference area.

In step S15, within the search frame F1, the pixel of interest is moved pixel by pixel from top to bottom along the center line C of the inspection window image I2, and the lower edge of the reference line where the color changes from blue to uncolored white is identified based on the color space value of the pixel of interest and the color space value of a target pixel located one pixel or a predetermined number pixels above the pixel of interest. Specifically, the position where the color space value of the pixel of interest is lower than the color space value of the target pixel and where the difference between the color space values of these two pixels, i.e., the variation in color space value, is maximum, is identified. A reference line RL is set parallel to the width direction, passing through that position.

Next, in step S16, a blue block B(B) of a predetermined size is set at a position a predetermined number of pixels P2 above the reference line RL. The size of the blue block B(B) is set, for example, to a width of 20% of the number of pixels representing the total width of the inspection window image I2, and a height of 10% of the number of pixels representing the total width of the inspection window image I2. The number of pixels P2 is set so that the entire area of the blue block B(B) is included within the reference line. In step S17, the average color space value ave.(B(B)) is calculated for multiple pixels on the inspection window image I2 included in the blue block B(B). Typically, when pixel values are expressed in RGB format, the average values of the R value, G value, and B value are calculated. Note that the average color space value ave.(B(B)) for multiple pixels in the blue block B(B) indicates the standard color space value within the reference line.

In step S18, the area between the reference area and the judgment area is set as a search frame F2, and within this search frame F2, a white block B(W) of the same size as the blue block B(B) is moved downward along the center line C of the inspection window image I2 at one pixel pitch, and the average color space value ave.B(W) of the pixels contained in the white block B(W) at each position is calculated. Then, the average color space value ave.B(W) that is furthest in color space from the average color space value ave.(B(B)) of the pixels in the blue block B(B), that is, the average color space value ave.B(W) that shows the greatest distance in color space from the average color space value ave.B(B) of the color space values of the pixels in the blue block B(B), is identified.

In step S19, as shown in Figures 8(a) and 8(b), the color space distance d(x,y(I2)) between the average color space value ave.B(W) of the searched white block B(W) and the color space value p(x,y(I2)) for each pixel in the inspection window image I2 is calculated for each pixel. This converts the inspection window image I2 into a distance image I3. Furthermore, in step S20, the pixel value of each pixel in the distance image I3 is converted into a ratio (d(x,y(I2))/D) to the color space distance D between the average color space value ave.B(B) for the pixels in the blue block B(B) and the average color space value ave.B(W) for the pixels in the white block. This generates a converted distance image I4. In other words, the process of generating the converted distance image I4 means that the color space values for each pixel on the inspection window image I2 are normalized (scaled) between the average value ave.B(W) of the color space values for the pixels in the white block B(W) and the average value ave.B(B) of the color space values for the pixels in the blue block B(B) in order to eliminate dependency on the characteristics of the camera 15 and the shooting environment such as lighting, and to improve and stabilize the judgment accuracy.

Next, in step S21, a search frame F3 is set at a position a predetermined number of pixels P3 below the reference line RL. The center of the search frame F3 is set at a position a predetermined distance from the reference line RL, that is, a predetermined number of pixels P3 below, the width is set to the entire area of the inspection window image I2, and the height is preset to a size that surrounds the judgment area within the inspection window 33 where the influenza virus judgment line may appear.

In step S22, influenza block B(FLU), which is the same size as blue block B(B), is moved vertically and horizontally by one pixel within search frame F3, and the average ave.B(FLU) of pixel values for multiple pixels on converted distance image I4 contained in influenza block B(FLU) at each position is calculated. Then, in step S23, a maximum value MAXave.B(FLU) is determined from the average ave.B(FLU) of multiple pixel values.

In step S24, as shown in FIG. 9, the maximum value MAXave.B(FLU) is compared with a predetermined threshold value TH, for example 0.3. The positive/negative result for influenza virus is determined according to the comparison result. When the maximum value MAXave.B(FLU) of influenza block B(FLU) is lower than the threshold value TH, it is determined as negative, and when it is higher, it is determined as positive. The value of the threshold value TH can be changed arbitrarily.

Next, in step S25, a search frame F4 is set at a position a predetermined number of pixels P4 below the reference line RL. The center of the search frame F4 is set at a position a predetermined distance from the reference line RL, that is, a predetermined number of pixels P4 below, the width is set to the entire area of the test window image I2, and the height is preset to a size that surrounds the test area within the test window 33 where a test line for coronavirus may appear.

In step S26, corona block B(COV), which is the same size as blue block B(B), is moved vertically and horizontally by one pixel within search frame F4, and the average ave.B(COV) of the pixel values of multiple pixels on converted distance image I4 contained in corona block B(COV) at each position is calculated. Then, in step S27, the maximum value MAXave.B(COV) is determined from the average ave.B(COV).

In step S28, the maximum value MAXave.B(COV) is compared with a predetermined threshold value TH. A positive/negative result for coronavirus is determined according to the comparison result. When the maximum value MAXave.B(COV) of corona block B(COV) is lower than the threshold value TH, it is determined as negative, and when it is higher, it is determined as positive. Typically, the value of the threshold value TH is the same for influenza virus determination and coronavirus determination, but it may be set separately for each.

As shown in FIG. 10, in step S29, the judgment result is displayed on the display 19. Alternatively, data related to the judgment result is transmitted from the server device 3 to the portable information processing terminal 1 and displayed on the display 19 of the portable information processing terminal 1. A histogram obtained by integrating the pixel value p(x,y(I4)) in the horizontal direction may be displayed on the right side of the displayed converted distance image I4, and a histogram obtained by integrating the pixel value p(x,y(I4)) in the vertical direction may be displayed below. These histograms allow the integral value of the reference line to be compared with the integral value of the reaction line, and the positive/negative judgment result and its accuracy can be roughly confirmed visually.

According to this embodiment, it is possible to improve the accuracy of determining the presence or absence of a test subject using a test kit based on an antigen-antibody test method using immunochromatography. More specifically,
By normalizing (scaling) the color space values within the judgment line between the standard color space values within the reference line and the color space values of the non-colored area between the reference area and the judgment area, dependency on the characteristics of the camera 15 and the shooting environment such as lighting can be eliminated, thereby improving and stabilizing the accuracy of judgment regarding the presence or absence of the object to be inspected.

Although several embodiments of the present invention have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These embodiments can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the gist of the invention. These embodiments and their modifications are within the scope of the invention and its equivalents as set forth in the claims, as well as the scope and gist of the invention.

1...Portable information processing terminal, 11...Processor, 10...Data/control bus, 12...RAM, 13...ROM, 14...Memory unit, 15...Camera, 17...Touch panel, 19...Display, 20...Communication module.

Claims (10)

a storage unit for storing data of an original image relating to a reaction cassette for reacting a reagent with a specimen and detecting the presence or absence of a test target substance based on a color change state of the reagent caused by the reaction, and a program for processing the original image to determine the presence or absence of the test target substance;
An information processing device comprising: a processor that executes the program;
When the processor executes the program,
means for identifying, from an inspection window image in the original image, color space values for pixels in a reference line appearing in a reference region, and color space values for pixels in a region between the reference region and a judgment region in which a judgment line appears;
means for normalizing color space values for pixels in the decision line between color space values for pixels in the reference line and color space values for pixels in an area between the reference area and the decision area;
and determining the presence or absence of the test target substance based on the normalized color space values for the pixels within the determination line. a storage unit for storing data of an original image relating to a reaction cassette for reacting a reagent with a specimen and detecting the presence or absence of a test target substance based on a color change state of the reagent caused by the reaction, and a program for processing the original image to determine the presence or absence of the test target substance;
An information processing device comprising: a processor that executes the program;
When the processor executes the program,
means for extracting an interrogation window image relating to an interrogation window on the reaction cassette from the original image;
means for identifying an average of color space values for pixels in a first block on a reference line appearing in a reference region from the interrogation window image;
means for identifying an average of color space values of pixels in a second block that shows a maximum distance in color space to an average of color space values of pixels in the first block in a region between the reference region and a judgment region in which a judgment line appears;
means for determining a maximum value of average color space values for pixels in a third block on the decision line;
an information processing device that realizes a means for determining the presence or absence of the substance to be tested based on a ratio of the distance in color space between the average of the color space values for the pixels in the second block and the average of the color space values for the pixels in the first block to the distance in color space between the average of the color space values for the pixels in the second block and the maximum of the averages of the color space values for the pixels in the third block. The information processing device according to claim 2, wherein the means for extracting the inspection window image extracts the inspection window image by performing a contour extraction process on the original image. The information processing device according to claim 2, further comprising a means for resizing the inspection window image to a predetermined matrix size. means for searching color space values along a centerline of the interrogation window image to identify a location where the color space values exhibit the greatest variation;
3. The information processing apparatus according to claim 2, further comprising: means for setting the first block at a position a predetermined distance above the position. The information processing device according to claim 5 further includes a means for setting a second search frame at a position a predetermined distance below the position for searching for the maximum value while moving the third block. The information processing device according to claim 2, further comprising a means for calculating the distance in color space of the color space value of each pixel of the inspection window image relative to the average of the color space values of the pixels in the two blocks, and generating a distance image. The information processing device of claim 7 further includes a means for converting the distance of each pixel of the distance image into a ratio to the distance in color space between the average of the color space values for the pixels in the two blocks and the average of the color space values for the pixels in the first block, and converting the distance into a converted distance image. a means for identifying color space values of pixels in a reference line appearing in a reference region and color space values of pixels in a region between the reference region and a judgment region where a judgment line appears, from an inspection window image in an original image of a reaction cassette for reacting a reagent with a sample and detecting the presence or absence of a substance to be inspected based on the color development state of the reagent due to the reaction;
means for normalizing color space values for pixels in the decision line between color space values for pixels in the reference line and color space values for pixels in an area between the reference area and the decision area;
and determining the presence or absence of the test target substance based on the normalized color space values of the pixels within the determination line. a means for extracting an image of an inspection window on a reaction cassette from an original image of the reaction cassette for detecting the presence or absence of a test target substance based on a color change state of the reagent caused by the reaction;
means for identifying an average of color space values for pixels in a first block on a reference line appearing in a reference region from the interrogation window image;
means for identifying an average of color space values of pixels in a second block that shows a maximum distance in color space to an average of color space values of pixels in the first block in a region between the reference region and a judgment region in which a judgment line appears;
means for identifying a maximum value of average color space values for pixels in a third block on the decision line;
a means for determining the presence or absence of the substance to be tested based on the ratio of the distance in color space between the average of the color space values for the pixels in the second block and the average of the color space values for the pixels in the first block to the distance in color space between the average of the color space values for the pixels in the second block and the maximum of the averages of the color space values for the pixels in the third block.