Deprecated: The each() function is deprecated. This message will be suppressed on further calls in /home/zhenxiangba/zhenxiangba.com/public_html/phproxy-improved-master/index.php on line 456
NZ797422B2 - Feature amount acquisition device, similar image search device, display device, feature amount acquisition method, similar image search method, display method, and program - Google Patents
[go: Go Back, main page]

NZ797422B2 - Feature amount acquisition device, similar image search device, display device, feature amount acquisition method, similar image search method, display method, and program - Google Patents

Feature amount acquisition device, similar image search device, display device, feature amount acquisition method, similar image search method, display method, and program

Info

Publication number
NZ797422B2
NZ797422B2 NZ797422A NZ79742221A NZ797422B2 NZ 797422 B2 NZ797422 B2 NZ 797422B2 NZ 797422 A NZ797422 A NZ 797422A NZ 79742221 A NZ79742221 A NZ 79742221A NZ 797422 B2 NZ797422 B2 NZ 797422B2
Authority
NZ
New Zealand
Prior art keywords
image
activation level
similar
processing
feature amount
Prior art date
Application number
NZ797422A
Other versions
NZ797422A (en
Inventor
Kazuhisa Matsunaga
Original Assignee
Casio Computer Co Ltd
Filing date
Publication date
Priority claimed from JP2020137310A external-priority patent/JP7056698B2/en
Application filed by Casio Computer Co Ltd filed Critical Casio Computer Co Ltd
Publication of NZ797422A publication Critical patent/NZ797422A/en
Publication of NZ797422B2 publication Critical patent/NZ797422B2/en

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
    • G06F16/58—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
    • G06F16/583—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10024—Color image
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30004—Biomedical image processing
    • G06T2207/30024—Cell structures in vitro; Tissue sections in vitro
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30004—Biomedical image processing
    • G06T2207/30088—Skin; Dermal
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30004—Biomedical image processing
    • G06T2207/30096—Tumor; Lesion
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0012—Biomedical image inspection
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
    • G06V10/443—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components by matching or filtering
    • G06V10/449—Biologically inspired filters, e.g. difference of Gaussians [DoG] or Gabor filters
    • G06V10/451—Biologically inspired filters, e.g. difference of Gaussians [DoG] or Gabor filters with interaction between the filter responses, e.g. cortical complex cells
    • G06V10/454—Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
    • G06V10/761—Proximity, similarity or dissimilarity measures
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V2201/00—Indexing scheme relating to image or video recognition or understanding
    • G06V2201/03—Recognition of patterns in medical or anatomical images
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/107—Static hand or arm

Abstract

feature amount acquisition device includes activation level derivation means configured to receive an input image of an object under examination, wherein the image captures a first target and a second target surrounding the first target. In response to receiving a user instruction to find a similar image, the activation level derivation means processes the input image to derive an activation level in a classifier including a plurality of layers and configured to, by processing input data based on image data from the received input image, in the plurality of layers, output a result of classifying the first target, a unit in a layer among the plurality of layers influences a classification result of the classifier. Feature amount acquisition means then acquires a feature amount of the input image using the derived activation level as weighting data, such that regions with lower activation levels contribute less to the feature amount than regions with higher activation levels. Search means use the acquired feature amount to search a plurality of reference images for one similar to the input image in both visual similarity and classification-relevant information.

Claims (16)

1. A similar image search device configured to receive input images and output similar images for use in identifying classification of targets of which the input images are captured, 5 comprising: a feature amount acquisition device comprising: activation level derivation means configured to receive an input image captured of an object being examined , wherein a first target in the examined object and a second target around the first target in the examined object are captured in the received input image, the 10 activation level derivation means being configured to process the received input image, in response to receiving a user instruction to find a similar image similar to the received input image, the processing comprising: deriving, as an activation level, a level at which, in a classifier including a plurality of layers and configured to, by processing input data based on image data from the received input image, in the plurality of layers, output a 15 result of classifying the first target, a unit in a layer among the plurality of layers influences a classification result of the classifier; and feature amount acquisition means for acquiring, using the activation level derived by the activation level derivation means as weighting data, and the image data from the received input image, a feature amount of the received input image in such a way that a 20 feature amount of a low activation level image region is smaller than a feature amount of a high activation level image region, the low activation level image region being a region in the input image corresponding to a second unit serving as the unit having the activation level lower than the activation level of a first unit serving as the unit, the high activation level image region being a region in the input image corresponding to the first unit; and 25 search means for searching a plurality of reference images for a similar image which is determined to be similar to the received input image in both visual similarity and information used the classification of the first target, based on the feature amount acquired by the feature amount acquisition means. 22003443_1 (GHMatters) P120258.NZ
2. The similar image search device according to claim 1, wherein the feature amount acquisition device further comprises: image processing means for acquiring image data of a post-processing image by subjecting 5 the image data of the input image to image processing based on the activation level derived by the activation level derivation means in such a way that the feature amount of the low activation level image region is smaller than the feature amount of the high activation level image region, the low activation level image region being a region in the input image corresponding to the second unit serving as the unit having the activation level lower than the activation level of the first unit 10 serving as the unit, the high activation level image region being a region in the input image corresponding to the first unit, wherein the feature amount acquisition means acquires a feature amount of the post- processing image, based on the image data of the post-processing image acquired by the image processing means.
3. The similar image search device according to claim 2, wherein the image processing means subjects the input image to image processing to calculate a weighted average of each pixel value in the image data of the input image and a pixel value for masking processing, based on the activation level.
4. The similar image search device according to claim 2 or 3, wherein the feature amount acquisition device further comprises: determination means for determining whether or not the second target is a specific target, wherein the image processing means differentiates the image processing to which the image 25 data of the input image is subjected between when the second target is determined to be a specific target by the determination means and when the second target is determined to be not a specific target by the determination means. 22003443_1 (GHMatters) P120258.NZ
5. The similar image search device according to claim 4, wherein when the second target is determined to be a specific target by the determination means, in a case in which the activation level is less than a criterion value, the image processing means determines an input pixel weight, the input pixel weight being a weight of a 5 pixel value in the image data of the input image, based on the criterion value, in a case in which the activation level is greater than or equal to the criterion value, the image processing means determines the input pixel weight based on a value of the activation level, the image processing means determines, based on the determined input pixel weight, 10 a masking weight that is a weight of a pixel value for masking processing, and the image processing means subjects the input image to image processing to calculate a weighted average of each pixel value in the image data of the input image and a pixel value for the masking processing, based on the determined input pixel weight and the determined masking weight, and 15 when the second target is determined to be not the specific target by the determination means, the image processing means determines, regardless of a value of the activation level, a value of the activation level as the input pixel weight and determines, as the masking weight, a value obtained by subtracting the determined input pixel weight from 1, and 20 the image processing means subjects the input image to image processing to calculate a weighted average of each pixel value in the image data of the input image and a pixel value for the masking processing, based on the determined input pixel weight and the determined masking weight. 25
6. The similar image search device according to claim 4, wherein when the second target is determined to be a specific target by the determination means, the image processing means acquires a post-masking processing low activation level image region by subjecting the low activation level image region to masking processing, and acquires the post- 22003443_1 (GHMatters) P120258.NZ processing image by integrating the acquired post-masking processing low activation level image region and the input image with each other, and when the second target is determined to be not a specific target by the determination means, the image processing means acquires a post-masking processing low activation level image region 5 by subjecting the low activation level image region to masking processing, and acquires the post- processing image by integrating the acquired post-masking processing low activation level image region and a region other than the low activation level image region within the input image with each other. 10
7. The similar image search device according to any one of claims 4 to 6, wherein the specific target is palms and soles.
8. The similar image search device according to any one of claims 2 to 7, wherein the image processing, by changing each pixel value in the low activation level image region to a pixel 15 value representing the second target by subjecting the input image to the image processing, reduces a feature amount of the low activation level image region.
9. The similar image search device according to any one of claims 1 to 8, wherein the second target is tissue of a living thing, and 20 the first target is a diseased part or a part suspected to be diseased of the tissue.
10. A display device, comprising: display control means for displaying a similar image found in a search by the search means of the similar image search device according to claim 2, 25 wherein the display control means further displays at least one of an activation map visualizing the activation level and the post-processing image.
11. A similar image search method performed by a similar image search device to output 22003443_1 (GHMatters) P120258.NZ similar images from input images, for use in identifying classification of targets in objects of which the input images are captured, the method comprising: receiving an input image captured of an object being examined, wherein a first target in the examined object and a second target around the first target in the examined object are captured in 5 the received input image; receiving a user instruction to find from a plurality of reference images a similar image similar to the received input image; upon receiving the user instruction, process the received input image, comprising acquiring a feature amount by: 10 deriving, as an activation level, a level at which, in a classifier including a plurality of layers and configured to, by processing input data based on image data from the received input image in the plurality of layers, output a result of classifying the first target, an unit in a layer among the plurality of layers influences a classification result of the classifier; and acquiring, using the derived activation level as weighting data, and the image data 15 from the received input image, a feature amount of the received input image in such a way that a feature amount of a low activation level image region is smaller than a feature amount of a high activation level image region, the low activation level image region being a region in the input image corresponding to a second unit serving as the unit having the activation level lower than the activation level of a first unit serving as the unit, the high activation 20 level image region being a region in the input image corresponding to the first unit; and searching a plurality of reference images for a similar image which is determined to be similar to the received input image in both visual similarity and information used for the classification of the first target, based on the feature amount acquired in the feature amount acquisition method.
12. A program causing a computer of a similar image search device to, upon receiving an input image captured of an object being examined, wherein a first target in the examined object and a second target around the first target in the examined object are captured in the input image, 22003443_1 (GHMatters) P120258.NZ and receiving a user instruction to find from a plurality of reference images a similar image similar to the received input image, process the received input image, by: deriving, as an activation level, a level at which, in a classifier including a plurality of layers and configured to, by processing input data based on image data from the received input image in 5 the plurality of layers, output a result of classifying the first target, an unit in a layer among the plurality of layers influences a classification result of the classifier; acquiring, using the derived activation level as weighting data, and the image data from the received input image, a feature amount of the received input image in such a way that a feature amount of a low activation level image region is smaller than a feature amount of a high activation 10 level image region, the low activation level image region being a region in the input image corresponding to a second unit serving as the unit having the activation level lower than the activation level of a first unit serving as the unit, the high activation level image region being a region in the input image corresponding to the first unit; and search the plurality of reference images for a similar image which is determined to be 15 similar to the received input image in both visual similarity and information used for the classification, based on the feature amount acquired.
13. A similar image search method performed by a similar image search device to output similar images from input images, for use in identifying classification of targets of which the input 20 images are captured, the method comprising: receiving an input image captured of an object being examined, wherein a first target in the examined object and a second target around the first target in the examined object are captured in received the input image; receiving a user instruction to find from a plurality of reference images a similar image 25 similar to the input image; upon receiving the user instruction, processing the received input image, the processing comprising: deriving, as an activation level, a level at which, in a classifier including a plurality of layers 22003443_1 (GHMatters) P120258.NZ and configured to, by processing input data based on image data from the received input image in the plurality of layers, output a result of classifying the first target, an unit in a layer among the plurality of layers influences a classification result of the classifier; acquiring, using the derived activation level as weighting data, image data of a post- 5 processing image by subjecting the image data from the received input image to image processing in such a way that a feature amount of a low activation level image region is smaller than a feature amount of a high activation level image region, the low activation level image region being a region in the input image corresponding to a second unit serving as the unit having the activation level lower than the activation level of a first unit serving as the unit, the high activation level 10 image region being a region in the input image corresponding to the first unit; acquiring a feature amount of the post-processing image, based on image data of the acquired post-processing image; and searching a plurality of reference images for a similar image which is determined to be similar to the received input image in both visual similarity and information used for the 15 classification, based on the acquired feature amount.
14. A program causing a computer of a similar image search device to, upon receiving an input image captured of an object being examined, wherein a first target in the examined object and a second target around the first target in the examined object are captured in the input image, 20 and upon receiving a user instruction to find from a plurality of reference images a similar image similar to the received input image, process the received input image by: derive, as an activation level, a level at which, in a classifier including a plurality of layers and configured to, by processing input data based on image data from the received input image in the plurality of layers, output a result of classifying the first target, an unit in a layer among the 25 plurality of layers influences a classification result of the classifier; and acquire, using the derived activation level as weighting data, image data of a post-processing image by subjecting the image data from the received input image to image processing in such a way that a feature amount of a low activation level image region is smaller than a feature amount 22003443_1 (GHMatters) P120258.NZ of a high activation level image region, the low activation level image region being a region in the input image corresponding to a second unit serving as the unit having the activation level lower than the activation level of a first unit serving as the unit, the high activation level image region being a region in the input image corresponding to the first unit; 5 acquire a feature amount of the post-processing image, based on image data of the acquired post-processing image; and search the plurality of reference images for a similar image which is determined to be similar to the input image in both visual similarity and information used for the classification, based on the acquired feature amount.
15. A display method performed by a display device, the method comprising: searching for a similar image by the similar image search method according to claim 13; and displaying at least one of an activation map visualizing the activation level and the post- processing image in conjunction with the similar image found in the searching.
16. A program causing a computer of a display device to: search for a similar image by causing the program according to claim 14 to be executed; and display at least one of an activation map visualizing the activation level and the post- processing image in conjunction with the similar image found in the searching. 22003443_1 (GHMatters) P120258.NZ
NZ797422A 2021-05-26 Feature amount acquisition device, similar image search device, display device, feature amount acquisition method, similar image search method, display method, and program NZ797422B2 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2020137310A JP7056698B2 (en) 2020-08-17 2020-08-17 Feature amount acquisition device, similar image search device, display device, feature amount acquisition method, similar image search method, display method and program
PCT/JP2021/019924 WO2022038855A1 (en) 2020-08-17 2021-05-26 Feature amount acquisition device, similar image search device, display device, feature amount acquisition method, similar image search method, display method, and program

Publications (2)

Publication Number Publication Date
NZ797422A NZ797422A (en) 2025-09-26
NZ797422B2 true NZ797422B2 (en) 2026-01-06

Family

ID=

Similar Documents

Publication Publication Date Title
Duan et al. Visual saliency detection by spatially weighted dissimilarity
Li et al. Content-partitioned structural similarity index for image quality assessment
US5602760A (en) Image-based detection and tracking system and processing method employing clutter measurements and signal-to-clutter ratios
Lazar et al. Microaneurysm detection in retinal images using a rotating cross-section based model
Asher et al. Regional effects of clutter on human target detection performance
CN111695616B (en) Lesion classification method based on multi-mode data and related products
KR102206621B1 (en) Programs and applications for sarcopenia analysis using deep learning algorithms
Kunar et al. Low prevalence search for cancers in mammograms: Evidence using laboratory experiments and computer aided detection.
US9679216B2 (en) Morphological automatic triangle orientation detection
US12548286B2 (en) Feature amount acquisition device, similar image search device, display device, feature amount acquisition method, similar image search method, display method, and program
Matzen et al. Using eye tracking metrics and visual saliency maps to assess image utility
US9342883B2 (en) Omnibus resolution assessment target for sensors
CN117437392A (en) Cruciate ligament insertion point marking and model training methods and arthroscopic system
Unay et al. Medical image search and retrieval using local binary patterns and KLT feature points
CN114359129B (en) DR image analysis method and electronic device
NZ797422B2 (en) Feature amount acquisition device, similar image search device, display device, feature amount acquisition method, similar image search method, display method, and program
WO2013037702A1 (en) Method and a system for medical imaging
Wang et al. Saliency detection using mutual consistency-guided spatial cues combination
Chang et al. Evaluation of human detection performance using target structure similarity clutter metrics
Howell et al. An evaluation of fusion algorithms using image fusion metrics and human identification performance
Yeh et al. Fast salient object detection through efficient subwindow search
CN116258662A (en) Analysis method of DR image, DR imaging device and electronic device
Krebs et al. Comparing behavioral receiver operating characteristic curves to multidimensional matched filters
Šprem et al. Classification of coronary artery calcifications according to motion artifacts in chest CT using a convolutional neural network
Le Meur et al. Performance assessment of a visual attention system entirely based on a human vision modeling