JP6685986B2 - Object information acquisition device, control method of object information acquisition device - Google Patents

Description

  The present invention relates to a subject information acquisition apparatus that acquires information inside a subject and a control method thereof.

  In the medical field, a photoacoustic imaging device that images the morphology and function of the inside of a tissue by irradiating a living body with light such as laser light and receiving ultrasonic waves generated from the inside of the living body due to the light. Many are used. When the measurement light such as the pulsed laser light is applied to the subject, an acoustic wave is generated when the measurement light is absorbed by the living tissue in the subject. The photoacoustic imaging apparatus can visualize the information (function information) related to the optical characteristics inside the subject by receiving the generated acoustic wave with the probe and analyzing it. Such a technique is called photoacoustic imaging.

  Further, an image diagnostic apparatus has been proposed which has a mechanism for mechanically scanning a probe in order to acquire ultrasonic waves from a wide range. For example, Patent Document 1 describes a photoacoustic imaging apparatus capable of acquiring ultrasonic waves from a wide range by mechanically scanning a probe.

JP, 2010-104816, A

In the photoacoustic imaging apparatus described above, scanning is performed by moving the probe on the surface of the subject. Therefore, if the subject moves during the scanning, there is a possibility that a defect may occur in the acquired image or that the data scheduled to be acquired cannot be acquired. Even in a non-scanning photoacoustic imaging device, an image is constructed by integrating acoustic waves generated from the subject for a certain period of time, so if the subject moves during measurement, a correct image is acquired. Can not do.
As described above, in an apparatus that obtains information about a subject using acoustic waves, care must be taken so that the subject does not move during measurement. However, when it is noticed after the measurement that the subject shifts during the measurement, it is necessary to perform the measurement again from the beginning while holding the subject under pressure, which imposes a heavy burden on the subject.

  The present invention has been made in view of the above problems of the prior art, and an object of the present invention is to provide an object information acquisition apparatus capable of notifying an operator that the position of the object has changed during measurement.

In order to solve the above problems, the object information acquisition device according to the present invention,
By receiving and analyzing an acoustic wave arriving from within the subject, a subject information acquisition device for obtaining information within the subject, wherein an acoustic wave probe for receiving an acoustic wave, and the subject position information acquisition means for acquiring position information which is information relating to the position, the acquired movement amount of the object based on the position information, when the movement amount exceeds a predetermined value, the subject out detection means for detection known that positional deviation occurs, the case where positional shift in the object has occurred, and the information misalignment subject has occurred, and an image representing the moving amount of the subject And an output unit for applying a force.
Further, the processing apparatus according to the present invention,
A processing device for processing information obtained by receiving an acoustic wave coming from the inside of a subject, receiving position information which is information relating to the position of the subject, and moving the subject based on the position information. If you get the amount, if the movement amount exceeds a predetermined value, the detection means for detection known that the position deviation to the subject occurs, the positional deviation in the subject occurs, the and having the information misalignment subject has occurred, said output means for output the image representing the moving amount of the subject.

In addition, the control method of the processing device according to the present invention,
A method of controlling a processing device for processing information obtained by receiving an acoustic wave coming from within a subject, wherein position information, which is information regarding a position of the subject, is received, and the object is detected based on the position information. get the movement amount of the specimen, when the movement amount exceeds a predetermined value, a detection step of detection known that the positional deviation to the subject occurs, the positional deviation in the object has occurred when, wherein the includes information misalignment subject occurs, the output step of output the image representing the moving amount of the subject.

  According to the present invention, it is possible to provide an object information acquiring apparatus that can notify the operator that the position of the object has changed during measurement.

The block diagram of the photoacoustic measuring device which concerns on 1st embodiment. The operation | movement flowchart figure of the photoacoustic measuring device which concerns on 1st embodiment. The figure explaining the operation console of the photoacoustic measuring device which concerns on 1st embodiment. The figure explaining the operation timing of each component which a photoacoustic measuring device has. The figure explaining the operation console of the photoacoustic measuring device which concerns on a modification. The block diagram of the photoacoustic measuring device which concerns on 2nd embodiment. The operation | movement flowchart figure of the photoacoustic measuring device which concerns on 2nd embodiment. The figure explaining the operation console of the photoacoustic measuring device which concerns on 3rd embodiment.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. In principle, the same constituent elements will be given the same reference numerals, and the description thereof will be omitted.

(First embodiment)
The photoacoustic measurement apparatus according to the first embodiment of the present invention irradiates the subject with laser light, and receives and analyzes the photoacoustic wave generated in the subject due to the laser light, It is an apparatus for imaging optical characteristic value information in a subject. The optical characteristic value information is generally an initial sound pressure distribution, a light absorption energy density distribution, an absorption coefficient distribution, or a concentration distribution of a substance constituting a tissue.

<System configuration>
The configuration of the photoacoustic measurement apparatus according to the first embodiment will be described with reference to FIG. 1. The photoacoustic measuring device according to the first embodiment includes a light source 11, an optical system 13, an acoustic wave probe 17, a signal processing unit 18, a data processing unit 19, an input / output unit 20, a measuring unit 21, and a change detecting unit 22. , Notification unit 23.

The measurement is performed by inserting a subject 15 (for example, a breast) into an opening (not shown) provided in the device.
First, the pulsed light 12 emitted from the light source 11 is applied to the subject 15 via the optical system 13. When a part of the energy of the light propagating inside the subject is absorbed by the light absorber such as blood, the acoustic wave 16 is generated from the light absorber due to thermal expansion. The acoustic wave generated in the subject is received by the acoustic wave probe 17 and analyzed by the signal processing unit 18 and the data processing unit 19. The analysis result is converted into image data (optical characteristic value information data) representing characteristic information inside the subject, and is output through the input / output unit 20.
Further, in the photoacoustic measurement apparatus according to the present embodiment, when the measurement unit 21 acquires information indicating the position of the subject (positional information) and a position change of the subject that affects the measurement occurs, the change detection unit. 22 detects this and notifies the operator through the notification unit 23. This allows the operator to know that the position of the subject has changed during the measurement (that is, a remeasurement or other treatment is required).
Hereinafter, each means which comprises the photoacoustic measuring device which concerns on this embodiment is demonstrated.

<< light source 11 >>
The light source 11 is a device that generates pulsed light with which a subject is irradiated. The light source is preferably a laser light source in order to obtain a large output, but a light emitting diode, a flash lamp or the like can be used instead of the laser. When a laser is used as the light source, various ones such as a solid-state laser, a gas laser, a dye laser and a semiconductor laser can be used. Irradiation timing, waveform, intensity, etc. are controlled by a light source controller (not shown). The light source controller may be integrated with the light source.
Further, in order to effectively generate a photoacoustic wave, it is necessary to irradiate light for a sufficiently short time according to the thermal characteristics of the subject. When the subject is a living body, the pulse width of the pulsed light generated from the light source is preferably about 10 to 50 nanoseconds. Further, it is desirable that the wavelength of the pulsed light is a wavelength at which the light propagates inside the subject. Specifically, when the subject is a living body, it is preferably 500 nm or more and 1200 nm or less. Furthermore, it is desirable that the wavelength of the pulsed light has a high absorption coefficient with respect to the observation target.

<< Optical system 13 >>
The optical system 13 is a unit that guides the pulsed light 12 generated by the light source 11 to the subject 15, and typically includes a mirror that reflects light, a lens that condenses, magnifies, or changes the shape of the light, and a light. It is composed of a diffusion plate for diffusion. By using these optical members, it is possible to set the irradiation conditions such as the irradiation shape of the pulsed light, the light density, and the irradiation direction to the object to be arbitrary. In addition, it is preferable that the light is spread over a certain area rather than being collected by a lens, from the viewpoint of safety for a subject and expansion of a diagnostic region. The light source 11 and the optical system 13 are the light irradiation section in the present invention.

<< Subject 15 >>
The subject 15 and the light absorber 14 do not constitute the present invention, but will be described here. The subject 15 is an object to be subjected to photoacoustic measurement, and is typically a human body or animal breast, fingers, limbs, or the like. Here, the human breast is the subject.
The photoacoustic measurement apparatus according to the present embodiment can image the light absorber 14 inside the subject 15 and having a relatively large light absorption coefficient. When the subject is a living body, the light absorber 14 is specifically water, lipid, melanin, collagen, protein, oxyhemoglobin, reduced hemoglobin, or the like. Alternatively, a blood vessel containing a large amount of oxidized or reduced hemoglobin, a malignant tumor containing a large number of new blood vessels, or the like. By imaging the light absorber, the photoacoustic measurement apparatus according to the present embodiment can perform imaging of blood vessels, diagnosis of malignant tumors or vascular diseases of humans or animals, and follow-up observation of chemotherapy.

<< acoustic wave probe 17 >>
The acoustic wave probe 17 is means for receiving an acoustic wave generated inside the subject due to the light applied to the subject 15 and converting the acoustic wave into an analog electric signal. Note that the acoustic wave in the present invention is typically an ultrasonic wave, and includes acoustic waves, ultrasonic waves, photoacoustic waves, and elastic waves called optical ultrasonic waves. The acoustic wave probe 17 receives these elastic waves generated or reflected in the subject.
The acoustic wave probe 17 is also called a probe or a transducer. The acoustic wave probe 17 may be composed of a single acoustic detector or plural acoustic detectors. Further, the acoustic wave probe 17 may have a plurality of receiving elements arranged one-dimensionally or two-dimensionally. When the multidimensional array element is used, acoustic waves can be received at a plurality of places at the same time, so that the measurement time can be shortened and the influence of vibration of the subject can be reduced.

  Further, the acoustic wave probe 17 preferably has high sensitivity and a wide frequency band. Specific examples include PZT (piezoelectric ceramics), PVDF (polyvinylidene fluoride resin), CMUT (capacitive micromachine ultrasonic transducer), and one using a Fabry-Perot interferometer. However, the present invention is not limited to those listed here, and may be any one as long as it fulfills the function as the acoustic wave probe.

<< Signal processing unit 18 >>
The signal processing unit 18 is means for amplifying the electric signal obtained by the acoustic wave probe 17 and converting it into a digital signal. The signal processing unit 18 is typically composed of an amplifier, an A / D converter, an FPGA (Field Programmable Gate Array) chip, and the like. When there are a plurality of detection signals obtained from the probe, it is desirable to be able to process a plurality of signals at the same time.

<< Data processing unit 19 >>
The data processing unit 19 is means for generating image data (image reconstruction) by processing the digital signal obtained by the signal processing unit 18. The image reconstruction method executed by the data processing unit 19 includes, for example, the Fourier transform method, the universal back projection method, the filtered back projection method, the iterative reconstruction method, and the like, but any image reconstruction method may be used. I do not care. Further, the signal processing unit 18 and the data processing unit 19 may be integrated. The signal processing unit 18 and the data processing unit 19 are image acquisition means in the present invention.

<< I / O unit 20 >>
The input / output unit 20 is a unit that outputs the image generated by the data processing unit 19 and receives an input operation from the operator, and is a touch panel display in this embodiment. The input / output unit 20 is means for displaying detailed information about the positional deviation when the change detecting unit 22 described later detects the positional deviation of the subject. The input / output unit 20 does not necessarily have to be integrated with the photoacoustic measurement device, and may be a device connected to the outside.

<< Measurement part 21 >>
The measurement unit 21 is a measurement unit for acquiring position information of the subject, and specifically, a visible light camera, an infrared camera, a distance sensor that measures the shape of the subject, etc. is there. When a camera is used as the measuring unit 21, the frame rate and the resolution thereof need only be such that a change in the position of the subject that affects the measurement can be detected. The measurement unit 21 is the position information acquisition means in the present invention.
The measuring unit 21 may be a plurality of visible light cameras, or may be one or more sensors capable of measuring the distance to the subject. Further, any device such as an infrared camera capable of measuring the blood vessel shape on the surface of the subject may be used as long as it can detect that the subject has moved or deformed.
In the first embodiment, a visible light camera capable of capturing the entire measurement target region of the subject is used as the measurement unit 21.

<< Change detector 22 >>
The change detection unit 22 is a unit that detects that the subject is displaced during the photoacoustic measurement based on the subject image obtained by the measurement unit 21.
Here, the displacement of the subject will be described. In photoacoustic measurement, an acoustic wave generation source is estimated by receiving an acoustic wave generated in a subject with a probe. That is, if the subject moves or deforms during the photoacoustic measurement, the positional relationship with respect to the probe shifts, and an image is generated based on incorrect information. The change detection unit 22 is a unit that detects a displacement of the subject (hereinafter, simply referred to as a position displacement) that affects such measurement. Note that the positional displacement includes parallel movement, expansion, contraction, rotation, distortion, etc. of the subject in the measurement target region. The motion of the detection target may be any motion as long as the motion of the subject affects the measurement.

The change detection unit 22 acquires a plurality of subject images via the measurement unit 21, and uses each of the images to detect that the subject is displaced. The specific method will be described later.
The signal processing unit 18, the data processing unit 19, and the change detection unit 22 may be a computer having a CPU, a main storage device, and an auxiliary storage device, or may be a microcomputer or a hardware such as a dedicated FPGA. It may be wear.

<< Notification section 23 >>
The notification unit 23 is an interface for notifying the operator that the positional deviation has been detected by the change detection unit 22. The change detection unit 22 and the notification unit 23 are the notification means in the present invention. In the present embodiment, the notification unit 23 is a lamp that can emit light in a plurality of colors (for example, it is normally green and becomes red when a position shift occurs). The detailed contents of the above may be displayed on a display or a display panel.
In addition to this, when the displacement occurs, the operator may be notified through hearing such as emitting a warning sound or a melody. Any method may be used for the notification as long as the operator can recognize that the displacement of the subject has occurred. The notification unit 23 does not necessarily have to be integrated with the photoacoustic measurement device, and may be a device connected to the outside. Further, it may be integrated with the input / output unit 20.

・・・式（１） <How to detect misalignment>
Next, a method in which the change detection unit 22 detects the displacement of the subject will be described. In this example, a target region (region of interest) in which the positional displacement is detected is determined in the subject image, and the positional displacement within the region is detected. The region of interest may be designated in advance by the operator or may be automatically set by the device.
The misregistration is detected by comparing the template image with the subject image periodically acquired during the measurement. First, a subject image is acquired before the start of measurement, and the image is temporarily stored as a template image. Then, after the measurement is started, the subject image is acquired at regular intervals, and the template image and the subject image in each frame are matched. Specifically, a normalized cross-correlation (ZNCC: Zero-mean Normalized Cross-Correlation) as shown in Equation 1 is calculated to calculate the amount of change in the position of the subject.
In this embodiment, the calculation based on the normalized cross-correlation is performed, but the calculation may be performed by another method as long as it indicates a change in the position of the subject. For example, any method may be used as long as it is a method such as SSD (Sum of Squared Difference) or SAD (Sum of Absolute Difference) that can detect a change in the position of the subject. In this example, the entire subject image is set as the region of interest, but when designating the region of interest, the region of interest may be cut out from each image and the calculation may be performed.

... Formula (1)

Here, M and N are the numbers of pixels in the X and Y directions in the XY coordinate system of each image. Further, I (i, j) is a luminance value in the region of interest of the subject image being measured, and I _avg is an average value of the luminance in the region of interest. Further, T (i, j) is the brightness value in the region of interest of the template image, and _Tavg is the average value of the brightness in the region of interest.
Using Equation 1, the similarity R between the region of interest of the template image and the region of interest of the subject image can be obtained. In addition, by performing matching while shifting the coordinates and acquiring the shift amount that maximizes the similarity, the shift widths in the X direction and the Y direction can be acquired. The shift width is the amount of movement of the subject that has occurred after the start of measurement.

  As described above, the change detection unit 22 acquires the displacement width in each of the X direction and the Y direction, and determines that the position displacement has occurred in the subject based on the displacement width. Details of the determination method will be described later.

<Processing flow chart>
The processing performed by the photoacoustic measurement apparatus according to this embodiment will be described with reference to FIG.
First, the operator inputs the threshold value of the displacement width of the subject, that is, the allowable maximum value of the movement amount in the number of pixels (S1). The threshold value is preferably input from the input / output unit 20, but the threshold value may be stored as a predetermined value in the device in advance, or the device may automatically calculate.

Next, a living body (for example, a breast) that is a subject is inserted into the photoacoustic measurement device. At this time, the measuring unit 21 (visible light camera) images the images before and after inserting the subject, and acquires the difference between the images. The image acquired here becomes a template image for comparison (S2). In the following description, the subject image is the difference between the image captured while the subject is inserted and the image captured before the subject is inserted (that is, an image representing only the subject). Shall be
When step S2 ends, photoacoustic measurement is started. First, the measurement unit 21 acquires the subject image (S3), and the change detection unit 22 detects the displacement of the subject (S4). Here, as described above, the object image acquired before the measurement is started, and the deviation width of the object image captured a plurality of times at regular intervals during the measurement is acquired in number of pixels, and the threshold value is set in advance. Make a comparison. As a result, when the displacement width exceeds the threshold value, it is determined that the subject has a displacement.
In addition to this, each time a subject image is acquired, the deviation width from the start of measurement is integrated, and when the integrated deviation width exceeds a threshold value, it is determined that the position deviation of the object has occurred. Good.

If the deviation width is within the threshold as a result of executing step S4, pulse light is generated from the light source 11 and the subject is irradiated with pulse light through the optical system 13 (S5).
Then, the acoustic wave generated in the subject due to the pulsed light is acquired by the acoustic wave probe 17 (S6). When pulsed light is emitted a predetermined number of times and acoustic wave acquisition is completed, it is determined whether or not all measurements have been completed (S7), and if completed, the process ends. If not completed, the process returns to step S3, and the subject image is acquired again.
When the deviation width exceeds the threshold value as a result of executing step S4, the process transitions to step S8, and the operator is notified of that fact through the input / output unit 20 and the notification unit 23.

  FIG. 3 is a diagram showing an operation console included in the photoacoustic measurement apparatus according to the present embodiment. The operation console includes an input / output unit 20 (touch panel display) and a notification unit 23 (lamp). When the displacement of the subject occurs, the lamp normally lit in green changes to red, and detailed information (reference numeral 24) about the displacement is displayed on the touch panel display. In the present embodiment, the detailed information is the position change amount (the number of changed pixels in each of the X direction and the Y direction) on the image of the subject.

The position change amount of the subject may be displayed as a length (millimeter), or may be displayed as a change voxel value or a change amount vector. Further, when the number of changed pixels in the Z direction can be detected, they may be displayed together. Any value can be displayed as long as the value can be handled by the device.
Further, the subject image before the measurement is started and the subject image after the positional displacement may be superimposed and displayed. Further, a graphic representing the motion vector of the subject may be generated and further displayed in a superimposed manner. Any display may be performed as long as the operator can be notified of how the subject has moved or deformed.

  In addition, options for selecting the subsequent processing are displayed on the touch panel display. The content of the options may be any method that can be realized in the apparatus, such as “re-measure from the beginning”, “re-measure immediately before the deviation occurs”, and “stop measurement”. Good.

  FIG. 4 is a diagram showing the relationship between the operation timings of the measurement unit 21, the change detection unit 22, and the notification unit 23 and the laser light irradiation timing. The measurement unit 21 acquires the subject image and sends the result to the change detection unit 22. When the change detection unit 22 compares the template image with the acquired subject image and determines that the positional deviation has not occurred, the irradiation of the laser beam is started. When it is determined that the positional deviation has occurred, laser light irradiation is not performed and the operator is notified via the notification unit 23.

  According to the first embodiment, in the photoacoustic measuring device for imaging by accumulating the acoustic waves generated from the subject for a certain period of time, the positional deviation of the subject generated during measurement is accurately measured for the operator. You can be notified.

  In the present embodiment, the position shift of the subject is detected by obtaining the normalized cross-correlation between the template image and the subject image acquired at regular time intervals, but other methods may be used. . For example, in each frame, the contour may be extracted from the subject image, and the deviation may be calculated by matching the contours.

  Alternatively, the blood vessel image obtained using an infrared camera may be used as a template image, and the positional shift of the subject may be detected by calculating the normalized cross-correlation between frames. Further, the displacement of the subject may be detected by comparing the centroids of the template images, or the template image may be cut out using the region of interest set by the operator and matching with the subject image may be performed. The position shift of the sample may be detected.

  In addition, in the present embodiment, the lamp is used as the notification unit 23, but as shown in FIG. 5, the notification may be performed by a graphic displayed on the display, or a sound device may be used to notify by sound. You can go. When the notification is given by sound, an alarm sound or a melody may be used. Any sound may be used as long as it can be recognized by the operator.

(Second embodiment)
In the first embodiment, the acoustic wave probe 17 is fixed to the subject 15. On the other hand, the second embodiment is an embodiment in which the acoustic wave probe 17 is mechanically scanned to measure the object.
The configuration of the ultrasonic diagnostic apparatus according to the second embodiment is shown in FIG. The configuration of the ultrasonic diagnostic apparatus according to the second embodiment is the same as that of the first embodiment except that it has a scanning unit 26 that is a means for scanning the acoustic wave probe 17 in a two-dimensional direction. .

The scanning unit 26 is a unit that moves the acoustic wave probe 17 in a two-dimensional direction, and includes a scanning mechanism and a control unit thereof. By using the scanning unit 26, it is possible to perform photoacoustic measurement while scanning the acoustic wave probe 17 two-dimensionally. In the present embodiment, the subject 15 is fixed, and the relative positions of the subject and the acoustic probe are changed by moving the acoustic probe on the XY stage.
Although the acoustic wave probe 17 is moved by the scanning mechanism in the present embodiment, the ultrasonic probe may be fixed and the subject may be moved. In this case, a supporting unit (not shown) that supports the subject may be moved by the scanning mechanism.

Note that both the subject 15 and the acoustic wave probe 17 may be movable. When the subject 15 is moved, it is preferable that the measuring unit 21 follow the subject and make the same movement, but if the movement of the subject can be captured, the measurement unit 21 does not necessarily have to make the same movement. Further, the scanning is preferably performed while the probe is continuously moved, but may be performed while the probe is intermittently moved. Further, the scanning mechanism for scanning is preferably an electric type using a stepping motor or the like, but may be a type for manually scanning.
The type of the scanning mechanism and the scanning method are not limited to those listed here, and any one can be used as long as at least one of the subject 15 and the acoustic wave probe 17 can be moved. May be used.

  FIG. 7 shows a flowchart of processing performed by the photoacoustic measurement apparatus according to the second embodiment. The process performed by the photoacoustic measurement apparatus according to the second embodiment is almost the same as that of the first embodiment, but before the irradiation of the pulsed light is performed in step S5, the acoustic wave probe 17 is scanned by the scanning unit 26. The only difference is that step S41 for moving is added.

  As described above, the present invention can be applied to a photoacoustic measurement device that performs measurement on a subject by scanning the acoustic wave probe.

(Third embodiment)
In the first and second embodiments, the shift width between the template image acquired before starting the photoacoustic measurement and the subject image acquired during the measurement is acquired. That is, the displacement of the subject is represented by one vector. On the other hand, the third embodiment is an embodiment in which the feature points on the surface of the subject are extracted, the displacement amount is obtained for each feature point, and then the occurrence of displacement of the subject due to the location is comprehensively determined. Is.

The configuration of the ultrasonic diagnostic apparatus according to the third embodiment is the same as that of the second embodiment, but the measuring unit 21 is not an ordinary camera but a stereo camera capable of acquiring a distance. It is different from the second embodiment in points.
In addition, the change detection unit 22 according to the third embodiment does not perform pattern matching between captured images, but extracts characteristic points from each image and detects movement of the extracted characteristic points to detect positional deviation. It differs from the second embodiment in that the occurrence is determined.

A known technique can be used to extract the feature points. For example, feature points may be extracted from edge information obtained by filtering the image, or anatomical features in the image (eg, nipple in breast, shadow of blood vessel, melanin pigmentation, contour of breast, wrinkle). The feature points may be extracted from The method for extracting the feature points is not particularly limited as long as the change in the position of the feature points can be tracked between frames.
Alternatively, the feature points may be extracted from information obtained by integrating these pieces of information for a certain period of time between frames and averaging after the integration. Further, when obtaining the feature points from each of the captured frames, only a part of the image may be used, or the whole image may be used. Further, the operator may set a region of interest through the input / output unit 20 and track the feature points in the region of interest.
The feature point is a minute area for tracking the movement of the subject and does not necessarily have to be a point corresponding to one pixel.

The process flowchart of the photoacoustic measurement apparatus according to the third embodiment will be described focusing on the differences from the second embodiment.
Similar to the second embodiment, step S1 is a step of setting a threshold value of the displacement of the subject, that is, an allowable maximum value of the displacement width. However, in the third embodiment, the movement amount of the entire subject is set. The allowable maximum value of the deformation amount is set as the threshold value instead of the corresponding value. Specifically, the "allowable maximum value of the shift width of the feature point having the largest displacement amount" is set as the threshold value. The allowable value of the shift width may be input in the number of pixels, or may be input as a voxel conversion value, a distance conversion value, or the like.
The threshold may be set automatically. For example, the displacement information (for example, displacement vector value or its absolute value) corresponding to each feature point is acquired after the measurement object is inserted into the device and ready for measurement, and then the displacement information is acquired. A value obtained by multiplying by a fixed value may be used as the threshold value. These calculations may be performed by the measurement unit 21 or the change detection unit 22.

  In step S2, instead of acquiring the template image, the coordinates of the feature point in the state before the start of measurement are acquired. Specifically, the stereo camera captures the state of the subject inserted into the device before the start of measurement. Then, a plurality of corresponding feature points are extracted from the obtained set of images, and a set of coordinates of the obtained plurality of feature points is acquired. The feature points are preferably extracted from the subject portion of the subject image. The coordinates of the feature points are represented by a coordinate system whose origin is the midpoint of the stereo camera, but any coordinate system may be used as long as each coordinate can be set.

  In step S3, the plurality of feature points acquired in step S2 are tracked, and a motion vector connecting the corresponding feature points to each other in the original frame and the frame after a predetermined time is calculated. The feature point may be obtained from one frame, or may be obtained using the centroids of the corresponding feature points in a plurality of frames.

  The determination of whether or not the displacement of the subject is within a threshold value (step S4) is performed using the motion vector calculated for each feature point. In the present embodiment, the feature point having the largest movement distance is specified and the movement distance is compared with the threshold value, but the determination may be performed using another method. For example, an average value of moving distances of all feature points between two frames may be obtained, and the average value may be compared with a threshold value, or an integrated value from the start of measurement of moving distances of all feature points is obtained. , May be compared with a threshold value. Any method can be used to determine whether or not the positional deviation has occurred.

When it is determined that the position shift has occurred, RANSAC (Random Sample Consensus
) Method, it is possible to further estimate whether the subject is translating or deforming. In the RANSAC method, n characteristic points are randomly extracted, and a conversion matrix is obtained from the corresponding characteristic points.
Then, the transformation matrix is applied to other randomly extracted feature points. As a result, when a significantly large number of transformation matrices that minimize the residual sum of squares are obtained, it can be determined that a shift due to parallel movement has occurred. On the contrary, if a large number is not obtained, it can be determined that rotation or deformation has occurred. Of course, the method is not limited to the above, and another method may be used.

The processing of steps S41 and S5 to S7 is the same as that of the second embodiment.
In step S8, similar to the first and second embodiments, the operator is notified by voice, lamp, screen display, etc. In addition to this, what kind of positional displacement has occurred is further notified. You may For example, if there is no deviation, green light, if parallel movement occurs yellow light, if deformation or rotation occurs red light, etc. Good.

Alternatively, a graphic representing the motion vector of each feature point may be generated and displayed so as to be superimposed on the subject image. This allows the operator to be notified of minute changes in the position of the subject. FIG. 8 is an example of a screen when a display (reference numeral 28) representing the motion vector of the feature point is generated and superposed on the subject image 27. Here, the feature points having similar motion vectors are clustered and displayed. This allows the operator to easily understand how the subject is deformed.
The motion vector may be indicated by a method other than the exemplified method. For example, similar motion vectors may be displayed in similar colors, or the color of lines used for clustering may be changed. Further, the motion vector may be represented by using other than the arrow, or only the region having a large motion vector may be enlarged and displayed without displaying the entire subject.

  As described above, in the photoacoustic measurement apparatus according to the third embodiment, by extracting the feature points and calculating the motion vector, it is possible to deal with a case in which a part of the subject is deformed, It is possible to accurately notify the operator of how the subject is displaced.

(Modification)
It should be noted that the description of each embodiment is an example for explaining the present invention, and the present invention can be implemented by being appropriately modified or combined without departing from the spirit of the invention. The present invention can also be implemented as a method for controlling an object information acquiring apparatus including at least a part of the above processing. The above processes and means can be freely combined and implemented as long as no technical contradiction occurs.

For example, in the description of the embodiment, the example in which the subject image is pattern-matched and the example in which the coordinates of the feature points are compared are given. However, other information is used as the information for detecting the displacement of the subject. May be. For example, the background part of the subject, the inter-frame difference information, the inter-frame difference information with the frame after a certain time, the histogram information of the subject part, the texture information of the subject, the optical flow information by the gradient method or the block matching method, etc. May be
In addition, KLT (Kanade-Lucas-Tomasi)
) Method, a method by associating local correlations, or information by a method considering global consistency. Alternatively, it may be determined that the displacement has occurred simply when the subject protrudes from a preset region. Any information may be used to determine the occurrence of the positional deviation as long as the information indicates the position and the change in outer shape of the subject.

Further, the value set as the threshold value and the value presented to the operator may be the following values in addition to the number of pixels changed from the initial state as described in the present embodiment. For example, the amount of displacement of each feature point between frames, the integrated value of the amount of displacement of the feature point within a fixed time, the change direction of the feature point in space, the voxel conversion value of the acquired change data, the millimeter / centimeter conversion value And so on.
In addition, the result of classifying the amount of deviation into large, medium, and small, the type of positional deviation (for example, "parallel movement" or "partial distortion"), the deviation value from the initial state on each axis in the coordinate system, etc. May be Any value may be used as long as it can express what the positional deviation is.

Further, in each of the embodiments, the photoacoustic measuring device has been described as an example, but the present invention has acoustic wave transmitting means for transmitting ultrasonic waves to the subject, and receives the ultrasonic waves reflected in the subject. By doing so, it may be applied to an ultrasonic measurement device that visualizes information related to acoustic characteristics in the subject. The present invention can be applied to any device as long as it is an apparatus that acquires information in the subject by receiving an acoustic wave coming from the subject.

  17 ... Acoustic wave probe, 21 ... Measuring unit, 22 ... Change detecting unit, 23 ... Notification unit

Claims (22)

By receiving and analyzing an acoustic wave arriving from within the subject, a subject information acquisition device for acquiring information within the subject,
An acoustic wave probe that receives acoustic waves,
Position information acquisition means for acquiring position information that is information related to the position of the subject,
The acquired movement amount of the object based on the position information, when the movement amount exceeds a predetermined value, a detection means for detection known that the positional deviation in the object has occurred,
When said positional deviation to the subject occurs, and the information misalignment subject occurs, and output means for output the image representing the moving amount of the subject,
A subject information acquisition apparatus, comprising: Further comprising a light irradiation unit for irradiating the subject with light, and an image acquisition unit,
The acoustic wave probe outputs an electric signal by receiving an acoustic wave generated by irradiating the subject with light from the light irradiation unit,
The object information acquiring apparatus according to claim 1, wherein the image acquiring unit images information related to optical characteristics in the object based on the electric signal. Further having an image acquisition means,
The acoustic wave probe transmits an acoustic wave in the subject, and outputs an electric signal by receiving the acoustic wave reflected in the subject,
The object information acquiring apparatus according to claim 1, wherein the image acquiring unit images information related to acoustic characteristics in the object based on the electric signal. A processing device for processing information obtained by receiving an acoustic wave coming from the inside of a subject,
Position information that is information regarding the position of the subject is received, the amount of movement of the subject is acquired based on the position information, and when the amount of movement exceeds a predetermined value , the position shifts to the subject. There detection means for detection knowledge of the occurrence,
When said positional deviation to the subject occurs, and the information misalignment subject occurs, and output means for output the image representing the moving amount of the subject,
And wherein the Turkey of having a processing device. The detection means estimates the movement of the subject by pattern matching a plurality of subject images captured at different timings,
And the output means, and outputs an image representing the moving amount of the subject, the processing apparatus according to claim 4. The detection means acquires a deformation amount of the subject based on the position information,
And the output means, when the deformation amount is greater than a predetermined value, and outputs an image representing the moving amount of the subject, the processing apparatus according to claim 4. The detection means extracts one or more feature points from each of a plurality of subject images captured at different timings, and detects the change in the coordinates of the one or more feature points to move the subject. Estimate the quantity ,
And the output means, and outputs an image representing the moving amount of the subject, the processing apparatus according to claim 4. The output means is
Calculating the motion vector of the subject,
The processing device according to any one of claims 4 to 7, wherein an image showing a motion vector of the subject is output as an image representing the amount of movement of the subject. And the output means, the image shown in the figure, and outputs an image showing the motion vector of the subject by the arrow, the processing apparatus according to claim 8. The output means is
Calculate the motion vector of a plurality of points of the subject,
The processing device according to claim 8 or 9, wherein an image in which each of the motion vectors of the plurality of points is represented by a graphic is output as an image representing the amount of movement of the subject. The output means is
Calculate the motion vector of a plurality of points of the subject,
The processing device according to claim 8 or 9, wherein an image representing a movement amount of the subject is output by clustering similar motion vectors among the motion vectors of the plurality of points. The output means is
The processing device according to any one of claims 4 to 7, wherein an image showing the movement of the subject is output as a text representing the amount of movement of the subject. An image acquisition unit for imaging information related to optical characteristics in the subject based on a received signal of an acoustic wave generated by irradiating the subject with light. Item 13. The processing device according to any one of items 4 to 12. The image acquisition means for imaging the information related to the acoustic characteristic in the subject based on the received signal of the reflected wave of the acoustic wave transmitted to the subject is further included. 13. The processing device according to any one of 12. A method for controlling a processing device for processing information obtained by receiving an acoustic wave coming from within a subject, comprising:
Position information that is information regarding the position of the subject is received, the amount of movement of the subject is acquired based on the position information, and when the amount of movement exceeds a predetermined value , the position shifts to the subject. a detection step to detect known that but that have occurred,
When said positional deviation to the subject occurs, characterized in that it contains the information that misalignment subject has occurred, the output step of output the image representing the moving amount of the object, the processing unit Control method. In the output step,
Calculating the motion vector of the subject,
The control method of the processing apparatus according to claim 15, wherein an image showing a motion vector of the subject is output as an image representing the amount of movement of the subject. The control method of the processing device according to claim 16, wherein, in the output step, an image in which a motion vector of the subject is indicated by an arrow is output as the image shown in the figure. In the output step,
Calculate the motion vector of a plurality of points of the subject,
The control method of the processing device according to claim 16 or 17, wherein an image in which each of the motion vectors of the plurality of points is graphically displayed is output as an image representing the movement amount of the subject. In the output step,
Calculate the motion vector of a plurality of points of the subject,
The control method of the processing device according to claim 16 or 17, wherein an image representing a movement amount of the subject is output by clustering similar motion vectors among the motion vectors of the plurality of points. In the output step,
The control method of the processing device according to claim 15, wherein an image showing a movement of the subject is output as an image representing the movement amount of the subject. An image acquisition step of imaging information related to optical characteristics in the subject based on a received signal of an acoustic wave generated by irradiating the subject with light is further included. Item 21. A method of controlling a processing device according to any one of Items 15 to 20. The image acquisition step of imaging information related to acoustic characteristics in the subject based on a reception signal of a reflected wave of an acoustic wave transmitted to the subject, further comprising: 21. The method for controlling the processing device according to any one of 20.

Images