JP7658538B2 - Position estimation device, automatic injection device, and program - Google Patents

Description

Application of Article 30, Paragraph 2 of the Patent Act March 8, 2021 Information, Intelligence, and Precision Instruments Division (IIP Division) Lecture Presentation in the collection of lecture papers

The present invention relates to a position estimation device, an automatic injection device, a position estimation method, and a program for estimating the three-dimensional position of an object.

A position estimation device is known that acquires first and second images of an object by photographing the object from different directions, and calculates the three-dimensional position of the object based on the acquired first and second images (see, for example, Patent Document 1).

Patent No. 6532042

However, the above position estimation device does not take into account the optical properties of the surface covering the object, resulting in reduced accuracy in estimating the three-dimensional position of the object.

The present invention has been made to solve these problems, and its main objective is to provide a position estimation device, an automatic injection device, a position estimation method, and a program that can calculate the three-dimensional position of an object with higher accuracy.

In order to achieve the above object, one aspect of the present invention is to
a first image capturing means for capturing a first image and a second image capturing means for capturing a second image of an object from different directions;
an approximation equation calculation means for calculating first and second approximation equations of a line representing a characteristic shape of the object in the first and second images based on the first and second images acquired by the first and second image acquisition means, respectively;
a position calculation means for calculating a position of the object,
The position calculation means
a first surface passing through the position of the first image acquisition means and the line of the first approximation equation calculated by the approximation equation calculation means is calculated, a first intersection line between the first surface and a surface covering the object is calculated, and a first refracted line is calculated by refracting a line passing through each of the first division points obtained by dividing the first intersection line and the first image acquisition means at each of the first division points based on a refractive index within the surface,
a second surface passing through the position of the second image acquisition means and the line of the second approximation equation calculated by the approximation equation calculation means is calculated, a second intersection line between the second surface and a surface covering the object is calculated, and second refracted lines are calculated by refracting a line passing through each second division point obtained by dividing the second intersection line and the second image acquisition means at each second division point based on a refractive index within the surface,
calculating points at which the first refraction line and the second refraction line intersect, and calculating a three-dimensional position of the object based on the calculated points;
It is a position estimation device.
In this embodiment,
The approximation equation calculation means may calculate a linear linear equation or a curve equation of second or higher order as the approximation equation representing the characteristic shape of the object.
In this embodiment,
The first and second image acquisition means may be infrared cameras.
In this embodiment,
The object may be a blood vessel.
In order to achieve the above object, one aspect of the present invention is to
a puncture needle is inserted into the blood vessel based on the three-dimensional position of the blood vessel calculated by the position estimation device;
The automatic injection device may be characterized in that
In this embodiment,
A moving means for moving the puncture needle;
a force detection means for detecting a force acting on the puncture needle;
a determination means for determining whether or not the puncture needle has reached the blood vessel based on the force of the puncture needle detected by the force detection means;
A control means for controlling the moving means based on a result of the determination by the determination means;
The device may include:
In this embodiment,
The determination means may determine that the puncture needle has reached the blood vessel when the force of the puncture needle detected by the force detection means reaches a second peak.
In this embodiment,
The control means may control the moving means to withdraw the puncture needle from the blood vessel when the force of the puncture needle detected by the force detection means reaches or exceeds a threshold value.
In order to achieve the above object, one aspect of the present invention is to
photographing an object from different directions using first and second image acquisition means to acquire first and second images of the object;
calculating, based on the acquired first and second images, first and second approximation equations of a line representing a characteristic shape of the object in the first and second images, respectively;
a step of calculating a first surface passing through the position of the first image acquisition means and the calculated line of the first approximation equation, calculating a first intersection line between the first surface and a surface covering the object, and calculating first refracted lines obtained by refracting a line passing through each of first division points obtained by dividing the first intersection line and the first image acquisition means at each of the first division points based on a refractive index within the surface;
a step of calculating a second plane passing through the position of the second image acquisition means and the line of the calculated second approximation equation, calculating a second intersection line between the second plane and a surface covering the object, and calculating second refracted lines obtained by refracting a line passing through each second division point obtained by dividing the second intersection line and the second image acquisition means at each second division point based on a refractive index within the surface;
calculating points at which the first refraction line and the second refraction line intersect, and calculating a three-dimensional position of the object based on the calculated points;
The present invention may be a position estimation method including the steps of:
In order to achieve the above object, one aspect of the present invention is to
A process of photographing an object from different directions using first and second image acquisition means to acquire first and second images of the object;
a process of calculating first and second approximation equations of a line representing a characteristic shape of the object in the first and second images, respectively, based on the acquired first and second images;
a process of calculating a first surface passing through the position of the first image acquisition means and the calculated line of the first approximation equation, calculating a first intersection line between the first surface and a surface covering the object, and calculating first refracted lines obtained by refracting a line passing through each of first division points obtained by dividing the first intersection line and the first image acquisition means at each of the first division points based on a refractive index within the surface;
a process of calculating a second surface passing through the position of the second image acquisition means and the line of the calculated second approximation equation, calculating a second intersection line between the second surface and a surface covering the object, and calculating second refracted lines obtained by refracting a line passing through each second division point obtained by dividing the second intersection line and the second image acquisition means at each second division point based on a refractive index within the surface;
A process of calculating points where the first refraction line and the second refraction line intersect, and calculating a three-dimensional position of the object based on each of the calculated points;
The above-mentioned program may be executed by a computer.

The present invention provides a position estimation device, an automatic injection device, a position estimation method, and a program that can calculate the three-dimensional position of an object with high accuracy.

1 is a block diagram showing a schematic configuration of a position estimation device according to a first embodiment of the present invention. 11A and 11B are diagrams illustrating a specific method for calculating the first and second refraction lines. FIG. 2 is a flowchart showing the flow of a position estimation method according to the first embodiment. FIG. 13 is a diagram showing an experimental puncture success rate. 1A and 1B are diagrams showing estimated positions of a blood vessel cross section according to a conventional position estimation method and the position estimation method according to the first embodiment; FIG. 11 is a block diagram showing a schematic system configuration of an automatic injection device according to a second embodiment of the present invention. FIG. 2 is a diagram showing a schematic configuration of a moving section.

EMBODIMENT 1
DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS A position estimation device according to a first embodiment of the present invention can estimate three-dimensional positions of blood vessels in a human body with high accuracy, for example, in a medical setting.

However, conventional position estimation devices do not take into account the optical properties of human skin, such as the refractive index. This has led to problems with reduced estimation accuracy when estimating the three-dimensional positions of blood vessels in the human body, such as blood vessels being estimated to be shallower than their actual depth.

In contrast, in the position estimation device 1 according to the first embodiment, the three-dimensional positions of the blood vessels in the human body are estimated taking into account the optical characteristics of the skin of the human body, as described below. This allows the three-dimensional positions of the blood vessels in the human body to be estimated with high accuracy.

Fig. 1 is a block diagram showing a schematic configuration of a position estimation device according to the first embodiment. The position estimation device 1 according to the first embodiment includes first and second cameras 2 and 3 for acquiring images of blood vessels, an approximate equation calculation unit 4 for calculating an approximate equation of the blood vessels, and a position calculation unit 5 for calculating the three-dimensional position of the blood vessels.

The first and second cameras 2, 3 are specific examples of first and second image acquisition means. The first and second cameras 2, 3 each capture images of blood vessels from different directions and capture first and second images as stereo images of the blood vessels. The first and second cameras 2, 3 are preferably infrared cameras. This makes it possible to, for example, see through and capture images of blood vessels beneath the skin. The first and second cameras 2, 3 output the captured first and second images to the approximation equation calculation unit 4.

The approximate equation calculation unit 4 is a specific example of an approximate equation calculation means. The approximate equation calculation unit 4 calculates an approximate equation that represents the characteristic shape of the blood vessel based on the first and second images of the blood vessel acquired by the first and second cameras 2 and 3, respectively.

The approximate equation calculation unit 4 extracts blood vessel portions from the first and second images acquired by the first and second cameras 2 and 3. For example, the approximate equation calculation unit 4 extracts, from the first and second images, areas that have color characteristics specific to blood vessels and whose brightness and saturation are within specific ranges as blood vessel portions.

The approximation equation calculation unit 4 calculates, for example, a linear equation as an approximation equation that represents the characteristic shape of the extracted blood vessel portion. Normally, blood vessels (veins) in the arm, where blood is collected or transfused, extend in a straight line, so the characteristic shape of the blood vessel can be represented by a linear equation. This makes it possible to easily represent the characteristic shape of the blood vessel by an approximation equation.

The approximation equation calculation unit 4 binarizes the first image captured by the first camera 2. The approximation equation calculation unit 4 generates binary image data, for example, in which the areas occupied by blood vessels are designated "1" and other areas are designated "0". The approximation equation calculation unit 4 then extracts the blood vessel portion from the binarized first image. The approximation equation calculation unit 4 calculates, as the first approximation equation, a linear equation of the central axis that passes through the center of the extracted blood vessel portion (area of "1"), for example, using the least squares method.

Similarly, the approximation equation calculation unit 4, for example, binarizes the second image acquired by the second camera 3 and extracts the blood vessel portion from the binarized second image. The approximation equation calculation unit 4 calculates, as the second approximation equation, a linear equation of the central axis passing through the center of the extracted blood vessel portion, using the least squares method or the like.

In this way, an approximation formula that indicates the characteristic shape of the blood vessel, which is the target object, can be easily calculated without using complicated processing. The approximation formula calculation unit 4 outputs the calculated first and second approximation formulas to the position calculation unit 5.

The position calculation unit 5 is a specific example of a position calculation means. The position calculation unit 5 calculates the three-dimensional position of the blood vessel with high accuracy based on the first and second approximate equations of the blood vessel calculated by the approximate equation calculation unit 4, taking into account the refractive index within the skin surface (e.g., the skin model surface).

The position calculation unit 5 calculates an equation of a first surface that passes through the focal position of the first camera 2 and the line of the first approximation equation calculated by the approximation equation calculation unit 4. The position calculation unit 5 calculates a first intersection line between the calculated first surface and a plane representing the skin model surface. This first intersection line is the blood vessel image as seen from the first camera 2. The skin model surface is a specific example of a surface covering an object.

As described above, light refracts at the skin surface, but the optical path of the refracted light within the skin surface does not lie on the same plane. For this reason, in this embodiment 1, the first intersection line calculated above is finely divided, and the refraction angle is obtained from the incident light point by point, and the line of sight from the first camera 2 is rotated by different angles. In this way, the three-dimensional position of the blood vessels is estimated with high accuracy, taking into account the difference in refraction angle depending on the line of sight direction of the camera.

Specifically, the position calculation unit 5 calculates first refraction lines by refracting a line passing through each point (hereinafter, first division points) obtained by dividing the first intersection line at a predetermined interval and the focal position of the first camera 2 at each first division point based on the refractive index within the skin model surface.

For example, the first refraction line can be calculated as follows. Figure 2 is a diagram explaining a specific method for calculating the first and second refraction lines.

The unit vector from the first division point p1 _,n to the focal position _C1 of the first camera 2 is defined as the incident light V1 _,n . n is the division number. The division number is, for example, 100, but is not limited to this and may be set to any number. Increasing the division number improves the estimation accuracy, but also increases the amount of calculation. Therefore, it is preferable to set the division number to an optimal value experimentally obtained by taking both into consideration. Furthermore, the division points are equally spaced, but may be unequal.

By calculating the inner product of the incident light V1 _,n and the unit normal vector _Ns of the plane _Ls representing the skin model surface, the incident angle θ1, _{n from the focal position C1 of the first camera 2 to the first division point p1, n can} be obtained. Then, the refraction angle φ1 _,n is derived from Snell's law. The incident angle θ1 _,n and the refraction angle φ1 _,n are calculated by the following formulas.
θ _1,n = cos ⁻¹ (N _s・V _1,n )
φ _1,n =sin ⁻¹ {(n _air /n _skin )sinθ _1,n }

For example, the refractive index in the skin model surface is n _skin =1.56, and the refractive index in air is n _air =1.00. The rotation axis ω _1,n that refracts the incident light is obtained by taking the cross product of the incident light V _1,n and the normal vector N _s . The rotation angle γ _1,n of the incident light _{V 1,n} is calculated from the following formula.
ω _1,n =V _1,n ×N _s
γ _1,n = θ _1,n −φ _1,n

When the incident light V _1,n is rotated around the rotation axis ω _1,n by a rotation angle γ _1,n , the vector v _1,n after refraction is calculated as follows:
v _1,n =R _1,n ^ω1,nγ1,n (-V _1,n )

R _i,n ^ω1,nγ1,n is the Rodrigues rotation matrix. If a vector from a point on the blood vessel image in the v _1,n direction is a refracted light, the equation of the first refracted line representing the optical path is as follows:
L _1,n =p _1,n +tv _1,n
Let t be any real number.

The position calculation unit 5 calculates the second refraction line in the same manner as the first refraction line described above. The position calculation unit 5 calculates second refraction lines by refracting a line passing through each of the second division points obtained by dividing the second intersection line at a predetermined interval and the focal position C2 of the second camera ₃ at each of the second division points based on the refractive index within the skin model surface.

Specifically, the second refraction line is calculated as follows.
The unit vector from the second division point p2 _,n to the focal position _C2 of the second camera 3 is defined as the incident light V2 _,n . The incident angle θ2,n from the focal position _C2 of the second camera 3 to the _second division point p2 _, n can be obtained by calculating the inner product of the incident light V2 _{,n and} the unit normal vector Ns of the plane Ls representing the skin model surface. Then, the refraction angle φ2 _{, n} is derived from Snell's law. The incident angle θ2 _,n and the refraction angle φ2 _,n are calculated by the following formulas.
θ _2,n = cos ⁻¹ (N _s・V _2,n )
φ _2,n =sin ⁻¹ {(n _air /n _skin )sinθ _2,n }

The rotation axis ω2 _,n for refracting the incident light is obtained by taking the cross product of the incident light V2 _,n and the normal vector _Ns . The rotation angle γ2 _,n of the incident light _V2,n is calculated by the following formula:
ω _2,n =V _2,n ×N _s
γ _2,n = θ _2,n −φ _2,n

When the incident light V _2,n is rotated around the rotation axis ω _2,n by a rotation angle γ _2,n , the vector v _2,n after refraction is calculated as follows:
v _2,n =R _2,n ^ω2,nγ2,n (-V _2,n )

If a vector from a point on the blood vessel image in the v2 _,n direction is taken as refracted light, the equation of the second refracted line representing the optical path is given as follows:
L _2,n =p _2,n +tv _2,n

The position calculation unit 5 calculates the points where the first refraction line and the second refraction line intersect with each other. The position calculation unit 5 calculates the points where the first refraction line L1 _,n = p1 _,n + tv1 _,n and the second refraction line L2 _,n = _p2,n + tv2 _,n intersect with each other for the n division points.

More specifically, the points on the first and second refraction lines where the distance between the first refraction line L1 _,n = p1 _,n + _tv1,n and the second refraction line L2 _,n = p2 _,n + tv2 _,n are shortest are defined as t1 _,n and t2 _,n , respectively. The position calculation unit 5 calculates these points t1 _,n and t2 _,n using the following equations.

The position calculation unit 5 calculates the midpoints of the shortest points t1 _{, n} , t2 _{, and n} . The position calculation unit 5 calculates the position of the blood vessel as a three-dimensional linear equation based on the calculated midpoint group. The position calculation unit 5 calculates the position of the blood vessel as a three-dimensional linear equation by performing, for example, principal component analysis on the calculated midpoint group.

Figure 3 is a flowchart showing the flow of the position estimation method according to the first embodiment.

The first and second cameras 2, 3 each capture an image of the blood vessel from a different direction, acquire a first and second image of the blood vessel, and output the acquired first and second images to the approximation equation calculation unit 4 (step S101).

The approximation formula calculation unit 4 calculates first and second approximation formulas representing the characteristic shape of the blood vessel based on the first and second images of the blood vessel output from the first and second cameras 2 and 3, respectively, and outputs the calculated first and second approximation formulas to the position calculation unit 5 (step S102).

The position calculation unit 5 calculates an equation of a first surface that passes through the focal position of the first camera 2 and the line of the first approximation equation calculated by the approximation equation calculation unit (step S103). The position calculation unit 5 calculates a first intersection line between the calculated first surface and a plane representing the skin model surface (step S104).

The position calculation unit 5 calculates first refraction lines obtained by refracting a line passing through each of the first division points obtained by dividing the first intersection line at a predetermined interval and the focal position of the first camera 2 at each of the first division points based on the refractive index within the skin model surface (step S105).

The position calculation unit 5 calculates the equation of a second surface that passes through the focal position of the second camera 3 and the line of the second approximation equation calculated by the approximation equation calculation unit 4 (step S106). The position calculation unit 5 calculates a second intersection line between the calculated second surface and a plane representing the skin model surface (step S107).

The position calculation unit 5 calculates second refraction lines obtained by refracting a line passing through each of the second division points obtained by dividing the second intersection line at a predetermined interval and the focal position of the second camera 3 at each of the second division points based on the refractive index within the skin model surface (step S108).

The position calculation unit 5 calculates the points where the first and second refraction lines calculated above intersect (step S109). The position calculation unit 5 calculates the position of the blood vessel as a three-dimensional linear equation based on the calculated point group (step S110).

The estimation accuracy of the position estimation method according to this embodiment 1 is then confirmed by the following experiment. In this experiment, the robot arm automatically punctures the blood vessel based on the three-dimensional position of the blood vessel estimated by the position estimation method. The computer transmits the coordinate values (X, Y, Z) and posture angles (α, β, γ) of the hand T to the controller, and the controller controls the multi-jointed robot arm to that position.

Based on images captured simultaneously by two infrared cameras, the three-dimensional positions of blood vessels are estimated using the position estimation method according to the first embodiment and a conventional position estimation method. Note that the conventional position estimation method estimates the three-dimensional positions without taking into account the refraction at the skin surface as described above.

The skin model was created by passing a silicone tube with an outer diameter of 4 mm and an inner diameter of 3 mm as a simulated blood vessel through a case made of acrylic board, and then filling it with ultra-soft urethane modeling resin (human skin gel hardness 0, Exseal). India ink was placed inside the tube as a simulated blood sample, and three markers were attached to the surface of the skin model to define the skin plane.

The skin model was placed in two areas, close and far from the camera. In each area, the blood vessel position was estimated when the skin model was horizontal and when it was tilted ±10° around the blood vessel axis, and the automatic puncture operation was performed. The skin model was placed five times under each condition, at different positions and angles for each sample, and the success or failure of the puncture was compared. The syringe was pulled after the puncture operation, and if backflow of blood was confirmed, it was considered a success.

Figure 4 shows the puncture success rate from the above experiment. As shown in Figure 4, with the conventional position estimation method, the puncture success rate for all distances (short and long distances) was 33%, but with the position estimation method according to this embodiment 1, the puncture success rate for all distances is 76%, which is a significant improvement. In particular, with the position estimation method according to this embodiment, estimation accuracy is maintained even in a position where the skin plane is tilted (±10°), which is a weakness of the conventional position estimation method, and the puncture success rate is improved.

Furthermore, with the conventional position estimation method, the position was estimated to be shallower than the actual blood vessel position, and the needle stopped before reaching the simulated blood vessel, resulting in a failed puncture. In contrast, with the position estimation method according to the first embodiment, the needle reached the inside of the blood vessel and was successful in puncturing. This type of result was often seen when the puncture target was located far away from the camera.

Figure 5 is a diagram showing the estimated positions of blood vessel cross sections using a conventional position estimation method and the position estimation method according to this embodiment 1. As shown in Figure 5, it can be seen that the position estimation method according to this embodiment 1 estimates a location closer to the true blood vessel position than the conventional position estimation method.

As described above, the position estimation device 1 according to the first embodiment calculates a first plane passing through the position of the first camera 2 and the line of the first approximation formula calculated by the approximation formula calculation unit 4, calculates a first intersection line between the first plane and the skin surface covering the blood vessel, calculates a first refraction line by refracting a line passing through each of the first division points obtained by dividing the first intersection line at a predetermined interval and the first camera 2 at each of the first division points based on the refractive index in the skin surface, calculates a second plane passing through the position of the second camera 3 and the line of the second approximation formula calculated by the approximation formula calculation unit 4, calculates a second intersection line between the second plane and the skin surface covering the blood vessel, calculates a second refraction line by refracting a line passing through each of the second division points obtained by dividing the second intersection line at a predetermined interval and the second camera 3 at each of the second division points based on the refractive index in the skin surface, calculates the points at which the first refraction line and the second refraction line intersect, and calculates the three-dimensional position of the blood vessel based on each calculated point.

This allows the three-dimensional position of the blood vessel to be calculated with high accuracy, taking into account the refractive index within the skin surface that covers the blood vessel. In addition, the first and second intersection lines are divided, the refraction angle is obtained from the incident light for each of the first and second division points, and the lines of sight from the first and second cameras 2, 3 are rotated by different angles. This allows the three-dimensional position of the blood vessel to be estimated with high accuracy, taking into account the difference in refraction angle depending on the line of sight direction of the camera.

In the above embodiment 1, the target object is a human blood vessel in a medical setting, but the target object is not limited to this and any object may be used. As a result, the three-dimensional position of any object can be calculated with high accuracy using a simple method, just like in the case of blood vessels.

For example, when an object is covered with a surface (glass, liquid, etc.) and light is refracted on the surface, the position estimation device 1 according to the first embodiment can estimate the three-dimensional position of the object with high accuracy. In particular, the position estimation device 1 according to the first embodiment can estimate the three-dimensional position of the object with high accuracy not only when the surface is flat, but also when the surface is uneven.

In the above-mentioned first embodiment, the approximation formula calculation unit 4 calculates linear equations as the first and second approximation formulas, but is not limited to this. The approximation formula calculation unit 4 may calculate curved equations such as two-dimensional curves or three-dimensional curves as the first and second approximation formulas.

In this case, the position calculation unit 5 calculates a first surface equation of the first surface that includes the position of the first camera 2 and the first curve (first approximate equation) on the first image calculated by the approximate equation calculation unit 4.

The position calculation unit 5 calculates a first intersection line between the calculated first curved surface and a plane representing the skin model surface. The position calculation unit 5 calculates first refraction lines obtained by refracting a line passing through each of the first division points obtained by dividing the first intersection line at a predetermined interval and the focal position of the first camera 2 at each of the first division points based on the refractive index within the skin model surface.

Similarly, the position calculation unit 5 calculates a second surface equation of the second surface including the position of the second camera 3 and the second curve (second approximate equation) on the second image calculated by the approximate equation calculation unit 4.

The position calculation unit 5 calculates a second intersection line between the calculated second curved surface and a plane representing the skin model surface. The position calculation unit 5 calculates second refraction lines obtained by refracting a line passing through each of the second division points obtained by dividing the second intersection line at a predetermined interval and the focal position of the second camera 3 at each of the second division points based on the refractive index within the skin model surface.

The position calculation unit 5 calculates the points where the first refraction line and the second refraction line calculated above intersect. The position calculation unit 5 calculates the three-dimensional position of the blood vessel based on the calculated point group.

EMBODIMENT 2
The automatic injection device according to the second embodiment of the present invention automatically punctures a puncture needle into a person's blood vessel in, for example, a medical facility, etc., to collect blood, perform blood transfusion, etc. The automatic injection device according to the second embodiment automates blood collection, transfusion, etc., performed in a medical facility, thereby preventing secondary infection among medical staff, eliminating waiting times in hospitals, and resolving problems associated with blood collection, transfusion, etc., such as puncture errors.

Figure 6 is a block diagram showing the general system configuration of the automatic injection device according to the second embodiment.

The automatic injection device 10 according to the second embodiment inserts a puncture needle into a blood vessel based on the three-dimensional position of the blood vessel calculated by the position estimation device 1 according to the first embodiment. The blood vessel is, for example, a superficial venous vessel in the antecubital fossa of a person's forearm.

The automatic injection device 10 includes a position estimation device 1, a movement unit 11 that moves the puncture needle, a force detection unit 12 that detects the force applied to the puncture needle, a determination unit 13 that determines whether the puncture needle has reached a blood vessel, and a control unit 14 that controls the movement unit 11.

As described above, the position estimation device 1 calculates the three-dimensional position of the blood vessel, and outputs the calculated three-dimensional position of the blood vessel to the control unit 14. The position estimation device 1 has a hardware configuration of a normal computer including, for example, a processor 1a such as a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit), an internal memory 1b such as a RAM (Random Access Memory) or a ROM (Read Only Memory), a storage device 1c such as a HDD (Hard Disk Drive) or an SDD (Solid State Drive), an input/output I/F 1d for connecting peripheral devices such as a display, and a communication I/F 1e for communicating with devices outside the device.
The position estimation device 1, the determination unit 13, and the control unit 14 may be integrally configured.

The moving unit 11 is a specific example of a moving means. For example, as shown in FIG. 7, the moving unit 11 is configured as a manipulator that can move the puncture needle to any position. The moving unit 11 has an arm unit 111, a slide mechanism 112 connected to the tip of the arm unit 111, and a linear actuator 113 that drives the slide mechanism 112. The arm unit 111 is a multi-joint arm composed of multiple joints 114 and links 115. Each joint 114 is provided with a servo motor that rotates each joint 114, a rotation sensor that detects rotation information of each joint 114, and the like.

A syringe 116 having a puncture needle is connected to the slide mechanism 112. The linear actuator 113 drives the slide mechanism 112 to linearly move the puncture needle of the syringe 116 closer to or further away from the blood vessel.

The force detection unit 12 is a specific example of a force detection means. The force detection unit 12 is composed of, for example, a six-axis force sensor. The force detection unit 12 is provided, for example, in the slide mechanism 112. The force detection unit 12 can detect the force acting on the puncture needle when the slide mechanism 112 brings the syringe 116 close to the blood vessel and punctures the puncture needle into the blood vessel.

The determination unit 13 is a specific example of a determination means. The determination unit 13 determines whether or not the puncture needle has reached (penetrated) a blood vessel based on the force of the puncture needle detected by the force detection unit 12. The determination unit 13 determines that the puncture needle has reached a blood vessel when the force of the puncture needle detected by the force detection unit 12 reaches a second peak, for example. This is because the first peak indicates that the puncture needle has penetrated the skin, and the second peak indicates that the puncture needle has penetrated the blood vessel wall. In this way, it is possible to easily and accurately determine whether the puncture needle has reached a blood vessel.

The control unit 14 is a specific example of a control means. The control unit 14 performs, for example, feedback control or robust control by sending control signals to the arm unit 111 and the linear actuator 113. This allows the control unit 14 to move the puncture needle of the syringe 116 to any position, such as moving the puncture needle of the syringe 116 closer to or farther away from the blood vessel.

The control unit 14 controls the arm unit 111 and the linear actuator 113 based on the three-dimensional position of the blood vessel output from the position estimation device 1 to insert the puncture needle of the syringe 116 into the blood vessel.

For example, the control unit 14 controls the arm unit 111 based on the three-dimensional position of the blood vessel output from the position estimation device 1 to bring the syringe 116 within a predetermined distance from the blood vessel. Next, the control unit 14 controls the linear actuator 113 to move the puncture needle of the syringe 116 in a straight line to puncture the blood vessel.

At this time, the control unit 14 moves the puncture needle to a target position, for example, a position preset on the intersection line indicating the blood vessel, based on the three-dimensional position of the blood vessel output from the position estimation device 1. For example, the center position of the intersection line, a position at a predetermined ratio, a position specified by the user, etc. are preset in the memory.

The control unit 14 controls the linear actuator 113 of the moving unit 11 based on the determination result by the determination unit 13. For example, based on the determination result by the determination unit 13 that the puncture needle has reached a blood vessel, the control unit 14 controls the linear actuator 113 to stop the linear movement of the puncture needle of the syringe 116. The control unit 14 then controls blood collection and transfusion from the puncture needle of the syringe 116. Note that the mechanisms and controls for blood collection and transfusion are well known and are described in detail in, for example, JP 2014-239831 A, and therefore this can be used by reference.

When the control unit 14 determines that blood collection has been completed, it may control the linear actuator 113 to automatically remove the puncture needle of the syringe 116 from the blood vessel. This automates blood collection and transfusion, greatly reducing the burden on medical staff.

When inserting the puncture needle of the syringe 116 into a blood vessel, it is preferable for safety reasons to monitor in real time whether any abnormal force is acting on the puncture needle, and if any abnormal force is acting, to immediately remove the puncture needle from the blood vessel.

In contrast, the automatic injection device 10 according to the second embodiment withdraws the puncture needle from the blood vessel when the force acting on the puncture needle exceeds a threshold value. In other words, if an abnormal force is acting on the puncture needle such that the force acting on the puncture needle exceeds a threshold value, the puncture needle is immediately withdrawn from the blood vessel.

For example, when the control unit 14 determines that the force of the puncture needle detected by the force detection unit 12 exceeds a threshold value, it controls the linear actuator 113 of the moving unit 11 to withdraw the puncture needle from the blood vessel. This allows the puncture needle to be withdrawn from the blood vessel immediately, improving safety, even if an abnormal force is applied when inserting the puncture needle into the blood vessel.

In this embodiment 2, the automatic injection device 10 is configured to insert a puncture needle into a blood vessel and perform blood sampling, etc., based on the three-dimensional position of the blood vessel calculated by the position estimation device 1, but is not limited to this. In this embodiment 2, the automatic injection device 10 may be configured to puncture a blood vessel (such as an artery in the foot or base of the hand) based on the three-dimensional position of the blood vessel calculated by the position estimation device 1, and insert a catheter into the blood vessel.

Furthermore, the position estimation device 1 may calculate the three-dimensional position of a nerve under the skin, and the automatic injection device 10 may perform anesthesia injections such as lumbar anesthesia or nerve block injections based on the three-dimensional position of the nerve calculated by the position estimation device 1.

In this embodiment 2, the position estimation device 1 may estimate the position of only a small, predetermined area of the entire blood vessel image that includes the area where the puncture will be performed, rather than estimating the position of the entire blood vessel image. This can reduce the amount of calculation and shorten the time required for position estimation.

Although several embodiments of the present invention have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These novel embodiments can be embodied in various other forms, and various omissions, substitutions, and modifications can be made without departing from the gist of the invention. These embodiments and their modifications are included within the scope and gist of the invention, and are included in the scope of the invention and its equivalents as set forth in the claims.

The present invention can also be realized, for example, by having a CPU execute a computer program to perform the process shown in FIG. 3.

The program can be stored and provided to a computer using various types of non-transitory computer readable media. Non-transitory computer readable media include various types of tangible storage media. Examples of non-transitory computer readable media include magnetic recording media (e.g., flexible disks, magnetic tapes, hard disk drives), magneto-optical recording media (e.g., magneto-optical disks), CD-ROMs (Read Only Memory), CD-Rs, CD-R/Ws, and semiconductor memories (e.g., mask ROMs, PROMs (Programmable ROMs), EPROMs (Erasable PROMs), flash ROMs, and RAMs (random access memories)).

The program may be provided to the computer by various types of transitory computer readable media. Examples of transitory computer readable media include electrical signals, optical signals, and electromagnetic waves. The transitory computer readable medium may provide the program to the computer via a wired communication path, such as an electrical wire or optical fiber, or via a wireless communication path.

REFERENCE SIGNS LIST 1 Position estimation device 2 First camera 3 Second camera 4 Approximation formula calculation unit 5 Position calculation unit 10 Automatic injection device 11 Movement unit 12 Force detection unit 13 Determination unit 14 Control unit 111 Arm unit 112 Slide mechanism 113 Linear actuator 114 Joint unit 115 Link 116 Syringe

Claims (9)

a first image capturing means for capturing a first image and a second image capturing means for capturing a second image of an object from different directions;
an approximation equation calculation means for calculating first and second approximation equations of a line representing a characteristic shape of the object in the first and second images based on the first and second images acquired by the first and second image acquisition means, respectively;
a position calculation means for calculating a position of the object,
The position calculation means
a first surface passing through the position of the first image acquisition means and the line of the first approximation equation calculated by the approximation equation calculation means; a first intersection line between the first surface and a surface covering the object; and a plurality of first refracted lines obtained by refracting a plurality of lines passing through the first image acquisition means and a plurality of first division points dividing the first intersection line along an extension direction of the first intersection line at the plurality of first division points based on a refractive index within the surface,
a second surface passing through the position of the second image acquisition means and the line of the second approximation equation calculated by the approximation equation calculation means; a second intersection line between the second surface and a surface covering the object is calculated; and a plurality of second refracted lines are calculated by refracting a plurality of lines passing through a plurality of second division points that divide the second intersection line along an extension direction of the second intersection line and the second image acquisition means at each of the second division points based on a refractive index within the surface;
calculating points at which the plurality of first refraction lines intersect with the plurality of second refraction lines, and calculating a three-dimensional position of the object based on each of the calculated points;
Location estimation device. 2. The position estimation device according to claim 1,
the approximation equation calculation means calculates a linear equation or a curve equation of second or higher order as an approximation equation representing the characteristic shape of the object;
A position estimation device comprising: 3. The position estimation device according to claim 1,
The first and second image acquisition means are infrared cameras.
A position estimation device comprising: 4. A position estimation device according to claim 1, further comprising:
The object is a blood vessel.
A position estimation device comprising: A position estimation device according to claim 4,
a puncture needle is inserted into the blood vessel based on the three-dimensional position of the blood vessel calculated by the position estimation device;
The automatic injection device is characterized by: 6. The automatic injection device of claim 5,
A moving means for moving the puncture needle;
a force detection means for detecting a force acting on the puncture needle;
a determination means for determining whether or not the puncture needle has reached the blood vessel based on the force of the puncture needle detected by the force detection means;
A control means for controlling the moving means based on a result of the determination by the determination means;
1. An automatic injection device comprising: 7. The automatic injection device of claim 6,
the determination means determines that the puncture needle has reached the blood vessel when the force of the puncture needle detected by the force detection means reaches a second peak.
The automatic injection device is characterized by: 8. The automatic injection device of claim 6 or 7,
The control means controls the moving means to withdraw the puncture needle from the blood vessel when the force of the puncture needle detected by the force detection means becomes equal to or greater than a threshold value.
16. An automatic injection device comprising: A process of photographing an object from different directions using first and second image acquisition means to acquire first and second images of the object;
a process of calculating first and second approximation equations of a line representing a characteristic shape of the object in the first and second images, respectively, based on the acquired first and second images;
a process of calculating a first surface passing through the position of the first image acquisition means and the line of the calculated first approximation equation, calculating a first intersection line between the first surface and a surface covering the object, and calculating a plurality of first refracted lines obtained by refracting a plurality of lines passing through the first image acquisition means and a plurality of first division points dividing the first intersection line along an extension direction of the first intersection line at each of the first division points based on a refractive index within the surface;
a process of calculating a second surface passing through the position of the second image acquisition means and the line of the calculated second approximation equation, calculating a second intersection line between the second surface and a surface covering the object, and calculating a plurality of second refracted lines obtained by refracting a plurality of lines passing through the second image acquisition means and a plurality of second division points dividing the second intersection line along an extension direction of the second intersection line at each of the second division points based on a refractive index within the surface;
A process of calculating points at which the first refraction lines intersect with the second refraction lines, and calculating a three-dimensional position of the object based on each of the calculated points;
A program that causes a computer to execute the following.

Images