JP7801896B2 - Robot hand and robot hand control method - Google Patents

Description

The present invention relates to a robot hand and a method for controlling a robot hand.

A robot that performs a task is equipped with a hand that functions like a human hand and an arm that functions like a human arm. Such tasks are controlled, for example, by a control device. The control device performs the task based on images captured by a camera installed in the workspace, for example. In this case, when the hand approaches an object to grasp it, the arm or hand is in the blind spot of the camera, blocking its field of view. For this reason, it has been proposed to attach a camera to the tip of the robot arm (see, for example, Patent Document 1).

Japanese Patent Application Laid-Open No. 2020-196116

However, with conventional technology, if a finger, for example, blocks the field of view, the camera device cannot capture an image of the contact surface between the target object and the finger. Furthermore, with conventional technology, the target object may be in a blind spot, making it impossible to obtain contact information for each finger.

The present invention was made in consideration of the above-mentioned problems, and aims to provide a robotic hand and a robotic hand control method that can reduce blind spots within the robotic hand.

(1) To achieve the above object, a robot hand according to one aspect of the present invention comprises a base body, two or more fingers connected to the base body, and a first camera located between the two fingers and on the side of the base body.

(2) In addition, in a robot hand according to one aspect of the present invention, the first camera may be attached to the base body at an angle toward the grasping direction when grasping with the fingers.

(3) Furthermore, in a robot hand according to one aspect of the present invention, the two or more fingers may include a first finger portion close to the arm connected to the robot hand, a second finger portion close to the tip of the hand, and a second camera, the first camera being installed between the first finger portion and the second finger portion, and the second camera being installed on the side of the base between the second finger portion and the arm.

(4) In addition, in a robot hand according to one aspect of the present invention, the second camera may be installed in the ball of the little finger.

(5) Furthermore, in a robot hand according to one aspect of the present invention, stereo vision may be performed using images captured by the first camera and images captured by the second camera.

(6) Furthermore, in a robot hand according to one aspect of the present invention, a third camera may be provided on the palm surface side of the robot hand.

(7) In order to achieve the above-mentioned object, one aspect of the present invention provides a robot hand control method for operating a robot hand that includes a base unit, finger units connected to the base and capable of manipulating an object, and a camera located near the contact points of the finger units (places where the fingers come into contact with the object for grasping), and includes the steps of: determining the object; determining an environmental target for manipulating the object; determining a grasping method for grasping the object; capturing an image of the object and the environment for manipulating the object while the object is being grasped with the fingers using the camera; obtaining position information for the object based on the captured image; and determining the amount of manipulation of the object based on the obtained position information.

(8) Furthermore, in a robot hand control method according to one aspect of the present invention, the robot hands may be multiple, at least two of the multiple robot hands may be equipped with cameras, the cameras equipped on the at least two robot hands may capture images of the same target object, and the captured images may be used to perform cooperative control of the at least two robot hands.

In (1) to (8), a camera is installed on the side of the robot hand. As a result, according to (1) to (8), in this embodiment, the blind spot of the camera during grasping can be reduced.
Furthermore, according to (5) and (6), stereoscopic vision can be achieved using a camera equipped on the robot hand.

FIG. 1 is a diagram illustrating an example of the configuration of a robot hand according to an embodiment. FIG. 10 is a diagram illustrating another configuration example of a robot hand according to an embodiment. FIG. 2 is a diagram illustrating an example of installation of a camera according to an embodiment. FIG. 2 is a diagram showing an example of a shooting range of a camera according to an embodiment, as viewed from below. FIG. 2 is a diagram showing an example of the imaging range of two cameras according to the embodiment as viewed from the side. 10A and 10B are diagrams showing example images taken by a camera when a target object is grasped; This is a diagram to explain that no dead zone occurs when taking a picture. This is a diagram to explain that no dead zone occurs when taking a picture. FIG. 2 is a diagram illustrating an example of the arrangement of a stereo camera according to an embodiment. FIG. 2 is a diagram illustrating an example of the angle of view of a stereo camera. FIG. 1 is a diagram illustrating an example of the configuration of a robot including a robot hand having a camera according to an embodiment. 10 is a flowchart of an example of a control procedure for a robot hand by a robot control device according to an embodiment. FIG. 10 is a diagram illustrating a comparative example of a robot hand in which a camera is attached to an arm. FIG. 10 is a diagram for explaining a comparative example of a robot hand in which a camera is attached to a base body. FIG. 10 is a diagram showing an example in which a robot hand according to an embodiment is provided with cameras on the fingertips. FIG. 10 is a diagram for explaining an example of the installation position of a camera on a fingertip. FIG. 10 is a diagram for explaining an example of the installation position of the fingertip camera as viewed from the palm side. FIG. 10 is a diagram showing an example of the image capturing range of the palm camera and the fingertip camera.

Embodiments of the present invention will be described below with reference to the drawings. Note that the scale of each component in the drawings used in the following description has been appropriately adjusted to ensure that each component is large enough to be recognized.

(Example of robot hand configuration)
1 is a diagram showing an example of the configuration of a robot hand according to this embodiment. The robot hand 1 in FIG. 1 is an example having four fingers.
The robot hand 1 (end effector) in the configuration example of Fig. 1 includes finger portions 101, 102, 103, 104, a base 111, and cameras 131, 132, 133, and 134. The robot hand 1 is connected to an arm 121 via a joint. The robot hand 1 includes at least two fingers. The robot hand 1 also includes at least one camera.

Finger portion 101 corresponds, for example, to a human thumb. Finger portion 102 corresponds, for example, to a human index finger. Finger portion 103 corresponds, for example, to a human middle finger or ring finger. Finger portion 104 corresponds, for example, to a human ring finger or little finger. Each finger portion has a joint and a node. Base body 111 includes positions corresponding to the back and palm of a human hand.

Cameras 131-134 are, for example, RGB-D cameras capable of acquiring RGB information and seismic intensity information. Cameras 131-134 may also be a combination of a CCD (Charge Coupled Device) imaging device, a CMOS (Complementary MOS) imaging device, and a sensor capable of acquiring depth information.

In the embodiment, the first finger is one of the finger 101, the finger 102, the finger 103, and the finger 104. The second finger is a finger other than the first finger.
In the embodiment, the first camera is one of the cameras 131, 132, 133, and 134. The second camera is a camera other than the first camera.

Camera 131 is installed, for example, at a position corresponding to the ball of a human's thumb. Note that camera 131 may also be installed, for example, at a position corresponding to the outer side of the proximal joint of a human's thumb, not the index finger side. In this way, camera 131 is installed between finger portion 101 and finger portion 104.

Camera 132 is installed, for example, at a position corresponding to the ball of a human's thumb. Camera 132 may also be installed at a position corresponding to the index finger side of the proximal joint of a human's thumb. In this way, camera 132 is installed between finger portion 101 and finger portion 102.

Camera 133 is installed, for example, at a position corresponding to the side of the thumb at the base of the four human fingers, or the sides of the thumb and index finger at the thenar eminence. In this way, camera 133 is installed between finger portion 101 and finger portion 102.

Camera 134 is installed, for example, at a position corresponding to region 153 on the side of the little finger or ring finger at the base of a person's four fingers, or on the side of the little finger ball. In this way, camera 134 is installed between finger portion 104 and finger portion 101.

2 is a diagram showing another example of the configuration of the robot hand according to this embodiment. The robot hand 1 in FIG. 2 is an example having five fingers.
2 includes finger portions 101, 102, 103, 104, 105, a base 111, and cameras 131, 132, 133, and 134. The robot hand 1A is connected to an arm 121 via a joint. The robot hand 1 includes at least two fingers. The robot hand 1 also includes at least one camera.

Finger portion 101 corresponds, for example, to a human thumb. Finger portion 102 corresponds, for example, to a human index finger. Finger portion 103 corresponds, for example, to a human middle finger. Finger portion 104 corresponds, for example, to a human ring finger. Finger portion 105 corresponds, for example, to a human little finger or ring finger. Each finger portion has a joint and a node. Base body 111 includes positions corresponding to the back and palm of a human hand.

In the embodiment, the first finger portion is one of the finger portion 101, the finger portion 102, the finger portion 103, the finger portion 104, and the finger portion 105. The second finger portion is a finger portion other than the first finger portion.
In the embodiment, the first camera is one of the cameras 131, 132, 133, and 134. The second camera is a camera other than the first camera.

Camera 131 is installed in area 151, which corresponds to the ball of a human's thumb, for example. Note that area 151 may also include the outer side of the proximal part of the human thumb, not the side facing the index finger.

Camera 132 is installed in area 152, which corresponds to the ball of a human's foot, for example. Note that area 152 may also include the index finger side of the proximal part of the human thumb.

Camera 133 is installed in area 153, which corresponds to the side of the base of a person's four fingers on the thumb side, or the side of the thumb and index finger at the thenar eminence.

Camera 134 is installed in area 154, which corresponds to, for example, the side of the little finger or ring finger at the base of a human's four fingers, or the side of the hypothenar area.

In the robot hand 1 (or 1A), the cameras 131-134 are installed so that the grasping direction is visible from the cameras. The cameras 131-134 are also installed so that the grasping direction is aligned with the center of the camera's angle of view. The cameras 131-134 are installed so that the grasping can be seen from the wrist or the ball of the foot. The camera 134 is installed on the ball of the little finger so that the grasping can be seen. By installing the cameras in the robot hand 1 (or 1A) in this way, this embodiment improves visibility of contact between the fingers in the hand and the target object, meaning that blind spots are reduced. The following explanation uses a robot hand 1 with four fingers as an example.

(Example of camera installation)
3 is a diagram showing an example of camera installation according to this embodiment. The camera 133 is installed between the finger units 101 and 102 and on the side of the base 111. Furthermore, the camera 133 is attached to the base 111 at an angle to the gripping direction when the finger units grasp an object. That is, as shown in FIG. 3 , the angle with respect to the x-axis is, for example, less than 90 degrees (g11), and the angle with respect to the z-axis is, for example, less than 90 degrees (g12). Note that area g13 is an example of the imaging range of the camera 133.

(Example of camera shooting range)
FIG. 4 is a diagram showing an example of the imaging range of the camera according to this embodiment, viewed from below. Range g101 is an example of the imaging range of the camera 133. Range g102 is an example of the imaging range of the camera 134. In the example of FIG. 4 , when the robot hand 1 is not grasping an object, a line g51 outside the imaging area is at an angle of, for example, 45 degrees, as indicated by an arrow g52, with respect to a line g50 that is approximately parallel to the bases of the four fingers of the body 111. Furthermore, when the robot hand 1 is not grasping an object, one diagonal line g53 of the imaging range g11 is approximately parallel to the line g50 that is approximately parallel to the bases of the four fingers of the body 111.

Note that the imaging area shown in Figure 4 is an example and is not limited to this. The size of the imaging area varies depending on, for example, the distance between the camera and the target object. Furthermore, if the robot hand 1 is equipped with multiple cameras, it is preferable that the imaging areas of each camera overlap, as shown in Figures 4 and 5. Figure 5 is a side view of an example of the imaging range of two cameras according to this embodiment. Furthermore, the size of the imaging area of each camera may be the same or different.

In this embodiment, the camera is installed to capture this range. This improves visibility of contact between the fingers in the hand and the target object, meaning there are fewer blind spots.

(Example of captured image)
Next, an example of an image captured by the camera when grasping a target object will be described with reference to Fig. 6. Fig. 6 is a diagram showing an example of an image captured by the camera when grasping a target object. Fig. 6 shows an example of an attempt to grasp a target object with finger portions 101 and 104.

Image g201 is a side view of the robot hand 1 before it grasps the coin-shaped target object Obj. Image g211 is an image taken with a camera on the thumb side (e.g., camera 133). Image g212 is an image taken with a camera on the little finger side (e.g., camera 134). Images g211 and g212 are examples in which the fingertips are tilted relative to the target object.

Image g221 is an image taken with a camera on the thumb side (e.g., camera 133). Image g222 is an image taken with a camera on the little finger side (e.g., camera 134). Images g221 and g222 are examples in which the fingertip is tilted relative to the target object.

As shown in Figure 6, according to this embodiment, when grasping a target object, an image can be acquired using a camera installed on the side of the palm, etc. This allows the control device of the robot hand 1 to accurately control the grasping operation of the robot hand 1 using this acquired image.

Figures 7 and 8 are diagrams to explain how no dead zones are present when taking pictures. As in Figure 7, camera 134 is installed between finger portion 104 and finger portion 101 at a position and angle that prevents base 111 and other finger portions from interfering with the viewing angle or shooting range g102. As in Figure 8, camera 133 is installed between finger portion 101 and finger portion 102 at a position and angle that prevents base 111 and other finger portions from interfering with the viewing angle or shooting range g101.

(Example of a stereo camera)
Next, an example of using two cameras provided on a robot hand as a stereo camera will be described. Fig. 9 is a diagram showing an example of the arrangement of the stereo camera according to this embodiment. Note that Fig. 9 is a view of the robot hand as seen from the palm side.

9, the robot hand 1B includes two cameras, which are installed on the sides of the palm (base 111) or on the palm itself.
When the cameras are installed on the side of the palm (base 111), the first camera 133a is installed, for example, between finger portion 101 and finger portion 102, and the second camera 134a is installed, for example, between finger portion 104 and finger portion 101. When the cameras are installed on the side of the palm, there is an advantage that they can be used continuously even when the target object is grasped, because the camera does not come into contact with the object.

When installed on the surface of the palm, the first camera 133b is installed, for example, at a position corresponding to the mount of Jupiter on the palm of a human being below the finger portion 102, and the second camera 134b is installed, for example, at a position corresponding to the mount of Mercury on the palm of a human being below the finger portion 103. In this way, installing the cameras on the palm has the advantage of making it easier to align the baseline lengths of the cameras.

The cameras may be installed on the sides of the palm (base 111) and on the front side of the palm. In this case, stereo vision may be achieved by combining cameras on the front and back of the palm. In this case, for example, camera 133a or 134a corresponds to the first camera, camera 133a or 134a other than the first camera corresponds to the second camera, and camera 133b or 134b corresponds to the third camera.

In this way, the robot hand is equipped with at least two cameras, and stereo vision is achieved using two of the cameras. In this case, stereo vision of the target object is possible even if the wrist angle changes, and this has the advantage of making it easier to observe the object before grasping it than when stereo cameras are attached to the forearm or wrist.

Note that the cameras used for stereo vision are not limited to those described above. For example, any two of the cameras 131-134 provided on the robot hand 1 in FIG. 1 may be used, or any two of the cameras 131-134 provided on the robot hand 1A in FIG. 2 may be used.

Figure 10 shows an example of the angle of view of a stereo camera. As shown in Figure 10, in this case, the target object can be viewed in stereo even if the wrist angle changes. Also, as shown in Figure 10, in this case, it is easier to observe the target object before grasping it than when the stereo camera is attached to the forearm or wrist.

The images captured by each camera are used after being processed by a control device that controls a robot equipped with the robot hand 1B using a known method. The image capturing range is, for example, range g201.
The images taken by the multiple cameras may be used individually, overlapped, or used as a stereoscopic image.

(Robot Hand System)
Here, an example of the configuration of a robot equipped with a robot hand having a camera will be described.
11 is a diagram showing an example of the configuration of a robot equipped with a robot hand having a camera according to this embodiment. As shown in FIG. 11, the robot 10 includes a robot hand 1C-1 and a control device 5. Note that the robot 10 may also include two robot hands (1C-1, 1C-2).
The robot hand 1C-1 includes, for example, an actuator 201-1, a sensor 202-1, and a camera 203-2.
The robot hand 1C-2 includes, for example, an actuator 201-2, a sensor 202-2, and a camera 203-1.
The control device 5 includes, for example, an operation control unit 501 , a camera operation control unit 502 , an image processing unit 503 , and a storage unit 504 .

In the following description, if one of the robot hands 1C-1 and 1C-2 is not specified, it will be referred to as robot hand 1C. If one of the actuators 201-1 and 201-2 is not specified, it will be referred to as actuator 201. If one of the sensors 202-1 and 202-2 is not specified, it will be referred to as sensor 202. If one of the cameras 203-1 and 203-2 is not specified, it will be referred to as camera 203.

The robot hand 1C and the control device 5 are connected via wire or wirelessly. The control device 5 may be provided inside or outside the robot 10. All or some of the functions of the control device 5 may be implemented on the cloud.

The actuators 201 are installed, for example, at the joints of the fingers, the joints between the fingers and the base, and the joints between the base and the arm. The actuators 201 operate according to the control of the operation control unit 501.

Sensors 202 include, for example, encoders installed at each joint, angular velocity sensors installed in various parts, six-axis sensors, gyro sensors, etc.

There is at least one camera 203, for example, cameras (131-134) in Figure 1 or Figure 2, or cameras (133a, 133b, 134a, 134b) in Figure 9.

The operation control unit 501 controls the operation of the actuator 201 using the images captured by the camera 203 and the detection values detected by the sensor 202.

The camera operation control unit 502 controls the operation of the camera 203. The camera operation control unit 502 may, for example, change the angle of the camera depending on the holding state. The camera operation control unit 502 may, for example, change the angle of view of the camera 203 depending on the work content and the holding state. The camera operation control unit 502 may, for example, switch the sensitivity of the camera 203 depending on the work content and the holding state.

The image processing unit 503 performs image processing on the images captured by the camera 203 using well-known methods. If there are multiple cameras 203, the image processing unit 503 may process the images individually, overlay them, or process them as stereo images.

The memory unit 504 stores programs, thresholds, mathematical formulas, predetermined values, etc. used by the control device 5 for processing. The memory unit 504 may also store a trained object identification model for distinguishing target objects, a trained grasp plan model for generating a grasp plan, etc. All or part of the information stored in the memory unit 504 may be stored on another server or cloud.

(Example of robot hand control procedure)
Here, an example of a control procedure for the robot hand 1C by the control device 5 of the robot 10 will be described. Fig. 12 is a flowchart of an example of a control procedure for the robot hand by the robot control device according to this embodiment.

(Step S1) The control device 5 determines the target object for the work. Note that the target object may be determined based on instructions from the worker, for example, or on images captured by a camera. Alternatively, the control device 5 may input the captured image and the positional information of each object in the image into, for example, a trained model, to determine the target object.

(Step S2) The control device 5 determines the target object as an environmental target to be worked on.

(Step S3) The control device 5 determines a gripping method for gripping the target object, for example, using a taxonomy method (see, for example, Reference 1), depending on the target object.

Reference 1: Thomas Feix, Javier Romero, et al., “The GRASP Taxonomy of Human Grasp Types” IEEE Transactions on Human-Machine Systems (Volume: 46, Issue: 1, Feb. 2016), IEEE, p66-77

(Step S4) With the target object being held by the fingers, the control device 5 causes the camera to capture an image of the target object and the environment in which the target object is to be operated. The control device 5 acquires position information of the target object from the information contained in the captured image.

(Step S5) The control device 5 determines the amount of manipulation of the grasped object based on the acquired position information.

(Step S6) The control device 5 controls the operation of the robot hand based on the determined operation amount, for example, to grasp the target object.

The work is not limited to gripping, but may also be, for example, opening a lid on a bottle or pressing an operation button.
Furthermore, when the robot has at least two robot hands, each robot hand has a camera. The cameras of the robot hands capture images of the same target object. The control device 5 uses the captured images to perform cooperative control of the at least two robot hands.

(Comparison with conventional technology)
Next, the robot hand of this embodiment will be compared with a robot hand of the prior art.
13 is a diagram illustrating a comparative example of a robot hand having a camera attached to an arm. When a robot hand 901 having a camera 921 attached to an arm 931 grasps a target object Obj, the camera 921 cannot capture an image of the target object Obj because the field of view of the camera 921 is blocked by, for example, the finger 911, as shown in FIG.
In contrast to this, in this embodiment, for example, the camera is installed between the finger portions and on the outside of the palm, which reduces the possibility of the target object being blocked when being grasped.

14 is a diagram illustrating a comparative example of a robot hand in which a camera is attached to a base body. In a robot hand 901A in which a camera 921 is attached to the palm of a base body 941, for example, when a target object Obj is grasped with fingers 911 and 912 as shown in FIG. 14, the target object Obj blocks the field of view, making it impossible to see the hand tips (finger tips).
In contrast to this, in this embodiment, for example, the camera is installed on the fingertip of the finger, which reduces the possibility of the hand being blocked by the target object when grasping it.

(An example of a camera on your fingertip)
Next, an example will be described in which a robot hand is equipped with cameras at the fingertips as well.
15 is a diagram showing an example of a robot hand according to this embodiment that also has cameras on the fingertips. As shown in FIG. 15, a robot hand 1D has cameras 141 to 144 on the fingertips in addition to cameras 131 to 134.

The camera 141 is installed, for example, at the tip of the finger 101, which corresponds to the human thumb. The camera 141 may also be installed at the distal phalanx, distal phalanx, proximal phalanx, etc. of the finger 101.

The camera 142 is installed, for example, at the tip of the finger portion 102, which corresponds to the index finger of a human. The camera 142 may also be installed at the distal, middle, or other part of the finger portion 102.

The camera 143 is installed, for example, at the tip of the finger portion 103, which corresponds to the middle or ring finger of a human. The camera 143 may also be installed at the distal, distal, middle, or middle joint of the finger portion 103.

The camera 144 is installed, for example, at the tip of the finger portion 104, which corresponds to the little finger or ring finger of a human. The camera 144 may also be installed at the distal, middle, or other part of the finger portion 104.

Note that the installation positions of cameras 141-144 shown in Figure 15 are just an example and are not limited to this. Also, in the example shown in Figure 15, the robot hand has four fingers, but the number of fingers may be two or more.

Figure 16 is a diagram illustrating an example of the installation position of the fingertip camera. As shown in Figure 16, the fingertip camera 142 is installed, for example, between the skin portion 1021 and the nail portion 1022 of the finger portion 102. Alternatively, the camera 142 may be installed, for example, at the tip of the nail portion 1022. In this way, each finger portion has a skin portion and a nail portion. Note that the installation position shown in Figure 16 is an example and is not limited to this.

17 is a diagram illustrating an example of the installation position of the camera on the fingertip as viewed from the palm side. Note that the example shown in FIG. 17 is just an example and is not limiting.
17 , the camera of the finger portion 101 is installed offset from the central axis connecting the finger portion and the base toward the outside of the bending direction of the finger in the direction of arrow g301 with respect to the center line of the longitudinal direction of the finger portion. Furthermore, the camera of the finger portion 101 is installed with the z-axis g302 of the camera tilted toward the center of the finger.
The camera of the finger portion 102 is installed offset from the central axis connecting the finger portion and the base toward the outside in the bending direction of the finger, for example, in the direction of arrow g303 (direction of the finger portion 103) with respect to the center line of the longitudinal direction of the finger portion.
The camera of the finger portion 103 is installed offset from the central axis connecting the finger portion and the base in the direction of arrow g304 (direction of the finger portion 104) relative to the center line of the longitudinal direction of the finger portion, toward the outside in the bending direction of the finger.
The camera of the finger portion 104 is installed offset from the central axis connecting the finger portion and the base toward the outside in the bending direction of the finger, for example, in the direction of arrow g305 (outward) with respect to the center line of the longitudinal direction of the finger portion.
The reason for this installation position is that when the target object is grasped with the fingers, the camera is less likely to come into contact with the target object.

Fig. 18 is a diagram showing an example of the image capturing range of the palm camera and the fingertip camera. Note that the example in Fig. 18 is an example in which the palm camera has two cameras and the finger portion has four cameras.
The camera 141 is installed at the fingertip of the finger portion 101. An imaging range g401 is an example of the imaging range of the camera 141.
The camera 142 is installed at the fingertip of the finger portion 102. An imaging range g402 is an example of the imaging range of the camera 142.
The camera 143 is installed on the fingertip of the finger portion 103. An imaging range g403 is an example of the imaging range of the camera 143.
The camera 144 is installed at the fingertip of the finger portion 104. An imaging range g404 is an example of the imaging range of the camera 144.
The photographing range etc. shown in FIG. 18 is an example and is not limited to this.

As described above, in this embodiment, cameras are installed on the sides of the robot hand and on the fingertips, which makes it possible to reduce the blind spot of the camera when grasping an object.
Furthermore, when cameras are installed on the sides and fingertips of the robot hand as in this embodiment, the captured images can be used for grasping small target objects in the in-hand state, posture control before peg-in-hole in the in-hand state, fine position correction in visual servoing, hand posture correction in visual servoing, etc.

In addition, a program for implementing all or part of the functions of the control device 5 of the present invention may be recorded on a computer-readable recording medium, and the program recorded on this recording medium may be loaded into a computer system and executed to perform all or part of the processing performed by the control device 5. Note that the term "computer system" as used here includes hardware such as an OS and peripheral devices. It also includes a WWW system equipped with a homepage provision environment (or display environment). Furthermore, "computer-readable recording medium" refers to portable media such as flexible disks, optical magnetic disks, ROMs, and CD-ROMs, as well as storage devices such as hard disks built into computer systems. Furthermore, "computer-readable recording medium" also includes devices that retain a program for a certain period of time, such as volatile memory (RAM) within a computer system that acts as a server or client when a program is transmitted via a network such as the Internet or a communication line such as a telephone line.

The above program may also be transmitted from a computer system that stores the program in a storage device or the like to another computer system via a transmission medium, or by transmission waves within the transmission medium. Here, the "transmission medium" that transmits the program refers to a medium that has the function of transmitting information, such as a network (communication network) such as the Internet or a communication line (communication line) such as a telephone line. The above program may also be one that realizes part of the above-mentioned functions. Furthermore, it may be a so-called differential file (differential program) that can realize the above-mentioned functions in combination with a program already recorded on the computer system.

The above describes the form for carrying out the present invention using an embodiment, but the present invention is in no way limited to such an embodiment, and various modifications and substitutions can be made without departing from the spirit of the present invention.

1, 1A, 1B, 1C, 1D...Robot hand, 10...Robot, 101-105...Fingers, 111...Base, 131-134, 133a, 133b, 134a, 134b, 141-144...Cameras, 121...Arm, 201...Actuator, 202...Sensor, 203...Camera, 5...Control device, 501...Operation control unit, 502...Camera operation control unit, 503...Image processing unit, 504 Memory unit

Claims (6)

a base body including a position corresponding to a palm and to which the arm is connected ;
two or more fingers connected to the base , including a first finger close to the arm and a second finger close to a hand tip end, each having a joint, and configured to grip an object by bending the joint ;
a first camera disposed between the first finger portion and the second finger portion and on one side of the second finger portion at a position corresponding to a palm of a hand ;
a second camera installed on the other side of the second finger portion at a position corresponding to the palm of the hand ,
the first camera is attached inclined toward a gripping direction when the object is gripped by the finger portions;
Robot hand. The robot hand according to claim 1 , wherein the second camera is installed on the ball of the little finger. The robot hand according to claim 1 or 2 , wherein stereo vision is performed using the image captured by the first camera and the image captured by the second camera. The robot hand according to claim 3 , further comprising a third camera on the palm surface side of the robot hand. An operating method for operating the robot hand according to any one of claims 1 to 4 ,
Determine the target object;
determining an environmental target to manipulate the target object;
determining a gripping method for gripping the target object;
an image of the target object to be grasped and an environment in which the target object is to be manipulated is captured by the first camera and the second camera in a state where the target object is grasped with fingers;
acquiring position information of the target object based on the captured image;
determining an amount of manipulation of the target object based on the acquired position information;
Robot hand control method. the robot hand is a plurality of hands,
At least two of the plurality of robotic hands are equipped with cameras;
the first camera and the second camera provided in the at least two robot hands capture images of the same target object;
coordinate control of at least two robot hands using the captured images;
The robot hand control method according to claim 5 .