JP4387920B2 - Image recognition method and robot - Google Patents

Description

  The present invention relates to an image recognition method and a robot for performing image recognition using a robot including a camera and an image processing circuit.

  Image recognition by a conventional robot is realized by arranging a small CCD camera or the like on the robot, processing image information obtained at every sampling period, and comparing and comparing with image information registered in advance. (For example, refer to Patent Document 1, Patent Document 2, and Patent Document 3).

The applicant has not yet found prior art documents related to the present invention by the time of filing other than the prior art documents specified by the prior art document information described in this specification.
WO98 / 26598 Japanese Patent Laid-Open No. 11-136668 Japanese Patent Laid-Open No. 11-275559

However, in the technique of Patent Document 1, since a moving body is photographed using a stationary camera, for example, when the camera itself moves like a doll robot, image recognition of a subject cannot be performed correctly. There was a problem.
Further, in the technique of the cited document 2, frame data from which data of a moving subject is removed is created by obtaining a difference between the current frame data captured by the TV camera and the immediately preceding frame data. However, in the technique of the cited document 2, if a subject is photographed while the camera is moving, the image shift caused by the movement of the camera is considered to be caused by the movement of the subject, so that it is like a doll robot. However, when the camera itself moves, there is a problem in that image recognition of the subject cannot be performed correctly. Similarly, the technique of the cited document 3 also has a problem that when the camera itself moves like a doll robot, image recognition of the subject cannot be performed correctly.

  Usually, in order to perform image recognition correctly, the camera needs to be stationary in front of the subject to be recognized. The reason for this is pre-registered image data of the subject's face contour, eyes, nose, mouth, etc., and information about the subject's face contour, eyes, nose, mouth, etc. sent from the camera at every sampling cycle When the camera is moving with respect to the subject, the position of the subject's face outline, eyes, nose, mouth, etc. will differ depending on the sampled data, so comparison comparison cannot be performed. is there. In addition, when data at a certain moment is used, the probability of misrecognition increases because sample data for recognition is small. For this reason, in the technique of the cited document 2, after moving a camera, it stops and image | photographs with a camera. However, keeping the camera stationary with respect to the subject during image recognition limits the natural behavior of the robot. In addition, when the camera is temporarily stopped, it takes time until the camera is stopped. Therefore, there is a problem that it takes time until the subject is compared with the registered data after the subject is confirmed.

  The present invention has been made to solve the above-described problem, and an image recognition method capable of recognizing an image of a subject in a state that is not different from a stationary state even when a robot camera is moving, and The purpose is to provide a robot.

The present invention relates to an image recognition method for performing image recognition using a robot including a camera, an image processing circuit, and a storage device, a detection procedure for detecting the position and moving speed of the camera, and an image captured by the camera. Sampling procedure for capturing the image into the image processing circuit at regular intervals, a correction procedure for correcting the image captured from the camera by the image processing circuit based on the position and moving speed of the camera, and the corrected image An image recognition procedure for performing image recognition by comparing and collating with a recognition image registered in advance in the storage device, and the detection procedure includes a movement of a drive unit that moves the camera or the entire robot. obtain the position and the moving speed of the camera based on the output of the detection to the encoder, the correction procedure is taken when the camera is in position As a correction target image captured with the image captured by the image or a predetermined point in time but crowded from the camera is the same, in which so as to correct the image of the correction target.

The robot according to the present invention includes a camera that captures an image, a detection unit that detects the position and moving speed of the camera, a storage device that stores an image for recognition, and an image captured by the camera. An image captured every time, and the image captured from the camera is corrected based on the position and moving speed of the camera, and the corrected image is compared with a recognition image registered in advance in the storage device. An image processing circuit for performing recognition, and driving means for moving the camera or the entire robot , wherein the detecting means is an encoder for detecting movement of the driving means, and the position of the camera based on the output of the encoder. and it consists of a control circuit for obtaining the moving speed, the image processing circuit, the camera taken image or a predetermined time taken when in the predetermined position As a correction target image captured from an elaborate image the camera is the same, and corrects the image of the correction target.

  According to the present invention, even when the robot camera is moving, the position and moving speed of the camera is detected, and an image captured from the camera is corrected based on the position and moving speed of the camera. The image of the subject can be recognized in a state that is not different from the case where the image is being captured. As a result, in the present invention, the movement of the robot is not restricted by the recognition of the subject. The reason is usually that the camera needs to be stationary for the recognition of the subject. However, in the present invention, the subject can be recognized even when the camera is not stationary, so that the robot itself can behave naturally. Because. In the present invention, the subject can be recognized in a short time. The reason is usually that the camera needs to be stationary for the recognition of the subject, but in the present invention, it is not necessary for the camera to be stationary with respect to the subject, and the time until the operation of the camera or robot is stopped is reduced. Because it can.

  In the present invention, image recognition is possible regardless of whether the robot operation is accelerated or constant. This is because the position and moving speed of the camera are obtained based on the output of the encoder that detects the movement of the driving means.

[First Embodiment]
Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. FIG. 1 is a schematic configuration diagram of a robot according to a first embodiment of the present invention, and FIG. 2 is a cross-sectional view of the robot of FIG. The robot according to the present embodiment includes a head 1, a gear 2, a servo motor 3, an encoder 4, a control circuit 5, a camera 6, an image processing circuit 7, a storage device 8, and a body unit 9. Have The servo motor 3 constitutes drive means for rotating the camera 6, and the encoder 4 and the control circuit 5 constitute detection means for detecting the position and moving speed of the camera 6.

  A gear 2 is attached to the lower surface of the head 1 of the robot 1, and the gear 2 is pivotally supported by a body portion 9 of the robot. The robot head 1 is driven by a servo motor 3 through a gear 2 and rotates in a horizontal plane about the axis of the gear 2. The servo motor 3 is provided with an encoder 4 that detects its rotation, and pulse information output from the encoder 4 is transmitted to the control circuit 5. A camera 6 attached to the robot head 1 captures a front image, and an image signal output from the camera 6 is input to an image processing circuit 7. The control circuit 5, the image processing circuit 7, and the storage device 8 are connected so that they can exchange information with each other. The image processing circuit 7 compares and collates the image captured from the camera 6 with the recognition image registered in advance in the storage device 8.

  Next, the operation of the robot according to the present embodiment will be described with reference to the drawings. FIG. 3 is a plan view showing the operation of the robot. As shown in FIG. 3, the head 1 of the robot rotates at an angular velocity W in the direction of the arrow. The camera 6 has an angle of view of θ. A reference position at which the robot recognizes the subject 10 is A, and an image captured by the camera 6 attached to the robot head 1 at this time is shown in FIG. The image processing circuit 7 takes in an image signal from the camera 6 at a predetermined sampling period. If this sampling period is Δt seconds, the position of the camera 6 after Δt seconds from the time when the image signal is captured at the reference position A is as shown in FIG. FIG. 6 shows an image obtained by superposing the image captured at this time and the image captured at the reference position A, where B is the position after Δt seconds. In FIG. 6, 100A is an image captured at the reference position A, and 100B is an image captured at the position B. As can be seen from FIG. 6, since the camera 6 is moving with respect to the subject 10, an x shift occurs between the images 100A and 100B, and this cannot be used for accurate image recognition.

  Therefore, the control circuit 5 of the present embodiment calculates position information and speed information of the head 1 (camera 6) based on the pulse information output from the encoder 4. Based on the calculated position information and speed information, the image processing circuit 7 shifts the image in the horizontal direction so that the image captured by the camera 6 is the same position as the image captured at the predetermined reference position A. to correct. In this correction, the distance between the camera 6 and the subject 10 is measured by a distance measuring unit (not shown), and the image is captured at the reference position A from the measured distance, the position information and speed information of the camera 6, and the angle of view θ of the camera 6. What is necessary is just to calculate the deviation | shift amount with the obtained image. This correction can be realized by software processing by a computer or a hardware circuit.

  An image obtained by correcting the shift of the image captured at the position B is shown in FIG. In this case, since there is no deviation in the corrected image, it can be processed as if it were a still image. The image processing circuit 7 performs image recognition by comparing and comparing the corrected image data with the image data registered in advance in the storage device 8. The above processing is performed in real time every Δt seconds, and the subject 10 can be recognized while the head 1 of the robot is operating and in the same state as when it is stationary.

[Second Embodiment]
Next, another embodiment of the present invention will be described in detail with reference to the drawings. FIG. 8 is a sectional view of a robot according to the second embodiment of the present invention. In the first embodiment, the head of the robot is rotated in the horizontal direction, whereas in the present embodiment, the head of the robot is rotated in the vertical direction. The robot according to the present embodiment includes a head 11, a servo motor 13, an encoder 14, a control circuit 15, a camera 16, an image processing circuit 17, a storage device 18, a body unit 19, and a head frame 20. And a center frame 21. The servo motor 13 constitutes drive means for rotating the camera 16, and the encoder 14 and the control circuit 15 constitute detection means for detecting the position and moving speed of the camera 16.

  The center frame 21 is fixed to the upper surface of the body part 19 of the robot. A head frame 20 fixed to the head 11 of the robot is pivotally supported by a center frame 21 so as to be rotatable. The servo motor 13 meshes with a gear 22 provided on the head frame 20 and serves as a power source for rotating the head 11 of the robot. As a result, the head 11 (head frame 20) of the robot is driven by the servo motor 13 and rotates in the vertical plane about the shaft 23. An encoder 14 for detecting the rotation is attached to the servo motor 13, and pulse information output from the encoder 14 is transmitted to the control circuit 15. The camera 16 attached to the head frame 20 captures an image in front of the camera, and an image signal output from the camera 16 is input to the image processing circuit 17. The image processing circuit 17 compares and collates an image captured from the camera 16 with a recognition image registered in advance in the storage device 18.

  Next, the operation of the robot according to the present embodiment will be described with reference to the drawings. FIG. 9 is a side view showing the operation of the robot. As shown in FIG. 9, the head 11 of the robot rotates at the angular velocity W in the direction of the arrow. The camera 16 has an angle of view of θ. A reference position at which the robot recognizes the subject 10 is A. The image processing circuit 17 captures an image signal from the camera 16 at a predetermined sampling period. Let this sampling period be Δt seconds, and let B be the position of the camera 16 after Δt seconds from the time when the image signal was captured at the reference position A. An image obtained by superimposing the image captured at the reference position A and the image captured at the position B is shown in FIG. In FIG. 10, 101A is an image captured at the reference position A, and 101B is an image captured at the position B. As can be seen from FIG. 10, since the camera 16 is moving with respect to the subject 10, an x shift occurs between the images 101 </ b> A and 101 </ b> B and cannot be used for accurate image recognition as it is.

  Therefore, the control circuit 15 of the present embodiment calculates position information and speed information of the head 11 (camera 16) based on the pulse information output from the encoder 14. Based on the calculated position information and speed information, the image processing circuit 17 shifts the image in the vertical direction so that the image captured by the camera 16 is the same position as the image captured at the predetermined reference position A. to correct. In this correction, the distance between the camera 16 and the subject 10 is measured by a distance measuring unit (not shown), and an image is taken at the reference position A from the measured distance, the position information and speed information of the camera 16, and the angle of view θ of the camera 16. What is necessary is just to calculate the deviation | shift amount with the obtained image. This correction can be realized by software processing by a computer or a hardware circuit.

  Thereby, since there is no deviation in the corrected image, it can be processed as a still image. The image processing circuit 17 performs image recognition by comparing and comparing the corrected image data with image data registered in advance in the storage device 18. The above processing is performed in real time every Δt seconds, and the subject 10 can be recognized while the head 1 of the robot is operating and in the same state as when it is stationary.

[Third Embodiment]
Next, a third embodiment of the present invention will be described in detail with reference to the drawings. FIG. 11 is a sectional view of a robot according to the third embodiment of the present invention. In the first and second embodiments, the head of the robot is rotated in the horizontal direction or the vertical direction, whereas in the present embodiment, the robot is caused to self-run. The robot according to the present embodiment includes a head 31, a servo motor 33, an encoder 34, a control circuit 35, a camera 36, an image processing circuit 37, a storage device 38, a body part 39, a tire 40, 41 and a gear 42. The servo motor 33 constitutes drive means for running the robot, and the encoder 34 and the control circuit 35 constitute detection means for detecting the position and moving speed of the camera 36.

  The robot head 31 is fixed to the body 39. The robot body 39 is provided with a tire 40 and a small-diameter tire 41 that can turn. A gear 42 is attached to the large-diameter tire 40. The servo motor 33 meshes with the gear 42 and becomes a power source for running the robot. Thereby, the rotation of the servo motor 33 is transmitted to the tire 40 and the robot travels. The servo motor 33 is provided with an encoder 34 that detects its rotation, and pulse information output from the encoder 34 is transmitted to the control circuit 35. A camera 36 attached to the robot head 31 captures a front image, and an image signal output from the camera 36 is input to an image processing circuit 37. The image processing circuit 37 compares and collates an image captured from the camera 36 with a recognition image registered in advance in the storage device 38.

  Next, the operation of the robot according to the present embodiment will be described with reference to the drawings. The robot is moving in the direction of the arrow in FIG. 11 at a constant speed. Let A be the time when the robot starts image recognition. The image processing circuit 37 takes in an image signal from the camera 36 at every predetermined sampling period. Let this sampling period be Δt seconds, and let B be the time after Δt seconds from the time when the image signal was captured at the start time A. An image obtained by superimposing the image captured at the start time point A and the image captured at the time point B is shown in FIG. In FIG. 12, 102A is an image captured at the start time A, and 102B is an image captured at the time B. As can be seen from FIG. 12, since the camera 36 is moved with respect to the subject 10, a shift of x occurs between the images 102 </ b> A and 102 </ b> B and cannot be used for accurate image recognition as it is.

  Therefore, the control circuit 35 of the present embodiment calculates position information and speed information of the head 31 (camera 36) based on the pulse information output from the encoder 34. Based on the calculated position information and speed information, the image processing circuit 37 corrects the size of the image so that the image captured by the camera 36 has the same size as the image captured at the start time A. In this correction, the distance between the camera 36 and the subject 10 is measured by a distance measuring unit (not shown), and the image is picked up at the start time A from the measured distance, the position information and speed information of the camera 36, and the angle of view θ of the camera 36. It is only necessary to calculate the difference in size from the image and enlarge or reduce the image. This correction can be realized by software processing by a computer or a hardware circuit.

  Thereby, since there is no deviation in the corrected image, it can be processed as a still image. The image processing circuit 37 performs image recognition by comparing and comparing the corrected image data with image data registered in advance in the storage device 38. The above processing is performed in real time every Δt seconds, and the subject 10 can be recognized while the robot is running and in the same state as when the robot is stationary.

  In the first to third embodiments, the three types of movements of horizontal rotation, vertical rotation, and robot traveling of the robot head are handled separately. However, if these movements occur simultaneously, combining the corrections described in the first to third embodiments can provide the same effect even if three types of movements occur simultaneously. Needless to say. Further, an image captured by a normal camera is distorted by a so-called curvature at the edge of the angle of view, and it is more effective to add this distortion correction function to the image processing circuit in advance.

  Further, in the first to third embodiments, the example of feeding back the information of the encoder attached to the servo motor is taken as an example. However, the slit member, the photo sensor, etc. are provided on the final rotating body such as a tire. It goes without saying that an equivalent configuration can be realized by providing a combination with a potentiometer. Furthermore, if a stepping motor or the like is used, the input information can be directly used for image processing. However, in this case, if a delay in the transmission system or a motor step-out occurs between the input and the final output, a sufficient effect cannot be expected.

In the first to third embodiments, image recognition has been mainly listed, but it is obvious that the present invention can be used for data registration of a subject.
In the first to third embodiments, the control circuit, the image processing circuit, and the storage device are described separately for the sake of explanation. However, these are configured by circuits or chips on the same substrate. Needless to say, each signal route may be different.

  The present invention can be applied to image recognition by a robot.

It is a schematic block diagram of the robot used as the 1st Embodiment of this invention. It is sectional drawing of the robot used as the 1st Embodiment of this invention. It is a top view which shows operation | movement of the robot of the 1st Embodiment of this invention. It is a figure which shows the image which the camera of the robot imaged in the reference position in the 1st Embodiment of this invention. It is a top view which shows the position of the camera after a sampling period from the reference | standard position in the 1st Embodiment of this invention. It is a figure which shows the image which overlap | superposed the image taken in at the reference | standard position and the image taken in after the sampling period in the 1st Embodiment of this invention. It is a figure which shows the image after correction | amendment in the 1st Embodiment of this invention. It is sectional drawing of the robot used as the 2nd Embodiment of this invention. It is a side view which shows operation | movement of the robot of the 2nd Embodiment of this invention. It is a figure which shows the image which overlap | superposed the image taken in at the reference | standard position and the image taken in after the sampling period in the 2nd Embodiment of this invention. It is sectional drawing of the robot used as the 3rd Embodiment of this invention. It is a figure which shows the image which overlap | superposed the image taken in at the start time in the 3rd Embodiment of this invention with the image taken in after the sampling period.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1, 11, 31 ... Head 2, 22, 42 ... Gear 3, 13, 33 ... Servo motor 4, 14, 34 ... Encoder 5, 15, 35 ... Control circuit 6, 16, 36 ... Camera 7, 17, 37 ... image processing circuit, 8, 18, 38 ... storage device, 9, 19, 39 ... body part, 10 ... subject, 20 ... head frame, 21 ... center frame, 40, 41 ... tire.

Claims (2)

In an image recognition method for performing image recognition using a robot having a camera, an image processing circuit, and a storage device,
A detection procedure for detecting the position and moving speed of the camera;
Sampling procedures for capturing images captured by the camera into the image processing circuit at regular intervals;
A correction procedure for correcting an image captured from the camera by the image processing circuit based on the position and moving speed of the camera;
An image recognition procedure for performing image recognition by comparing and comparing the corrected image and a recognition image registered in advance in the storage device;
The detection procedure obtains the position and moving speed of the camera based on the output of an encoder that detects the movement of the driving means for moving the camera or the entire robot,
In the correction procedure, the image to be corrected is acquired so that an image captured when the camera is at a predetermined position or an image captured at a predetermined time is the same as an image to be corrected captured from the camera. An image recognition method comprising correcting the image. A camera for taking images,
Detecting means for detecting the position and moving speed of the camera;
A storage device for storing an image for recognition;
Images captured by the camera are captured at regular intervals, the image captured from the camera is corrected based on the position and movement speed of the camera, and the corrected image and recognition registered in advance in the storage device An image processing circuit for performing image recognition by comparing and comparing images for use, and
Driving means for moving the camera or the entire robot,
The detection means comprises an encoder that detects the movement of the drive means, and a control circuit that obtains the position and moving speed of the camera based on the output of the encoder,
The image processing circuit is configured to correct the image to be corrected so that an image captured when the camera is at a predetermined position or an image captured at a predetermined time is the same as an image to be corrected captured from the camera. A robot characterized by correcting.

Images