JPH0695849B2 - Guidance device for working machines for fruits and vegetables - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION [Industrial field of application] The present invention relates to a work hand for capturing a work target, an image pickup unit for picking up the work target, and a setting light amount for setting the work target when the image pickup unit picks up the image. Illumination means for illuminating with, an approach detection means for detecting that the work target has approached within a set distance, and a capture detection means for detecting that the work hand has captured the work target are provided. An image processing means for obtaining information on a direction in which the work target is located based on image pickup information of the image pickup means, and an image processing means for obtaining information on an image taken in a state in which the work hand is located at a preset initial position. The work hand is guided toward the direction in which the work target is positioned, and even after the approach detection means is detected, the work hand is held until the capture detection means is detected. A guide means for guiding the work hand by the set distance, and is started when the capture detection means does not operate even if the work hand is guided by the guide means, and the work hand with respect to the work target Provided are auxiliary guiding means for re-guiding from the position where the set distance is returned to the front side, and hand control means which is activated in response to the detection operation of the capture detection means and controls the capture operation of the working hand. The present invention relates to a guidance device for a work machine for fruit and vegetables.

[Conventional technology]

In the guidance device of the above-mentioned fruit and vegetables working machine, the work target is processed by performing image processing on the image information captured in a state where the work hand is positioned at the initial position to start the approach to the work target. Seeking the direction in which it is located,
The approaching hand is guided toward the obtained direction until the approach detection means performs the detection operation and after the approach detection means performs the detection operation until the capture detection means operates.

Therefore, if the work hand and the work target are separated from each other, there may be a case where an error occurs in the derived guiding direction and the work target cannot be properly captured. On the other hand, auxiliary guiding means for guiding the set distance from the position returned to the front side is provided.

However, conventionally, when the set distance is returned to the front side, the work hand is temporarily stopped and re-imaging is performed, and the re-imaging image information is subjected to image processing to re-detect the guiding direction with respect to the work target. After this, the auxiliary guide means was used to re-guide in the re-detected guide direction (see Japanese Patent Laid-Open No. 63-119612).

[Problems to be Solved by the Invention]

According to the above-mentioned conventional configuration, in order to re-guide by the auxiliary guiding means, it is necessary to temporarily stop the work hand, re-image, and perform image processing again. Therefore, until the guiding direction is detected again, the re-guidance of the work hand by the auxiliary guiding means cannot be started, and there is a disadvantage that the work efficiency is reduced.

The present invention has been made in view of the above circumstances, and an object thereof is to suppress a decrease in work efficiency when the auxiliary guiding means is activated.

[Means for Solving the Problems]

A guide device for a fruit and vegetable working machine according to the present invention is a work hand that captures a work target, and imaging means that images the work target, and illuminating means that illuminates the work target with a set light amount when the imaging means images. And an approach detection unit that detects that the work target has approached within a set distance, and a capture detection unit that detects that the work hand has captured the work target. An image processing unit that obtains information on a direction in which the work target is located based on image pickup information, and a work target that is obtained based on image information captured in a state where the work hand is located at a preset initial position. The work hand is guided toward the direction in which the work hand is positioned, and the work hand is set until the capture detection means detects and operates even after the approach detection means detects and operates. The guide means for guiding only the separation and the guide means are activated when the capture detection means does not operate even if the work hand is guided by the guide means, and the work hand is set to the set distance with respect to the work target. Provided are auxiliary guidance means for re-guidance from the position returned to the front side, and hand control means which is activated along with the detection operation of the capture detection means and controls the capture operation of the working hand. The characteristic configuration is as follows.

That is, when the guide means guides the work hand, the auxiliary guiding means is configured to obtain the direction in which the work target is located by performing re-imaging processing in advance at the time when the approach detecting means detects and operates. And
It is configured such that when it is activated, it is configured to redirect the direction toward the position of the work target, which is obtained in advance at the time when the approach detection unit performs the detection operation.

[Action]

That is, the work hand guided by the guiding means is
When the capture fails, the auxiliary guiding means will pass a position located on the near side with respect to the work target to be activated.

Therefore, when the work hand is being guided by the guiding means, if the direction in which the work target is located is obtained by re-imaging processing in advance at the time when the approach detecting means detects and operates, the auxiliary guiding means is activated. Thus, the work hand can be immediately re-guided from the position where the set distance is returned to the front side position with respect to the work target.

The image processing for re-detecting the guiding direction based on the re-imaged image information is performed by detecting the capture failure from the time when the approach detecting means detects and actuates the work hand with respect to the work target before the set distance. Since it may be completed while returning to the side, only the re-imaging processing may be performed at the time when the approach detection unit performs the detection operation, and the image processing may be performed only when the capture failure is detected.

〔The invention's effect〕

Therefore, it is not necessary to temporarily stop the work hand and perform the re-imaging process for the re-guidance by the auxiliary guidance means,
It has become possible to suppress a decrease in work efficiency.

〔Example〕

Hereinafter, an example when the present invention is applied to a guide device of a fruit harvester will be described with reference to the drawings.

As shown in FIG. 6, a boom (2) is attached to a vehicle body (V) provided with a pair of left and right traveling wheels (1) at the front and rear so as to be vertically driven and pivotally driven, and a tip portion of the boom (2) is An auxiliary boom (3) is swingably driven in the horizontal direction, and a working hand (H) for fruit harvesting.
A working manipulator (4) provided with is attached to the tip of the auxiliary boom (3).

In the figure, (5) is a boom lifting hydraulic cylinder, and (6)
Is a boom swing hydraulic cylinder, and (7) is an auxiliary boom swing hydraulic cylinder.

The work manipulator (4) includes an articulated telescopic arm (8) and a work hand (H) attached to the tip of the telescopic arm (8).

The telescopic arm (8) is swung around a vertical axis (Y) by a transmission motor (9a) and rocked around a horizontal axis (X) by an electric motor (9b). Further, the electric motor (9c) is configured to be expanded and contracted.

Explaining the working hand (H), as shown in FIG. 5, a vacuum pad (10) for adsorbing a fruit (F) as a work target and a vacuum pad (10) are attached to the tip end thereof. ) The catch portion case (12) for covering the fruit (F) adsorbed to the suction port (10a) and the handle portion of the fruit (F) taken in the catch portion case (12) are vertically sandwiched. Pattern cutting device (1
3) and are provided. The vacuum pad (1
0) is formed of a flexible member having elasticity such as rubber so as not to damage the fruit (F).

The trap case (12) is attached to the vacuum pad (1
Until the (0) adsorbs the fruit (F), it is retracted to the rear side of the hand so as not to interfere with the approach, and after the vacuum pad (10) adsorbs the fruit (F), the adsorbed fruit ( (F) so as to project to the front side of the hand so as to cover it, the ventilation pipe (1
It is externally fitted and supported so that it can be moved forward and backward with respect to 1).

The ventilation pipe (11) is a bellows type hose (14) (6th
(See the drawing), it is connected to the suction pump (17) mounted on the vehicle body (V) side by piping.

Further, the capturing part case (12) is an upper case (12) fixed to a support member (12a) fitted to the ventilation pipe (11).
b) and a lower case (12c) pivotally attached to the support member (12a) about a longitudinal axis so as to be swingable, and the lower case (12c) is connected to the ventilation pipe (12c). By swinging laterally with respect to 11), it is possible to discharge the fruit (F), whose handle is held in the trap case (12), from which the handle is cut, to the outside.

The handle cutting device (13) presses and supports the handle (13a) of a clipper type for cutting the handle of the fruit (F) and the handle (13a) of the handle of the fruit (F) from below. It comprises a handle support member (13b).

However, although not described in detail, the working hand (H) includes
Various pneumatic actuators for harvesting, such as an actuator for moving the catch case (12) in and out and an actuator for driving the handle cutting device (13), and fruits for the vacuum pad (10). A negative pressure sensor (S ₃ ) for detecting that (F) has been adsorbed (see FIG. 1), a sensor for detecting the retracted position of the trap case (12), and the handle cutting device (13). Various sensors such as a sensor for detecting the driving state will be provided.

That is, the negative pressure sensor (S ₃ ) corresponds to a capture detection unit that detects that the work hand (H) has captured the fruit (F) as the work target.

In the figure, (S ₁ ) is a color type image sensor as an image pickup means for picking up an image of the fruit (F) located in the direction in which the working hand (H) is facing, and (S ₂ ) is the working hand. A reflection type proximity sensor using infrared light as an approach detection means for detecting that (H) has approached a work target fruit (F) within a set distance, and both sensors (S ₁ ),
(S ₂ ) is provided inside the vacuum pad (10) so as not to disturb the approach and fruit harvest. Further, (15) is an illuminating device as illuminating means for illuminating the front side of the work hand (H) with a set light amount in synchronization with the image pickup process of the image sensor (S ₁ ), A so-called stroboscopic light emitting device is used for this.

Then, with the trap portion case (12) being withdrawn rearward and rearward so that the suction port (10a) of the vacuum pad (10) projects forward of the trap portion case (12), The hand (H) is positioned at the approach start position as a preset initial position, and the image sensor (S ₁ ) performs image pickup processing in the direction in which the fruit (F) to be harvested is positioned. At the same time, the approach direction as the guiding direction of the working hand (H) is determined based on the information obtained by image-processing the captured image information.

After determining the approach direction, each of the electric motors (9a), (9b), so that the working hand (H) moves linearly in the determined approach direction.
(9c) is actuated, and as the proximity sensor (S ₂ ) senses the fruit (F), the vacuum pad (10) is actuated to adsorb the fruit (F) to be harvested, and After the catching case (12) is projected and the fruit (F) is taken inside, the handle is cut and harvested.

However, when there are a plurality of fruits (F) within the imaging field of view of the image sensor (S ₁ ), which fruit (F) should be approached first is determined by the image sensor (S) as described later. It is automatically determined based on the brightness information of the captured image according to ₁ ).

By the way, the proximity sensor (S ₂ ) detects the approach to the fruit (F), and the value of the set distance is the suction operation of the vacuum pad (10) from the time when the proximity sensor (S ₂ ) detects and operates. The vacuum pad (10) is set so that its suction force becomes larger than the value required for fruit adsorption while the vacuum pad (10) moves the set distance. Further, the working hand (H) is moved at high speed until the proximity sensor (S ₂ ) detects and operates, and the working hand (H) moves after the proximity sensor (S ₂ ) detects and operates. The extension speed of the telescopic arm (8) is switched between high and low based on the detection information of the proximity sensor (S ₂ ) so as to move at a low speed. Then, as the negative pressure sensor (S ₃ ) is turned on to detect the fruit adsorption, the extension of the telescopic arm (8) is stopped and the harvesting operation is started.

As will be described in detail later, after the proximity sensor (S ₂ ) is activated, the working hand (H) is moved forward by a set distance at which the proximity sensor (S ₂ ) is activated. If the negative pressure sensor (S ₃ ) does not operate, it is determined that the harvest has failed, and the approach direction is corrected and re-approached from the position where the set distance is set back toward the front side.

Next, based on the image information captured by the image sensor (S ₁ ), the direction in which the fruit (F) is located, that is, the approach direction of the working hand (H) is obtained, and the working hand (H) is guided. A control configuration for doing so will be described.

As shown in FIG. 1, image processing means (100) for performing image processing on the imaged image information by the image sensor (S ₁ ) to obtain the direction in which the fruit (F) to be harvested is located, and its image processing means ( 100) based on the information of each hydraulic cylinder (5), (6), (7) and each electric motor (9a),
(9b) and (9c) are controlled to guide the working hand (H) from the approach start position toward the harvest target fruit (F) (100) and to move it when the harvest fails. Auxiliary guiding means (102) for guiding the working hand (H) again, and the negative pressure sensor (S) as capture detecting means.
₃ ) A hand control means (103) which is activated in accordance with the detection operation and controls the capturing operation of the working hand (H).
And a control device (16) using a microcomputer, which configures each of the above, is provided, and the control device (16) includes the sensors (S ₁ ), (S ₂ ), (S ₃ ), and the illumination device (15). , And the hydraulic cylinders (5), (6), (7) and the electric motors (9a), (9b), (9c) are connected.

Next, the operation of the control device (16) will be described with reference to the flowchart shown in FIG.
0), (101), (102), and (103) will be described in detail.

First, the work hand (H) is subjected to image pickup processing in a state where it is set at a preset approach start position, and the picked-up image information is subjected to image processing to obtain a specific color area corresponding to the color of the fruit (F). (F ₁ ) and its specific color area (F ₁ )
The highlight area (F ₂ ) in which the brightness is larger than the set value and the center of gravity (F ₃ ) of the highlight area (F ₂ ) are obtained (see FIG. 3).

Then, based on the size of the highlight region (F ₂ ), the work hand (H) is moved in a direction in which the center of gravity (F ₃ ) of the largest size is located. The approach direction of the working hand (H) is determined and the telescopic arm (8) is extended.

In addition, since the brightness of the image information becomes inversely proportional to the distance, the brightness becomes brighter. Therefore, the size of the highlight area (F ₂ ) becomes larger as the distance is closer. (See FIG. 4). That is, the work hand (H) is approached first from the easily harvestable fruit (F) located on the front side.

That is, the image information of the image sensor (S ₁ ) is image-processed to perform the specific color area (F ₁ ), the highlight area (F ₂ ), and the center of gravity (F ₃ ) of the highlight area (F ₂ ). And the process of determining the approach direction based on the size of the highlight region (F ₂ ) corresponds to the image processing means (100).

After starting the extension of the telescopic arm (8), based on the information of the proximity sensor (S ₂ ), the working hand (H) reaches the set distance front side with respect to the fruit (F) to be harvested. It is determined whether or not it has approached, and the extension speed of the telescopic arm (8) is switched to a low speed as it approaches within the set distance, and the suction operation of the vacuum pad (10) is started. The working hand (H) is moved to the fruit side until the negative pressure sensor (S ₃ ) is turned on.

When the negative pressure sensor (S ₃ ) is turned on and fruit adsorption is detected, the telescopic arm (8) is stopped and the harvesting operation is started.

That is, the process of extending the telescopic arm (8) until the negative pressure sensor (S ₃ ) is turned on so that the working hand (H) moves in the determined approach direction, It corresponds to the guiding means (101), and the process of stopping the telescopic arm (8) and performing the harvesting operation as the negative pressure sensor (S ₃ ) is turned on,
It corresponds to the hand guiding means (103).

However, if the proximity sensor (S ₂ ) does not operate even when the telescopic arm (8) is extended until the set distance set based on the maximum extension distance is reached, it is determined that harvesting is impossible and the work hand (H ) Is returned to the approach start position.

Further, even if the proximity sensor (S ₂ ) detects and operates, but the negative pressure sensor (S ₃ ) does not operate during the movement of the set distance, it is determined that the harvest has failed. However,
When the proximity sensor (S ₂ ) is activated, the negative pressure sensor (S ₃ )
Is not activated, the working hand (H) is once returned to the position where the proximity sensor (S ₂ ) is activated, and is approached again from that position.

In addition, the telescopic arm (8) is moved along with the operation of the proximity sensor (S ₂ ) when guiding in the approach direction determined from the approach start position.
Of the work hand (H) by re-imaging the image sensor (S ₁ ) without stopping the expansion of
Re-detects the approach direction in the state in which the target fruit (F) is located on the front side of the set distance.

Then, when said negative pressure sensor is determined that harvest failure based on the information of the (S ₃₎ is returned to the front side of the working hand (H) to a position where the proximity sensor (S ₂₎ is activated, and The guiding direction of the working hand (H) is corrected to the approach direction previously obtained during the first approach, and the approach is performed again.

Even in this re-imaging process, for example, when a plurality of fruits (F) are densely packed, a plurality of the specific color areas (F ₁ ) and the highlight areas (F ₂ ) are In some cases, the approach order is re-determined based on the size of the extracted highlight region (F ₂ ) as described above.

That is, if the harvest fails even if the proximity sensor (S ₂ ) operates, the process of correcting and re-guiding to the approach direction obtained based on the image information re-imaged during the first approach Corresponds to the auxiliary guiding means (102).

When the fruit (F) can be properly harvested by the first approach or the re-approach, the work hand is held in a state where the fruit (F) whose pattern has been cut is held in the capturing part case (12). The fruit (F) is discharged by returning (H) to the approach start position.

After discharging the harvested fruits (F), whether the work is completed based on the number of specific color areas (F ₁ ) and highlight areas (F ₂ ) extracted from the first captured image information at the approach start position. If it is not the end of the work, the work hand (H) is reset to the approach start position to perform the imaging process, and the approach direction and approach order for the remaining fruit (F) are determined. do it,
Each process for guiding the work hand (H) toward the determined approach direction is repeated.

The image processing for extracting the specific color region (F ₁ ) will be described. The three primary color signals (R), (G), among the image signals output from the image sensor (S ₁ ),
By using (B), the signal obtained by subtracting the other color signal (G or B) from the red signal (R) including the color component of the fruit (F) is binarized based on the set threshold value. A binarized image obtained by extracting a specific color region (F ₁ ) corresponding to the color of the fruit (F) from the original image (see FIG. 3A) taken by the image sensor (S ₁ ) ( FIG. 3B) is obtained.

Then, using the specific color region (F ₁ ) as a mask,
The brightness signal (Y) of the image signal output from the image sensor (S ₁ ) is binarized based on a set threshold value so that the brightness within the specific color area (F ₁ ) is set to a set value. The binarized highlight area (F ₂ ) which is larger than the above is extracted.

However, although not described in detail, at the time of re-imaging processing performed at a set distance at which the proximity sensor (S ₂ ) operates, the illumination amount of the illumination device (15) is set to an appropriate light amount according to the set distance. It will be operated in a dimmed state.

By the way, in order to eliminate the influence of natural light, the image sensor (S ₁ ) includes an electronic shutter (1
8) is attached so that the light is incident only during the set time during the imaging process.

Therefore, when the shutter (18) is in the closed state, since the light is not incident on the image sensor (S _1), be monitored using a captured image display device or the like of the image sensor (S ₁₎ There will be a disadvantage that it will not be possible.

Therefore, as shown in FIG. 7, the shutter (18) is constantly kept open so that the imaging field of view of the image sensor (S ₁ ) can be freely monitored. Then, at the time of the image capturing process, after the image input command is given, the shutter (18) is once closed based on the trigger signal output in synchronization with the end point of the captured image information for one screen, Open only for a set time and close again, and while the shutter (18) is closed, after reading one screen of image information into the control device (16),
It is designed to return to the normally open state.

Further, the illumination device (15) includes the image sensor (S ₁ )
The light is repeatedly emitted in synchronization with the image pickup processing of.
Therefore, when the light emission repeating period is shortened, the amount of heat generation is increased, which may reduce the durability.

Therefore, as shown in FIG. 8, the working hand (H)
The inside of the illuminator (15) is forcibly cooled by using the compressed air sent from the air source (19) for driving the actuator. In the figure, (20) is a pipe connecting the air source (19) and the illumination device (15), and (21) is an exhaust port for discharging compressed air from the illumination device (15) to the outside of the device. , (22) are light sources such as a xenon tube.

[Another embodiment]

In the above examples, the case where the present invention is applied to the guiding device of the fruit harvesting working machine is illustrated, but the present invention can be applied to various fruit and vegetable working machines, such as a vegetable harvesting machine for tomatoes and the like. Therefore, the specific configuration of each part such as the specific configuration of the working hand (H) can be changed in various ways.

It should be noted that reference numerals are added to the claims for convenience of comparison with the drawings, but the present invention is not limited to the structures of the accompanying drawings by the entry.

[Brief description of drawings]

The drawings show an embodiment of a guiding device for a work implement for fruit and vegetables according to the present invention. Fig. 1 is a block diagram of a control configuration, Fig. 2 is a flowchart of control operation, and Figs. 3 (a) to (c) are FIG. 4 is an explanatory diagram of image processing, FIG. 4 is an explanatory diagram of each region, FIG. 5 is a cutaway side view showing a schematic configuration of a working hand, FIG. 6 is a schematic side view of a working machine, and FIG. A timing chart, FIG. 8 is an explanatory diagram of a cooling structure of the illumination device. (F) …… Work target, (H) …… Work hand, (S ₁ )
...... Image pickup means, (S ₂ ...... approach detection means, (S ₃ ) ... capture detection means, (15) ...... illumination means, (100) …… image processing means, (101) …… guidance means, ( 102) ... Auxiliary guidance means, (103) ... hand control means.

Claims (1)

[Claims] To 1. A working hand to capture work object (F) (H), the imaging means (S ₁₎ for imaging the work object (F), the work time of imaging of the imaging means (S ₁₎ Illuminating means (15) for illuminating the object (F) with a set amount of light, approach detection means (S ₂ ) for detecting that the object (F) is approaching within a set distance to the work object (F), and the work hand ( H) is provided with capture detection means (S ₃ ) for detecting that the work target (F) has been captured, and the work target (F) is positioned based on the imaging information of the imaging means (S ₁ ). The image processing means (100) for obtaining the information on the direction and the position of the work target (F) obtained based on the image information taken in the state where the work hand (H) is located at the preset initial position are located. Orient the work hand (H)
Induced, and the approach detection unit (S ₂₎ induction means for inducing said working hand the (H) by the set distance to said capture detection means even after the detection operation (S ₃₎ detects actuation ( 101) and when the capture detection means (S ₃ ) does not detect even if the work hand (H) is guided by the guide means (101), the work hand (H) is activated. Auxiliary guiding means (102) for guiding the working object (F) from the position where the set distance is returned to the front side.
And a hand control means (103) which is activated in response to the detection operation of the capture detection means (S ₃ ) and controls the capture operation of the work hand (H). A guiding device, wherein the auxiliary guiding means (102) preliminarily detects when the approach detecting means (S ₂ ) is activated when guiding the work hand (H) by the guiding means (101). Re-imaging processing and the work target (F)
Of the work target (F), which is preliminarily obtained at the time point when the approach detection means (S ₂ ) performs a detection operation upon activation. A guidance device for a work implement for fruit and vegetables that is configured to redirect.