JP7587503B2 - Harvester and its automatic operation method - Google Patents

Description

The present invention relates to the field of automatic operation of agricultural machinery, and in particular to a harvester and its automatic operation method.

Harvesters are crop harvesting machines that harvest the berries and straw of crops such as rice and wheat, and also include mowers and machinery and equipment for harvesting other crops such as vegetables and fruit. Grain harvesters are integrated crop harvesting machines that can cut and thresh crops in one operation, collect the grain in a storage room, and then transport it to a carrier vehicle via a conveyor. Fruit and vegetable harvesting machines can cut and harvest vegetables and fruit in a field, separate the fruit from the stems, and then sort it in one operation.

When performing harvesting work, the harvester needs to constantly observe the crops in the field and the field work area so as to adjust the work parameters of the harvester according to the height, maturity, fullness, etc. of the crops. When agricultural machinery works in the field, it is necessary to judge the work status of the field and the growth status of the crops in the field in real time, operate the operation of the agricultural machinery, and adjust the complex operation status of the work system. Currently, only machine operators with specialized skills can do this. When agricultural machinery works in the field, it is necessary to take into account many factors such as the worked area, unworked area, and boundary range of the field, and adjust the operation of the vehicle in real time according to the status of the crops during work and adjust the work parameters. Since it is necessary to take into account the complex work environment during operation, the conventional agricultural machinery requires the operator to adjust the operation of the agricultural machinery according to real-time information on the crops on the field. When the work of the agricultural machinery is controlled by manual operation, there is a high probability of judgment errors occurring and a high probability of the machinery breaking down during work.

This conventional harvester has at least one of the following defects. First, during operation, the harvester body shakes up and down due to the vibration of the harvester body itself and the unevenness of the farmland, so the camera device installed on the harvester body cannot capture images in a stable position. Therefore, the images captured by the imaging device are blurred and cannot provide information support for intelligent work and automatic driving. In addition, the imaging device of the conventional technology is fixedly attached to the harvester body, and can only capture images in a single direction, for example, an image in front of the harvester, and cannot adjust the shooting direction and position of the imaging device according to the situation. In addition, the mobile imaging device or fixed imaging device of the conventional technology, for example, an imaging device of a drone or an imaging device fixed to farmland, captures images around the harvester and then transmits them to the harvester body, and the harvester body reads the images captured by the imaging device. Although the problem of poor image capture has been resolved to some extent, images are captured based on the position of the imaging device itself or the drone, and images cannot be captured from the perspective of the harvester itself. Therefore, the captured images are not easily recognizable.

In the agricultural machinery of the prior art, the inaccuracy of the set working path usually causes errors in the work, which leads to machine failure. In addition, the satellite positioning using PTK has high performance requirements for the agricultural machinery, and the manufacturing and maintenance costs are relatively high, so such a conventional automatic driving positioning method is not applicable to the automatic driving mode of the conventional agricultural machinery.
<Prior Art Literature>
<Patent Documents>
<Patent Document 1> JP 2018-101860 A
<Patent Document 2> JP 2014-171457 A

One main advantage of the present invention is to provide a harvester and method for automatic operation thereof, in which the harvester recognizes an area of farmland in at least one captured visual image based on the image.

Another advantage of the present invention is to provide a harvester and method for automatic operation thereof, in which the harvester recognizes crop information, such as type, height, and maturity of the crop in at least one captured visual image based on the captured visual image.

Another advantage of the present invention is to provide a harvester and an automatic driving method thereof, in which the harvester recognizes unworked areas, worked areas, and farmland boundary areas in the visual image based on the visual image, and controls the travel path of the harvester based on the recognized areas.

Another advantage of the present invention is to provide a harvester and an automatic operation method thereof, in which the harvester recognizes crop information in the visual image based on the image, and adjusts the operation parameters of the harvester based on the recognized information in the image, thereby improving the operation quality and efficiency of the harvester.

Another advantage of the present invention is to provide a harvester and an automatic operation method thereof, in which the image acquisition device of the harvester is a camera-head imaging device, and the camera-head imaging device has an anti-shake imaging function, thereby improving the accuracy and stability of the harvester's visual image acquisition.

Another advantage of the present invention is to provide a harvester and an automatic operation method thereof, in which the image capture device is provided on the harvester body of the harvester, and the harvester captures images of the surroundings of the harvester body using the image capture device.

Another advantage of the present invention is to provide a harvester and an automatic operation method thereof, in which the image acquisition device is provided on the harvester body of the harvester, and the image acquisition device is provided on the harvester body, and captures at least one visual image or visual video based on the field of view position of the harvester body, thereby recognizing information about the surroundings of the harvester body based on the captured video information.

Another advantage of the present invention is to provide a harvester and an automatic operation method thereof, in which the image capture device captures images at different angles and in different directions while being adjusted based on the position of the harvester body, thereby capturing images of the harvester body in different directions.

Another advantage of the present invention is to provide a harvester and an automatic operation method thereof, wherein the image capture device is a mechanical or electronic camera, and the image capture device enhances the stability of the visual image.

Another advantage of the present invention is to provide a harvester and an automatic driving method thereof, in which a route planning system of the harvester automatically plans a route based on current vehicle positioning information, information recognized by an image processing system, and information from a navigation system.

Another advantage of the present invention is to provide a harvester and an automated operation method thereof, in which the harvester plans a travel path and a work path for the harvester based on the area recognized by the visual image.

Another advantage of the present invention is to provide a harvester and method for automated operation thereof, in which an image capture device of the harvester captures the visual images of the surrounding farmland in real time and updates the route navigation information of the harvester plan in real time.

Another advantage of the present invention is to provide a harvester and an automatic operation method thereof, in which the harvester captures an image in real time using the image capture device, recognizes the area in the visual image, and updates or adjusts the work path of the harvester in real time in response to changes in the area, thereby improving the work quality of the harvester.

Another advantage of the present invention is to provide a harvester and an automatic operation method thereof, in which the image processing system of the harvester recognizes the unworked area, the worked area, and the farmland boundary area in the image using image segmentation techniques based on the acquired visual image information, and segments the boundary between two adjacent areas.

Another advantage of the present invention is to provide a harvester and an automatic operation method thereof, in which the image processing system of the harvester uses image segmentation technology to recognize plant information such as the type, height, and fruit fullness of the crop in the image based on the acquired visual image information, and the operation system of the harvester adjusts operation parameters based on the crop information.

Another advantage of the present invention is to provide a harvester and an automatic driving method thereof, in which the image processing system of the harvester recognizes area boundaries in the image based on acquired image information, and the route planning system plans a vehicle driving route based on the recognized area boundaries.

Another advantage of the present invention is that it provides a harvester and an automatic operation method thereof, which does not require highly accurate satellite positioning for the harvester, reducing the difficulty of producing and manufacturing an automatic operation machine and reducing the maintenance costs of the machine.

Another advantage of the present invention is that it provides a harvester and an automatic driving method thereof, in which the harvester realizes automatic driving and automatic driving operations by performing route planning based on the area segmentation information output by the image processing system.

Other advantages and features of the present invention are set forth fully in the following detailed description and are realized by the combination of means and apparatus particularly pointed out in the appended claims.

According to one aspect of the present invention, the above objects and other objects and advantages are attained by a harvester of the present invention, comprising:
A harvester body;
At least one image capture device provided on the harvester body for capturing images of the surroundings of the harvester body;
an image processing system that recognizes farmland information in the video based on the image captured by the image capture device;
The harvester body automatically controls its operation based on the farmland information recognized by the image processing system.

According to one embodiment of the present invention, the harvester further includes a route planning system that plans at least one travel plan route based on the farmland information recognized by the image processing system, and the harvester body controls operation according to the travel plan route planned by the route planning system.

According to one embodiment of the present invention, the image processing system recognizes farmland information in the image using image segmentation recognition technology, and plans the farmland area in the image based on the recognized information.

According to one embodiment of the present invention, the image processing system uses image segmentation recognition technology to recognize crop information in the image, and the harvester body automatically adjusts operation parameters based on the recognized information.

According to one embodiment of the present invention, the image capture device is a shake-proof camera head imaging device that is mounted on the harvester body and photographs an image of the surroundings of the harvester body based on the position of the harvester body.

According to one embodiment of the present invention, the image capture device is a mechanical shake-proof head device, and includes a head and at least one video camera, the video camera is attached to the harvester body by the head, and the video camera is mounted on the head and supported via the head so as to maintain a balanced state.

According to one embodiment of the present invention, the image capture device is an electronic camera head device that controls the viewpoint and zoom of the lens, thereby preventing camera shake of the lens of the image capture device.

According to one embodiment of the present invention, the image capture device is provided at the front of the harvester body, at the top of the harvester body, on the left side of the harvester body, on the right side of the harvester body, or at the rear of the harvester body.

According to one embodiment of the present invention, the image processing system comprises:
an image segmentation module for segmenting the image into a plurality of pixel regions, each pixel region including at least one pixel unit;
a characterization module for extracting features corresponding to each of the pixel regions based on the pixel units of the pixel regions;
and a region partitioning module for recognizing and partitioning regions of the image based on characteristics of the pixel regions.

According to one embodiment of the present invention, the harvester further includes a positioning device and a navigation system, the positioning device and the navigation system are provided in the harvester body, the positioning device acquires position information of the harvester body, and the navigation system provides navigation information to the grain processing body.

According to one embodiment of the present invention, the route planning system comprises:
a working area setting module for setting a boundary area of the farmland to obtain a working area and the working boundary of the farmland;
and a driving route planning module, wherein the image processing system recognizes area planning information of the image and navigation information of the navigation system based on positioning information of the harvester body, to obtain at least one driving planned route.

According to one embodiment of the present invention, the harvester body includes a vehicle body, at least one work system provided on the vehicle body, and an operation control system, the vehicle body drives and operates the work system, and the operation control system controls the operation of the vehicle body and the work parameters of the work system.

According to one embodiment of the present invention, the driving control system acquires information on images recognized by the image processing system and captured by the image capture device, automatically controls the travel path of the vehicle body, and controls the work parameters of the work system, thereby achieving unmanned automatic driving and mowing work.

According to another aspect of the present invention, there is further provided an automatic operation method for a harvester, the automatic operation method comprising:
(a) acquiring at least one image and recognizing farmland areas and farmland boundaries in said image;
(b) planning at least one travel plan route based on the recognition information;
and (c) controlling the harvester body to automatically travel along the travel planned route.

According to one embodiment of the present invention, step (a) of the autonomous driving method further includes a step of recognizing information corresponding to the crops in the farmland in the image, the information on the crops including the type of crop, the height of the crop, the fullness of the fruit, etc.

According to one embodiment of the present invention, the autonomous driving method step (b) comprises:
(b.1) recognizing and demarcating areas and boundaries of said image corresponding to agricultural fields;
and (b.2) planning at least one driving plan route based on the recognized area.

According to one embodiment of the present invention, step (b.1) of the autonomous driving method further includes a step of segmenting the image using an image segmentation technique and recognizing and partitioning regions of the image.

According to one embodiment of the present invention, in step (b) of the autonomous driving method, the image processing system uses an image segmentation technique to segment the image information and recognize and divide the image area into the unworked area, the worked area, and the farmland boundary area.

According to one embodiment of the present invention, step (b.1) of the autonomous driving method comprises:
Dividing the image into a plurality of said pixel regions and normalizing pixel values of said pixel regions into an array;
extracting features of the pixel regions corresponding to each of the arrays;
and outputting a classification label for the image based on features corresponding to the pixel regions.

According to one embodiment of the present invention, step (b.2) of the autonomous driving method further includes a step of planning the driving plan route based on positioning information of the harvester body, area planning information of the image, and navigation information of the navigation system.

According to one embodiment of the present invention, the autonomous driving method further includes a step (b.3) of comparing whether the area division and area boundary range recognized by the image processing system match the previous area boundary range, adjusting the area division and area boundary range corresponding to the image if they do not match, and maintaining the area division and boundary range unchanged if they match.

According to one embodiment of the present invention, the automatic driving method further includes a step (d) of adjusting the operation parameters of the operation system of the harvester body based on the image recognition information.

Further objects and advantages of the present invention will become apparent through a consideration of the following description and drawings.

These and other objects, features and advantages of the present invention will be fully explained through the following detailed description, drawings and claims.

FIG. 1 is a schematic diagram of a harvester system according to a first preferred embodiment of the present invention; FIG. 2 is a schematic diagram of the harvester acquiring images according to the preferred embodiment of the present invention; FIG. 2 is a schematic diagram of one image captured by the harvester according to the preferred embodiment of the present invention; FIG. 13 is a schematic diagram of another image captured by the harvester according to the preferred embodiment of the present invention. FIG. 13 is a schematic diagram of another image captured by the harvester according to the preferred embodiment of the present invention. FIG. 2 is a schematic diagram showing how the image processing system of the harvester according to the preferred embodiment of the present invention divides and recognizes the image area. FIG. 2 is a schematic diagram of the image processing system of the harvester according to the preferred embodiment of the present invention dividing the image region; FIG. 2 is a block diagram of the image processing system of the harvester according to the preferred embodiment of the present invention. FIG. 2 is a schematic diagram of the image processing system of the harvester according to the preferred embodiment of the present invention extracting and recognizing the image region features. FIG. 4 is a schematic diagram of region segmentation of the image output by the image processing system of the harvester according to the preferred embodiment of the present invention. FIG. 13 is a schematic diagram of the division changes of boundary lines of the region division of the image output by the image processing system of the harvester according to the preferred embodiment of the present invention. FIG. 2 is a schematic diagram of an automatic operation scene of the harvester according to the preferred embodiment of the present invention. FIG. 2 is a schematic diagram of the harvester path planning system according to the preferred embodiment of the present invention; FIG. 2 is a schematic diagram of a farmland path plan generated by the path planning system of the harvester according to the preferred embodiment of the present invention; FIG. 2 is a schematic diagram of the route planning system of the harvester adjusting a travel route according to the preferred embodiment of the present invention; FIG. 2 is a schematic diagram of the overall structure of a harvester with an image capture device according to a second preferred embodiment of the present invention; FIG. 2 is a schematic diagram of the image capture device of the harvester according to the preferred embodiment of the present invention taking images. FIG. 2 is a structural schematic diagram of the image capture device of the harvester according to the preferred embodiment of the present invention, the image capture device being implemented as a mechanical pan head device. FIG. 2 is a schematic diagram of the mounting position of the image capture device of the harvester according to the preferred embodiment of the present invention. FIG. 2 is a schematic diagram of the image processing system of the harvester according to the preferred embodiment of the present invention recognizing farmland areas in images captured by the image capture device; FIG. 2 is a schematic diagram of the image processing system of the harvester according to the preferred embodiment of the present invention recognizing agricultural crops in images captured by the image capture device; FIG. 2 is a schematic diagram of another optional embodiment of the image capture device of the harvester according to the preferred embodiment of the present invention, wherein the image capture device is implemented as an electronic camera head device.

The following description is used to disclose the present invention so that those skilled in the art can practice the present invention. The preferred embodiments in the following description are merely examples, and other obvious variations may occur to those skilled in the art. The basic principles of the present invention defined in the following description may be applied to other embodiments, modifications, improvements, equivalents, and other technical solutions that do not deviate from the spirit and scope of the present invention.

Those skilled in the art will understand that in the description of the present invention, the orientations or positional relationships indicated by terms such as "longitudinal," "lateral," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inside," "outside," and the like are based on the orientations or positional relationships shown in the drawings, are merely for the purpose of simplifying the description and explanation of the present invention, and do not indicate or suggest that the devices or elements shown have a particular orientation or are configured or operated in a particular orientation, and therefore the above terms should not be understood as limitations on the present invention.

The term "one" should be understood as "at least one" or "one or more." That is, in one embodiment, the number of elements may be one, and in another embodiment, the number of elements may be multiple, and the term "one" should not be understood as a numerical limitation.

Referring to Figures 1 to 9 of the accompanying drawings of the present specification, a harvester and its automatic operation method according to a first preferred embodiment of the present invention are disclosed and described in the following description, and the harvester may be implemented as a crop harvester machine with grain processing function, a vegetable, fruit harvesting machine, a grass cutting machine, and other types of cutting devices. It can be understood that the types of harvesters described in the present invention are merely exemplary and not limiting. The harvester obtains at least one image of the surroundings, visually recognizes and processes farmland area types in the image, and segments the types and boundaries of various areas of the farmland in the image.

The area types of the farmland divided by the harvester include at least one worked area 100, at least one unworked area 200, and at least one farmland boundary area 300, and the navigation system plans the vehicle's driving route based on the area types divided by the harvester, thereby realizing unmanned automatic driving and unmanned automatic driving operations.

In addition, in order for an autonomous vehicle to recognize and acquire the vehicle's driving route in autonomous driving mode, it is necessary to acquire accurate vehicle positioning information, which usually requires highly accurate satellite positioning information. In order for the autonomous vehicle to realize the autonomous driving function in a high-speed driving state, it is necessary to constantly update information such as obstacle information on the road, road vehicle information, and road pedestrians. The image acquired by the harvester of the present invention is image data information corresponding to the agricultural crop grains in the farmland, and the image is an image of the surroundings of the vehicle acquired based on the current position of the vehicle. The harvester does not require ultra-high accuracy satellite positioning information, and only general satellite positioning with meter-level accuracy (such as GPS positioning or Beidou positioning) is required. Therefore, since the image acquired and processed by the harvester is different from that of an autonomous vehicle, the route plan and driving mode formed by the harvester are also not the same. It can be understood that the function of the harvester of the present invention to visually recognize the area of the farmland and the autonomous driving function are different from the recognition mode of an autonomous vehicle.

As shown in Figures 1 and 2, the harvester acquires at least one image of the surroundings, and based on the acquired image, recognizes and divides the area type and the boundary between the areas corresponding to the farmland. The harvester acquires the image of the surroundings of the harvester by taking fixed point photographs, video recording, photographs on the move, etc. It can be understood that the manner in which the harvester acquires the image is merely exemplary and not limiting. The harvester includes a harvester body 10 and at least one image acquisition device 20, and the image acquisition device 20 acquires at least one image of the surroundings of the harvester body 10.

Preferably, the image acquisition device 20 is provided on the harvester body 10 and acquires the image of the periphery of the harvester body 10 by taking a photograph or a video. More preferably, the image acquisition device 20 is provided in front of the harvester body 10 and can acquire an image in front of the harvester body 10 in real time, and the harvester body 10 sets a travel route based on an area recognized and divided from the image information captured by the image acquisition device 20. Note that the image captured by the image acquisition device 20 is an image within the field of view of the harvester body 10. In other words, the image acquisition device 20 acquires an image in the field of view of the harvester body 10 and adjusts the travel direction of the harvester body 10 according to the mounting position of the harvester body 10.

The image capture device 20 also captures a view of the top and bottom in the direction of travel of the harvester body 10, and the image may be a captured two-dimensional planar image or a three-dimensional stereoscopic image. It can be understood that the types of images captured by the image capture device 20 are merely exemplary and not limiting.

In the first preferred embodiment of the present invention, the harvester body 10 is implemented as a grain harvester machine, and while being controlled, travels to an unworked area 200 of farmland to perform harvesting work, and harvests crops such as rice, wheat, and corn in the unworked area 200. The harvester body 10 automatically drives in the field based on an area segmented from an image acquired by the image acquisition device 20, and performs unmanned automatic driving. It can be understood that the type of the harvester body 10 is merely illustrative and not limiting.

As shown in Figures 3A to 3C, the image acquisition device 20 acquires images of the surroundings of the harvester body 10 in real time while the harvester body 10 is traveling. Figure 3A shows the image captured by the image acquisition device 20 when the harvester body 10 is a grain harvester. Depending on whether the grain has been harvested or not, the area within the farmland is divided into at least one uncut area 100a, at least one cut area 200a, and at least one farmland boundary area 300a, and the cut area 200a is an area where crops have been cut, and the original crops within the cut area 200a have been cut. The uncut area 100a is an area where crops are alive, and crops are growing in the uncut area 100a. The farmland boundary area 300a is the ridges that separate the crops within the farmland, the outer boundary around the farmland, and obstacle areas that exist within the farmland, and no crops are planted in the farmland boundary area 300a.

Figure 3B shows the image captured by the image capture device 20 when the harvester body 10 is a grass cutting machine. Depending on whether the grain is planted or not, the area within the farmland is divided into at least one uncut area 100b, at least one cut area 200b, and at least one farmland boundary area 300b, where the uncut area 100b represents an area where the crops have not been cut, the cut area 200b represents an area where the crops are planted, and the farmland boundary 300b represents the ridges separating the planted crops, the outer boundary around the farmland, and the obstacle area present within the farmland.

Figure 3C shows the image captured by the image capture device 20 when the harvester body 10 is a harvesting machine for root-stalk plants or fruits, for example, a vegetable harvester machine. Depending on whether the crops have been sprayed with a chemical solution or not, the area in the field is divided into at least one unharvested area 100c, at least one harvested area 200c, and at least one field boundary area 300c. The unharvested area 100c represents an area where crops have not been harvested, the harvested area 200c represents an area where crops have been harvested, and the field boundary 300b represents the ridges separating planted crops, the outer boundary around the field, and obstacle areas present in the field.

As shown in Figures 1 and 4, the unworked area 100, the worked area 200, and the farmland boundary area 300 are recognized from the image acquired by the image acquisition device 20 using image segmentation recognition technology, and the boundaries between the areas are distinguished. The harvester further includes an image processing system 30, which recognizes the unworked area 100, the worked area 200, and the farmland boundary area 300 from the image based on the image of the farmland acquired by the image acquisition device 20 using image segmentation recognition technology.

It can be understood that the image processing system 30 uses image segmentation recognition technology to recognize the areas and boundaries in the image as the areas and boundaries of the farmland ahead of the harvester body 10. The harvester body 10 travels and works in unworked areas of the farmland while being controlled based on the areas and boundaries recognized by the image processing system 30 using image segmentation recognition technology. For example, an image of the farmland ahead of the harvester machine is acquired by the image acquisition device 20 provided at the front end of the harvester machine, and the image captured by the image acquisition device 20 is segmented and recognized by the image processing system 30, and the unworked area 100, the worked area 200, and the farmland boundary area 300 in the farmland in the traveling direction of the harvester machine are recognized and segmented. The harvester body 10, i.e., the host computer of the harvester machine, plans a vehicle travel route and mowing work based on the areas and boundaries recognized by the image processing system 30.

It can be understood that the image processing system 30 uses image segmentation recognition technology to recognize information such as the type of crop, the height of the crop, and the fullness of the fruit of the crop in the image provided by the image acquisition device 20. The image processing system 30 can determine whether the crop has been cut or not based on the type of crop and the height of the crop recognized in the image, and adjust work parameters based on the fullness of the fruit of the crop recognized in the image. In other words, the image processing system 30 can recognize the area type and boundaries of the farmland based on the image provided by the image acquisition device 20, and can also recognize the type, height, fullness of the fruit, and ripeness of the crop in the farmland.

The image processing system 30 selects one of the segmentation recognition methods, such as a threshold-based segmentation method, an area-based segmentation method, an edge-based segmentation method, and a segmentation method based on a specific theory, to segment and recognize the image acquired by the image acquisition device 20, thereby recognizing the areas and boundaries in the image. Preferably, the image processing system 30 uses a deep learning algorithm to segment and recognize the image, and performs area segmentation and boundary definition for the image. In other words, the image processing system 30 uses a deep learning algorithm to recognize the area and boundary of the corresponding farmland in the image, and the harvester body travels and works according to the recognized and segmented area and boundary. More preferably, the image processing system 30 uses an image segmentation recognition technique of a convolutional neural network algorithm in the deep learning algorithm to recognize the unworked area 100, the worked area 200, and the farmland boundary area 300 in the corresponding farmland from the image.

The processing algorithm used by the image processing system 30 is merely illustrative and not limiting. Therefore, the image processing system 30 may further use other algorithms to segment and recognize the acquired image, thereby recognizing farmland areas and boundaries in the image.

5A and 6, the image processing system 30 divides the image acquired by the image acquisition device 20 into a plurality of pixel regions 301, each of which includes at least one pixel unit. Since the image corresponds to the area around the harvester body 10, the pixel regions 301 of the image correspond to image information of a particular area of farmland or crops in the photographed farmland. It can be understood that the pixel units of the pixel regions 301 are normalized to a numerical value or array corresponding to a pixel value by performing a normalization process on each of the divided pixel regions 301. In other words, the image processing system 30 normalizes the divided pixel regions 301 to a corresponding numerical value or array in order to extract image features and divide the area.

The image processing system 30 extracts image features corresponding to the pixel regions 301 based on the arrays corresponding to each pixel region 301. The image processing system 30 obtains image features corresponding to the pixel regions 301 based on the arrays corresponding to the pixel regions 301. When the image processing system 30 uses a convolutional neural network algorithm such as a two-dimensional convolutional neural network, a two-dimensional array or a three-dimensional array corresponding to the pixel regions 301 is input to the input layer of the convolutional neural network. The hidden layer of the convolutional neural network performs feature extraction on the array in the input layer, and performs feature selection and information filtering after feature extraction. The convolutional neural network outputs classification labels of the pixel regions 301 corresponding to the unworked area 100, the worked area 200, and the farmland boundary area 300, respectively, based on the features corresponding to the arrays.

As shown in FIG. 6 and FIG. 7, the image processing system 30 recognizes region features corresponding to the pixel region 301 by extracting features of an array corresponding to the pixel region 301. The features corresponding to the pixel region 301 mainly include crop stalk height features, spacing of crop stalks in the field, crop color, field land color, crop type features, field land features, crop fruit fullness, number of crop fruits, etc. The image processing system 30 outputs a classification label corresponding to the pixel region 301 based on the extracted features, and the classification label marks the region type and boundary line corresponding to the pixel region 301 based on the feature information.

As shown in FIG. 5B, the image processing system 30 includes an image segmentation module 31, a characterization module 32, and an area partitioning module 33. The image segmentation module 31 acquires an image captured by the image acquisition module 20 and a plurality of pixel areas 301 formed by segmenting the image, each of which includes at least one pixel unit. The characterization module 32 uses a deep learning algorithm to extract feature types corresponding to the pixel areas 301, select features, and filter information. The area partitioning module 33 partitions the image based on the features corresponding to the pixel areas 301 extracted by the characterization module 32, thereby generating classification labels corresponding to the unworked area 100, the worked area 200, and the farmland boundary area 300.

Preferably, the image segmentation module 31 segments the image into a plurality of pixel regions 301, each of which has the same size, shape and extent. It can be understood that the image segmentation module 31 may further segment based on the magnitude of the image pixel threshold, i.e., the pixel regions 301 segmented by the image segmentation module 31 may have different sizes, shapes and extents. When the characterization module 32 of the image processing system 30 uses a convolutional neural network algorithm, it is further preferred that the pixel regions 301 segmented by the image segmentation module 31 are single pixel units.

The characterization module 32 includes a pixel processing module 321, a feature extraction module 322, and a feature output module 323. The pixel processing module 321 processes an array corresponding to a pixel unit in the pixel region 301. That is, the pixel processing module 321 normalizes the pixel region 301 into an array that is easy to process. After inputting the array of the pixel region 301 processed by the pixel processing module 321, the feature extraction module 322 extracts a feature type corresponding to the array, selects features, filters information, keeps usable data, and eliminates interference data, so that the feature extraction result is more accurate. The feature output module 323 outputs the features extracted by the feature extraction module 322, and combines the features output by the feature output module 323 through the region partition module 33 to generate the classification label of the corresponding region.

The region partitioning module 33 divides each region corresponding to the image based on the features corresponding to the pixel regions 301 extracted by the characterization module 32, and sets region boundaries. Therefore, the region partitioning module 33 further includes a region division module 331 and a boundary division module 332, where the region division module 331 divides different regions based on the features of the pixel regions 301, and the boundary division module 332 confirms the range of the regions by dividing the boundary ranges corresponding to the regions.

The image acquisition device 20 acquires images in real time within the field of view in front of the harvester body 10 while the harvester body 10 of the harvester is traveling. The image processing system 30 then acquires images captured by the image acquisition device 20 in real time, and recognizes the area division and area boundary range of the image corresponding to the farmland using image division recognition technology. If the area division and area boundary range recognized by the image processing system 30 do not match the previous area boundary range, the area division and area boundary range corresponding to the image are adjusted.

As shown in FIG. 8, problems such as vibration and deviation in running direction inevitably occur during running and operation of the harvester body 10. When the running direction of the harvester body 10 deviates or the vehicle vibrates, causing the area division to change, the image processing system 30 updates the area division and area boundary range corresponding to the image in real time.

As shown in FIG. 1, the harvester further includes a positioning device 40 and a navigation system 50, and the positioning device 40 is provided in the harvester body 10 to acquire positioning information of the harvester body 10. Preferably, the positioning device 40 acquires the position information of the harvester body 10 using satellite positioning information such as GPS or a Beidou positioning device. The navigation system 50 is provided in the harvester body 10, and by navigating the travel of the harvester body 10, the harvester body 10 realizes unmanned automatic driving and operation based on the positioning information of the positioning device 40, the area planning information obtained by the image processing system 30, and the navigation information of the navigation system 50.

It can be understood that the farmland area division and area boundary range information obtained by the image processing system 30 based on the image is updated in real time to the navigation system 50, and the navigation information of the navigation system 50 is updated. Preferably, the navigation system 50 is implemented as an inertial combined navigation system. It can be understood that the type of the navigation system 50 is merely exemplary and not limiting, and may be implemented as other types of navigation devices.

In response to the above, the harvester body 10 of the harvester includes a vehicle body 11, a work system 12 provided on the vehicle body 11, and a driving control system 13, and the work system 12 performs grain processing work such as harvesting work while being driven by the vehicle body 11. The driving control system 13 controls the travel of the vehicle body 11 and the work of the work system 12. The driving control system 13 has an unmanned driving mode and an operational driving mode. In the unmanned driving mode of the harvester, the driving control system 13 controls the vehicle body 11 to automatically drive and the work system 12 to work. On the other hand, in the operational driving mode of the harvester, the driving control system allows the driver to manually operate the vehicle body 11 and control the work of the work system.

In a first preferred embodiment of the present invention, the harvester is a harvester machine, and the work system 12 is implemented as a harvesting work machine. The driving control system 13 controls the travel of the vehicle body 11 and the work of the work system 12. In other words, the driving control system 13 controls the adjustment of the work parameters of the work system 12 while the vehicle body 11 is traveling. The driving control system 13 acquires information such as the type of crop, the height of the crop, the fullness of the fruit, and the diameter of the stem of the crop in the image recognized by the image processing system 30, and adjusts the work parameters of the work system 12 based on the acquired information, for example, adjusting the work speed, work width, and work height of the work system 12, and adjusting post-processing parameters, etc.

9 of the attached drawings of the specification of the present invention shows an embodiment of the unmanned operation and harvesting work of the harvester on farmland. In the unmanned operation mode, the operation control system 13 of the harvester body 10 acquires the positioning information of the vehicle body 11 provided by the positioning device 40, the navigation information provided by the navigation system 50, and the area recognition information provided by the image processing system 30, and further controls the vehicle body 11 to travel to the unworked area 100 of the farmland and perform the harvesting work. The image acquisition device 20 acquires the image of the area ahead of the vehicle body 11 in real time while the harvester body 10 is traveling and working, and the image processing system 30 recognizes the area range and boundary range from the image using image segmentation recognition technology. If the area division and boundary range obtained by the image processing system 30 do not match the previous area division and boundary range, the image processing system 30 replaces the original area division and boundary range and updates the navigation data of the navigation system 50 so that the driving control system 13 obtains new navigation information and adjusts the driving and work paths.

As shown in Figures 10 to 11B, the harvester generates at least one planned route based on the position information of the harvester body 10 acquired by the positioning device 40, the area planning information of the image recognized by the image processing system 30, and the navigation information of the navigation system 50. The driving control system 13 of the harvester body 10 controls the travel of the vehicle body 11 and the work of the work system 12 according to the generated planned route. Therefore, the harvester further includes a route planning system 60 that plans at least one vehicle travel route for the harvester body 10. The route planning system 60 acquires the positioning information of the positioning device 40, the area planning information of the image recognized by the image processing system 30, and the navigation information of the navigation system 50, and plans a travel route for the vehicle body 11 based on the acquired information.

11A and 11B, the route planning system 60 recognizes or sets at least one corresponding work area 601 and work boundary 602 in the farmland, the work area 601 being the widest working range of the harvester, and the driving control system 13 controls the vehicle body 11 to travel within the range of the work boundary 602. It can be understood that, regarding the work area 601 and the work boundary 602, the image processing system 30 may recognize the widest range and boundary of the work area 601 by recognizing the farmland boundary area 300 in the image. Alternatively, the route planning system 60 may set the work area 601 of the harvester.

The path planning system 60 plans at least one travel route based on the work boundary 602 at the outermost side of the work area 601. When the width of the work area 601 is wider than the work width of the work system 12, the path planning system 60 plans a "circle"-shaped travel route or an "S"-shaped travel route. It can be understood that the form of the travel route planned by the path planning system 60 is merely exemplary and not limiting. Therefore, other forms of travel routes are also applicable.

Preferably, when the vehicle body 11 travels to the far boundary of the working area 601, the route planning system 60 re-plans at least one travel route based on the extent of the current unworked area 100. In other words, when the vehicle body 11 travels to the far boundary of the working area 601, the route planning system 60 updates the working area 601 and the working boundary 602 for the vehicle body 11, and plans a new travel route based on the updated working area 601.

It can be understood that the driving control system 13 controls the vehicle body 11 to travel according to the travel route planned by the route planning system 60, and controls the work system 12 to mow the crops on the outermost side of the work area 401. In other words, the driving control system 13 controls the work system 12 to mow the crops in the unworked area 100 according to the work boundary 602.

As shown in FIG. 10, the route planning system 60 of the harvester includes a work area setting module 61, a travel route planning module 62, and a route adjustment module 63. The work area setting module 61 obtains the work area 601 and the work boundary 602 of the farmland based on the boundary area of the farmland in the image recognized by the image processing system 30, or sets the work area 601 and the work boundary 602 in which the harvester body 10 works on the farmland. Because the unworked area 100 and the worked area 200 have changed due to the work of the harvester body 10, the work area setting module 61 generates new unworked area 100 and worked area 200 by updating the range of the work area 601 and the boundary line of the work boundary 602 in real time.

The travel route planning module 62 obtains at least one travel planned route 603 based on the positioning information of the harvester body 10, the area planning information of the image recognized by the image processing system 30, and the navigation information of the navigation system 50, and the driving control system 13 controls the vehicle body 11 to travel according to the travel planned route 603. The route adjustment module 63 adjusts the travel direction of the harvester body 10 based on the information of the agricultural crops in the image recognized by the image processing system 30, and forms a vehicle travel route 604 that is approximately overlapping or parallel to the travel planned route 603. If the image processing system 30 recognizes that the mowing range of the agricultural crops in the image should be adjusted, the vehicle travel route generated by the route adjustment module 63 deviates from the travel planned route 603.

With reference to Figs. 12 to 16B of the attached drawings of the specification of the present invention, a harvester with an image capture device according to a second preferred embodiment of the present invention is described below. The harvester includes a harvester body 10 and at least one image capture device 20, which is provided on the harvester body 10 and captures images or video images of the farmland on which the harvester body 10 is located, so that the harvester body 10 controls the running direction and/or work parameters based on the image or video information captured by the image capture device 20. The image capture device 20 captures farmland information around the farmland on which the harvester body 10 is located, based on the position of the harvester body 10. It can be understood that the image capture device 20 captures images within a field of view, for example, images within the field of view of the driver, and adjusts the operating parameters of the harvester body 10 based on the captured images, for example, adjusting the running path, running speed, work parameters, etc.

The image acquisition device 20 is mounted on the harvester body 10, and the image and video information acquired by the image acquisition device 20 is transmitted to the harvester body 10, and the harvester body 10 adjusts the operating parameters based on the information. The image acquisition device 20 is mounted on the harvester body 10, and captures clear images even if the harvester body 10 shakes. In other words, the image acquisition device 20 is an anti-shake imaging device, and can avoid vibration caused by mechanical vibration of the harvester body 10 itself and unevenness of the ground during shooting. The harvester body 10 controls the travel path and work parameters by the operator's operation based on the image information captured by the image acquisition device 20, or automatically controls the travel path and work parameters, thereby realizing the operation of the harvester. In other words, the harvester body 10 adjusts the operation and work parameters based on the image information captured by the image acquisition device 20, and realizes accurate work/or unmanned automatic driving work.

Preferably, in a second preferred embodiment of the present invention, the image capture device 20 is implemented as a camera-head imaging device that captures stable images or videos even when vibrated or shaken.

In a second preferred embodiment of the present invention, the image capture device 20 is a mechanical pan head device, which is mounted so as to be mechanically connected to the harvester body 10, and realizes image shake prevention by mechanical shake prevention. It can be understood that the type of the image capture device 20 is merely exemplary and not limiting. Therefore, other types of structures and mounting configurations are also applicable.

12 to 14, the image acquisition device 20 includes a camera platform 21 and at least one video camera 22, and the video camera 22 is attached to the harvester body 10 by the camera platform 21, and the mounting position of the video camera 22 is fixed by the camera platform 21. The bottom end of the camera platform 21 is mounted on the harvester body 10, and the camera platform 21 is fixed via the harvester body 10, and the top end of the camera platform 21 is connected to the video camera 22. The video camera 22 maintains a balanced state by being supported by the camera platform 21, and stably captures or films images. The video camera 22 captures images or videos around the harvester body 10 under the supporting action of the camera platform 21, and captures images within the field of view of the harvester body 10 based on the mounting position of the camera platform 21. It can be understood that the video camera 22 of the image capture device 20 captures at least one visual image by taking a photograph based on the position of the harvester body 10. In other words, the video camera 22 of the image capture device 20 captures the image within the field of view of the harvester body 10, thereby avoiding the problem of inaccurate image data due to changes in the position of the imaging device 20 and the position of the harvester body 10.

In addition, in order for an autonomous vehicle to recognize and acquire the vehicle's driving route in autonomous driving mode, it is necessary to acquire accurate vehicle positioning information, which usually requires highly accurate satellite positioning information. In order for the autonomous vehicle to realize the autonomous driving function in a high-speed driving state, it is necessary to update information such as obstacle information on the road, road vehicle information, and road pedestrians in real time. The image acquired by the harvester of the present invention is image data information corresponding to the agricultural crop grains in the farmland, and the image is an image of the surroundings of the vehicle acquired based on the current position of the vehicle. The harvester does not require ultra-high accuracy satellite positioning information, and only general satellite positioning with meter-level accuracy (such as GPS positioning or Beidou positioning) is required. Therefore, the image acquired and processed by the harvester is different from that of an autonomous vehicle, and the route plan and driving mode formed by the harvester are also not the same. It can be understood that the function of the harvester of the present invention to visually recognize the area of the farmland and the autonomous driving function are different from the recognition mode of an autonomous vehicle.

The tripod head 21 of the image acquisition device 20 further includes a tripod head fixing member 211 and at least one tripod head moving member 212, and the tripod head moving member 212 is movably connected to the tripod head fixing member 211. The tripod head fixing member 211 is fixedly provided on the harvester body 10, and the video camera 22 is attached to the tripod head moving member 212. The tripod head moving member 212 of the tripod head 21 movably supports the video camera 22, so that even if the harvester body 10 shakes, the video camera 22 maintains a stable relative position, thereby taking clear images.

In other words, when the harvester body 10 vibrates or swings, for example, mechanically vibrates or swings when harvesting in farmland, the camera head fixing member 211 of the camera head 21 vibrates together with the harvester body 10, and the camera head moving member 212 of the camera head 21 moves relative to the camera head fixing member 211, neutralizing the vibrations caused by the camera head fixing member 211, thereby maintaining the positional stability of the video camera 22. In more detail, the camera head moving member 212 neutralizes the shaking or vibrations of the motion fixing member 211 in the up-down, left-right, and front-back directions, maintaining the stability of the photographing position of the video camera 22, and capturing more stable image information.

As shown in FIG. 14, the video camera 22 of the image acquisition device 20 is provided on the head moving member 212 of the head 21 and is fixedly or movably attached to the head moving member 212 of the head 21. Preferably, the video camera 22 is movably provided on the moving member 212 and is rotatable with respect to the upper end of the head moving member 212 to capture images in different viewing directions. Optionally, the video camera 22 is fixedly attached to the upper end of the head moving member 212 and captures images within a specified viewing range under the fixed and supported action of the head 21, for example, capturing images within the forward viewing field of the harvester body 10.

The video camera 22 includes a video camera body 221 and at least one video camera driving device 222, and the video camera driving device 222 drives and moves the video camera body 221 so as to capture images of fields of view in different directions. The video camera body 221 is movably mounted on the pan head moving member 212 and can rotate up and down under the driving action of the video camera driving device 222, thereby capturing images of farmland and crops far and near the harvester body 10. It can be seen that when the video camera body 221 is driven by the video camera driving device 222 to rotate downward, it captures an image near the harvester body 10, thereby clearly recognizing information of the farmland in the image. When the video camera body 221 is driven by the video camera driving device 222 to rotate upward, it captures an image far from the harvester body 10, thereby recognizing the work area and farmland boundary area of the farmland from the image.

The video camera driving device 222 drives the video camera body 221 to rotate it left and right, causing the video camera body 221 to capture images of the left and right sides of the harvester body 10, thereby recognizing the unworked area 100, the worked area 200, and the farmland boundary area 300 of the farmland.

Figure 15 of the accompanying drawings of the present specification shows some optional mounting forms and mounting positions of the image capture device 20 attached to the harvester body 10. In a second preferred embodiment of the present invention, the image capture device 20 of the harvester is provided at the front position, top, left side, right side, rear position, etc. of the harvester body 10. It can be understood that the captured image is different and the information recognized from the image is not the same depending on the mounting position of the image capture device 20. The image capture device 20 provided at the front side of the harvester body 10 captures an image in front of the harvester body 10. It can be understood that when the harvester travels forward to work, the image capture device 20 at the front side of the harvester body 10 captures the working situation of the harvester body 10, and adjusts the travel path, working parameters, etc. of the harvester body 10 according to the captured working situation.

The image capture device 20 provided at the rear of the harvester body 10 captures an image of the rear of the harvester body 10, and captures an image of the worked area 200 when the harvester travels forward to work. By recognizing the image of the worked area 200 captured by the image capture device 20 at the rear of the harvester body 10, it is possible to recognize whether the harvester body 10 has passed the harvesting work, and it is easy to adjust the work parameters of the harvester body 10. In addition, the harvester body 10 recognizes whether the crops in the worked area 200 have been completely harvested and whether the fruits of the crops remain, etc., based on the image captured by the image capture device 20 provided at the rear of the harvester body 10. The harvester body 10 adjusts the work parameters based on the information recognized from the image, and further improves the harvesting work. In addition, when traveling backward, the image captured by the image capture device 20 provides the driver with a back guide image.

The image capture device 20 installed at the top of the harvester body 10 captures images of the harvester body 10 at a distance, and recognizes the work area of the farmland, the border area of the farmland, etc. based on the images. Preferably, the image capture device 20 installed at the top of the harvester body 10 is a rotatable pan-head video camera.

The image acquisition device 20 provided on the left or right side of the harvester body 10 captures an image of the left or right side of the harvester body 10. Based on the image of the left or right side of the harvester body 10, the unworked area 100, the worked area 200, and the farmland boundary area 300 are recognized by recognizing the state of the farm crops in the image.

As shown in Figures 12 and 14, the harvester further includes an image processing system 30, a positioning device 40, and a navigation system 50, and the image processing system 30, the positioning device 40, and the navigation system 50 are provided in the harvester body 10. The positioning device 40 acquires position information of the harvester body 10 and transmits the acquired position information to the harvester body 10. The navigation system 50 provides navigation information to the harvester body 10 based on the positioning information of the positioning device 40. The image processing system 30 recognizes the unworked area 100, the worked area 200, and the farmland boundary area 300 from the image based on the image of the farmland acquired by the image acquisition device 20.

Preferably, the image processing system 30 recognizes the unworked area 100, the worked area 200, and the farmland boundary area 300 from the image using image segmentation recognition technology. It can be understood that the image processing system 30 may recognize area and boundary information in the image in other forms. Therefore, in the second preferred embodiment of the present invention, the form in which the image processing system 30 recognizes the image is merely exemplary and not limiting.

As shown in Figures 16A and 16B, the image processing system 30 recognizes the farmland area and farmland boundaries in the image based on the image of the surroundings of the harvester body 10 captured by the image acquisition device 20, and recognizes information such as the type of crop in the farmland, the height of the crop, the fullness of the fruit, and the thickness and size of the stem.

The image processing system 30 selects one of the segmentation recognition methods, such as a segmentation method based on a threshold, a segmentation method based on an area, a segmentation method based on an edge, or a segmentation method based on a specific theory, and segments and recognizes the image acquired by the image acquisition device 20 to recognize the areas and boundaries in the image. Preferably, the image processing system 30 uses a deep learning algorithm to segment and recognize the image, and performs area segmentation and boundary definition for the image. In other words, the image processing system 30 uses a deep learning algorithm to recognize the area and boundary of the corresponding farmland in the image, and the harvester body 10 travels and works according to the recognized and segmented area and boundary. More preferably, the image processing system 30 uses an image segmentation recognition technique of a convolutional neural network algorithm in the deep learning algorithm to recognize the unworked area 100, the worked area 200, and the farmland boundary area 300 in the corresponding farmland from the image.

The processing algorithms used by the image processing system 30 are merely illustrative and not limiting. Therefore, the image processing system 30 may further use other algorithms to segment and recognize the acquired image and to recognize farmland areas and boundaries in the image.

The image processing system 30 is an image processor provided in the harvester body 10, and the image processor receives images or videos captured by the image acquisition device 20 and recognizes information in the images or videos. It can be understood that the harvester body 10 controls the travel path and adjusts the work parameters by operating in accordance with the information recognized by the image processing system 30.

12 and 14, the harvester body 10 further includes a vehicle body 11, a work system 12 provided on the vehicle body 11, and an operation control system 13. The work system 12 is connected to the vehicle body 11, and the vehicle body 11 drives the work system 12 to perform crop harvesting work. The operation control system 13 controls the travel of the vehicle body 11 and controls the work of the work system 12. The operation control system 13 has an unmanned operation mode and an operation operation mode. In the unmanned operation mode of the harvester, the operation control system 13 controls the vehicle body 11 to automatically operate and the work system 12 to operate. On the other hand, in the operation operation mode of the harvester, the operation control system allows the driver to manually operate the vehicle body 11 and control the work of the work system.

In a second preferred embodiment of the present invention, the driving control system 13 controls the travel of the vehicle body 11 and the mowing work of the work system 12. In other words, the driving control system 13 controls the adjustment of the work parameters of the work system 12 while the vehicle body 11 is traveling. The driving control system 13 acquires information such as the type of crop, the height of the crop, the degree of fruit fullness, and the diameter size of the stem of the crop in the image recognized by the image processing system 30, and adjusts the work parameters of the work system 12 based on the acquired information, for example, adjusting the work speed, work width, work height of the work system 12, and adjusting post-processing parameters, etc.

The work system 12 further includes at least one mowing device 121, at least one transport device 122, and at least one post-processing device 123, and the transport device 122 is configured to receive the crops cut by the mowing device 121 and transport the crops to the post-processing device 123 so that the post-processing device 123 performs post-processing on the crops. The mowing device 121, the transport device 122, and the post-processing device 123 of the work system are each provided and connected to the vehicle body 11 by transmission, and the vehicle body 11 drives the mowing device 121, the transport device 122, and the post-processing device 123 of the work system 12 to operate and work. The post-treatment device 123 is implemented as a subsequent treatment device after the crop is harvested. For example, if the harvester is a grain harvester, it is implemented as a threshing device. If the harvester is a grass cutting machine, it is implemented as a packing device. If the harvester is a vegetable or fruit harvesting machine, it is implemented as a vegetable or fruit screening and storage device.

The driving control system 13 controls the width, cutting height, and cutting speed of the mower 121 based on the image information recognized by the image processing system 30. When the density of crops in the farmland is high, the information of the crops in the farmland photographed by the image acquisition device 20 is recognized by the image processing system 30, and it can be understood that the driving control system 13 controls any of the work parameters of the mower 121, such as reducing the cutting width, increasing the cutting height, and reducing the cutting speed, based on the image information recognized by the image processing system 30.

The driving control system 13 controls the transport speed, transport power, etc. of the transport device 122 based on the image information recognized by the image processing system 30. When the stems of the crops in the farmland are coarse, and the crops are tall and dense, the information of the crops in the farmland photographed by the image acquisition device 20 is recognized by the image processing system 30, and it can be understood that the driving control system 13 controls the transport device 122 to improve work parameters such as the transport speed and transport power based on the image information recognized by the image processing system 30.

The operation control system 13 controls the post-processing parameters of the post-processing device 123 based on the image information recognized by the image processing system 30. The image information includes the fullness of the fruit of the crop in the farmland, the size of the fruit, the moisture content, the humidity, the type of the fruit of the crop, etc. It can be understood that the image processing system 30 recognizes the crop information of the crop in the farmland, and the operation control system 13 adjusts the post-processing parameters of the post-processing device, such as the blowing power and the rotation speed of the post-processing chamber, based on the image information recognized by the image processing system 30.

With reference to FIG. 17 of the accompanying drawings of the present specification, another optional embodiment of the image capture device 20A of the harvester according to the second preferred embodiment of the present invention is described in the following description. In this optional embodiment, the image capture device 20A realizes lens photography and prevents shake by controlling the lens viewpoint and zoom inside a video camera.

In response to the above, the image acquisition device 20A includes a camera mounting means 21A and at least one video camera 22A, and the video camera 22A is mounted on the harvester body 10 by the camera mounting means 21A. The bottom end of the camera mounting means 21A is mounted on the harvester body 10, the camera mounting means 21A is fixed via the harvester body 10, and the upper end of the camera mounting means 21A is connected to the video camera 22A. The video camera 22A maintains a balanced state by being supported by the camera mounting means 21A, and stably captures or films images. The video camera 22A captures images or videos of the surroundings of the harvester body 10 with the support action of the camera mounting means 21A, and the video camera 22A captures images within the field of view of the harvester body 10 based on the mounting position of the camera mounting means 21A.

The video camera 22A of the image capture device 20A captures at least one visual image by taking a photograph based on the position of the harvester body 10. In other words, it can be seen that the video camera 22A of the image capture device 20A captures the image within the field of view of the harvester body 10, thereby avoiding the problem of image data becoming inaccurate due to changes in the position of the imaging device 20A and the harvester body 10 position.

According to another aspect of the present invention, there is further provided an automatic operation method for a harvester, the automatic operation method comprising:
(a) acquiring at least one image and recognizing farmland areas and farmland boundaries in said image;
(b) planning at least one travel plan route 603 based on the recognition information;
and (c) controlling the harvester body 10 to automatically travel along the travel plan route 603.

In the above-mentioned automatic driving method step, the driving control system 13 controls the driving and operation of the harvester body 10 based on the area information and farmland boundaries recognized by the image processing system 30.

Step (a) of the above-mentioned autonomous driving method further includes a step of recognizing information corresponding to the crops in the farmland in the image, the crop information including information on the crop type, crop height, fruit fullness, etc.

The above-mentioned automatic driving method step (b)
(b.1) recognizing and demarcating areas and boundaries of said image corresponding to agricultural fields;
and (b.2) planning at least one driving plan route 603 based on the recognized area.

Step (b.1) of the above-mentioned autonomous driving method further includes a step of segmenting the image using an image segmentation technique and recognizing and partitioning regions of the image.

In step (a) of the above-mentioned automatic driving method, image information of the surroundings of the harvester body 10 is captured in real time according to the position and traveling direction of the harvester body 10. In other words, the image acquisition device 20 captures images in the vicinity of the harvester body 10 in real time.

In step (b) of the autonomous driving method, the image processing system uses an image segmentation technique to segment the image information, and recognizes and divides the image into the unworked area 100, the worked area 200, and the farmland boundary area 300. Accordingly, step (b.1) of the autonomous driving method includes:
Dividing the image into a plurality of pixel regions 301 and normalizing pixel values of the pixel regions 301 into an array;
extracting features of the pixel regions 301 corresponding to each of the arrays;
and outputting a classification label for the image based on the features corresponding to the pixel region 301.

It can be understood that the classification labels correspond to the unworked area 100, the worked area 200, and the farmland boundary area 300.

Step (b.2) of the above-mentioned automatic driving method further includes a step in which the image processing system 30 recognizes area planning information of the image and navigation information of the navigation system 50 based on the positioning information of the harvester body 10, and acquires the driving plan route 603.

Step (b.2) of the above-mentioned automatic driving method further includes a step of adjusting the traveling direction of the harvester body 10 based on the information of the agricultural crop in the image recognized by the image processing system 30, and forming a vehicle traveling path 604.

The above-mentioned autonomous driving method further includes a step (b.3) of comparing whether the area division and area boundary range recognized by the image processing system 30 match the previous area boundary range, adjusting the area division and area boundary range corresponding to the image if they do not match, and maintaining the area division and boundary range unchanged if they match.

In response to the above, in the above method step (c), the driving control system 13 controls the travel of the vehicle body 11 of the harvester body 10 based on the positioning information of the harvester body 10, the area planning information of the farmland obtained by the image processing system 30, and the navigation information.

The above-mentioned automatic operation method further includes a step (d) of adjusting the operation parameters of the operation system 12 of the harvester body 10 based on the image recognition information.

It can be understood by those skilled in the art that the embodiments of the present invention described above and shown in the drawings are merely examples and are not intended to limit the present invention. The objectives of the present invention have been fully and effectively achieved. The functions and structural principles of the present invention have been shown and explained in the examples, and the embodiments of the present invention are susceptible to any variation or modification without departing from the above principles.

Claims (13)

A harvester body;
At least one image capture device provided on the harvester body for capturing images of the surroundings of the harvester body;
and an image processing system that recognizes farmland information in the image based on the image captured by the image capture device,
The image processing system includes:
an image segmentation module for segmenting the image into a plurality of pixel regions, each pixel region including at least one pixel unit;
a characterization module for normalizing pixel values of the pixel regions into arrays based on the pixel units of the pixel regions, extracting features of the pixel regions corresponding to each of the arrays, and selecting the features ;
a region partitioning module for recognizing and partitioning regions of the image based on characteristics of the pixel regions;
The image processing system divides the farmland in the image into different areas by the image segmentation module, and the farmland information includes an area division and an area boundary range corresponding to the image; the image processing system acquires images captured by the image acquisition device in real time, and compares whether the recognized area division and area boundary range are consistent with the previous area boundary range, and if not consistent, adjusts the area division and area boundary range corresponding to the image; if consistent, keeps the area division and area boundary range unchanged;
A harvester characterized in that the harvester body automatically controls operation based on the farmland information recognized by the image processing system. Further comprising a route planning system that plans at least one travel route based on the farmland information recognized by the image processing system;
The harvester according to claim 1 , wherein the harvester body controls operation according to the travel plan route planned by the route planning system. The harvester of claim 2, wherein the image processing system uses image segmentation recognition technology to recognize crop information in the image, and the harvester body automatically adjusts operation parameters based on the recognized information. The harvester according to claim 1, wherein the image capture device is provided at the front of the harvester body, at the top of the harvester body, on the left side of the harvester body, on the right side of the harvester body, or at the rear of the harvester body. The harvester of claim 2, further comprising a positioning device and a navigation system, the positioning device and the navigation system being provided in the harvester body, the positioning device acquiring position information of the harvester body, and the navigation system providing navigation information to the harvester body. The path planning system comprises:
a working area setting module for setting a boundary area of the farmland to obtain a working area and a working boundary of the farmland;
A travel route planning module,
The harvester according to claim 5 , wherein the route planning system obtains at least one travel planned route based on positioning information of the harvester body , area planning information of the image recognized by the image processing system, and navigation information of the navigation system. The harvester according to claim 1, wherein the harvester body includes a vehicle body, at least one work system provided on the vehicle body, and an operation control system, the vehicle body drives and operates the work system, and the operation control system controls the operation of the vehicle body and the work parameters of the work system. The harvester according to claim 7, wherein the driving control system acquires information about images recognized by the image processing system and captured by the image capture device, automatically controls the travel path of the vehicle body, and controls the work parameters of the work system, thereby realizing unmanned automatic driving and mowing work. In the automatic operation method of a harvester,
(a) acquiring at least one image and recognizing farmland areas and farmland boundaries in said image;
(b) planning at least one driving plan route based on the recognition information;
and (c) a step of controlling the harvester body to automatically travel along the travel planned route,
The step (b)
(b.1) segmenting the image using image segmentation techniques to identify and demarcate areas and boundaries of the image corresponding to agricultural fields;
(b.2) planning at least one travel plan route based on the recognized area;
(b.3) comparing whether the area division and area boundary range recognized by the image processing system are consistent with the previous area boundary range, and if they are not consistent, adjusting the area division and area boundary range corresponding to the image, and if they are consistent, keeping the area division and area boundary range unchanged;
The step (b.1) comprises:
Dividing the image into a number of pixel regions and normalizing pixel values of the pixel regions into an array;
extracting features of the pixel regions corresponding to each of the arrays;
and outputting a classification label for the image based on features corresponding to the pixel region. The autonomous driving method according to claim 9, wherein step (a) further includes a step of recognizing information corresponding to the crops in the farmland in the image, and the information on the crops includes plant information such as the type of crop, the height of the crop, and the degree of fruit fullness. The autonomous driving method according to claim 9, wherein in step (b), the image processing system uses an image segmentation technique to segment the image, and recognizes and divides the areas of the image into an unworked area, a worked area, and a farmland boundary area. The autonomous driving method according to claim 9, wherein step (b.2) further includes a step of planning the driving plan route based on positioning information of the harvester body, area planning information of the image, and navigation information of a navigation system. The automatic driving method according to claim 9 , further comprising the step (d) of adjusting work parameters of a work system of the harvester body based on the image recognition information.