WO2020134236A1

WO2020134236A1 - Harvester and automatic driving method thereof

Info

Publication number: WO2020134236A1
Application number: PCT/CN2019/107551
Authority: WO
Inventors: 吴迪; 王清泉; 沈永泉; 王波; 张虓; 童超; 范顺; 陈睿
Original assignee: 丰疆智能科技股份有限公司
Priority date: 2018-12-29
Filing date: 2019-09-24
Publication date: 2020-07-02
Also published as: JP2022516898A

Abstract

A harvester and an automatic driving method thereof. The harvester comprises a harvester main device (10), at least one image acquisition device (20), a path planning system (60), and an image processing system (30). The image acquisition device (20) is disposed on the harvester main device (10), and the image acquisition device (20) captures an image around the harvester main device (10). The image processing system (30) identifies farmland information in the image on the basis of the image captured by the image acquisition device (20). The harvester main device (10) automatically controls driving according to the farmland information identified by the image processing system (30). The path planning system (60) plans at least one planned travel path on the basis of the farmland information identified by the image processing system (30). The harvester main device (10) controls driving according to the planned travel path planned by the path planning system (60).

Description

Harvester and its automatic driving method

Technical field

The invention relates to the field of automatic driving of agricultural machinery, in particular to a harvester and its automatic driving method.

Background technique

The harvester is a crop harvesting machine that harvests grains and straw of crops such as rice and wheat. In addition, the harvester also includes a lawn mower to harvest other crops, such as machinery and equipment for harvesting vegetables and fruits. The grain harvester is an integrated machine for harvesting crops. It can complete the harvesting, threshing, and concentrate the grains in the storage bin at one time, and then transport the grains to the transport vehicle through the conveyor belt. The fruit and vegetable harvesting equipment can harvest vegetables and fruits in the farmland at one time, separate the harvested fruits from the stalks, and then sort them.

When the harvester performs the harvesting operation, it is necessary to observe the conditions of the crops in the farmland and the farmland working area at all times, so as to adjust the operating parameters of the harvester according to the height, maturity, and grain fullness of the crops. When agricultural machinery and equipment are operating in the farmland, it is necessary to judge the farmland operation situation and the crop growth situation in the farmland in real time to operate the agricultural machinery equipment operation and adjust the operation system of the complicated situation of the operation system. Now it can only be done by machine operators with professional skills. Agricultural machinery and equipment need to consider many factors such as the operated area, unoperated area, and the boundary between the world and the earth when working in the farmland, and during the operation process, it is necessary to adjust the operation of the vehicle and adjust the operation in real time according to the conditions of the crops. parameter. Due to the need to consider the complex operating environment during driving, the prior art agricultural equipment also requires the operator to adjust the operation of the agricultural machinery equipment based on real-time farm crop information. The probability of occurrence of a judgment error in the operation of the agricultural machinery equipment controlled by manual operation is large, and the probability of failure of the machinery equipment during the operation is large.

This prior art harvester has at least one of the following defects: First, when the harvester is in operation, the vibration of the vehicle itself and the unevenness of the farmland and land will cause the harvester body to shake up and down, resulting in the installation of The camera device of the harvester body cannot capture images at stable positions. Therefore, the images acquired through the camera device are often blurred, and cannot provide information support for intelligent operations and automatic driving. Secondly, the prior art camera device is fixedly installed on the harvester body, and can only acquire images in a single direction, such as the image in front of the harvester, but cannot adjust the shooting direction of the camera device according to the situation And location. Thirdly, the prior art mobile camera equipment or fixed camera equipment, such as a drone camera device or a camera device fixed in the farmland, captures the image around the harvester and transmits it to the harvester body for the The harvester body reads the image captured by the camera device. Although the problem of unclear image capture is solved to some extent, images captured based on the camera device itself or based on the position of the drone cannot be obtained from the perspective of the harvester itself. Therefore, the acquired image cannot be well recognized.

The agricultural machinery and equipment in the prior art usually cause errors in the operation due to the inaccurate set operation path, and even serious mechanical failures. In addition, the satellite positioning method using PTK has high requirements for the performance of agricultural equipment, and the manufacturing cost and maintenance cost required are relatively high. Therefore, this prior art automatic driving positioning method is not applicable to the current In the automatic driving mode of agricultural machinery and equipment.

Summary of the invention

A main advantage of the present invention is to provide a harvester and its automatic driving method, wherein the harvester recognizes the area of the farmland in the graphic based on the captured at least one visual image.

Another advantage of the present invention is to provide a harvester and its automatic driving method, wherein the harvester recognizes crop information such as the type, height, and maturity of the crop in the graphic based on the captured at least one visual image.

Another advantage of the present invention is to provide a harvester and its automatic driving method, wherein the harvester recognizes the unoperated area, the operated area, and the field boundary area in the visual image based on the visual image, so that The identified area controls the driving path of the harvester.

Another advantage of the present invention is to provide a harvester and its automatic driving method, wherein the harvester recognizes the information of the crop in the image based on the visual image, and the harvester recognizes the information in the image Adjust the operating parameters of the harvester to improve the operating quality and efficiency of the harvester.

Another advantage of the present invention is to provide a harvester and its automatic driving method, wherein the image acquisition device of the harvester is a pan-tilt camera device, wherein the pan-tilt camera device has an anti-shake shooting function, which improves the harvesting The accuracy and stability of the machine to obtain visual images.

Another advantage of the present invention is to provide a harvester and its automatic driving method, wherein the image acquisition device is provided on a harvester body of the harvester, wherein the harvester photographs the image through the image acquisition device Images around the harvester body.

Another advantage of the present invention is to provide a harvester and its automatic driving method, wherein the image acquisition device is provided on a harvester body of the harvester, wherein the image acquisition device is provided on the harvester body , Wherein the image acquisition device shoots at least one visual image or visual image based on the position of the field of view of the harvester body, so as to identify the information around the harvester body according to the captured image information.

Another advantage of the present invention is to provide a harvester and its automatic driving method, wherein the image acquisition device can be adjusted to take images of different angles and different directions based on the position of the main machine of the harvester to facilitate acquisition of Describe the images of the harvester main machine in different directions.

Another advantage of the present invention is to provide a harvester and its automatic driving method, wherein the image acquisition device is a mechanical gimbal camera or an electronic gimbal camera, and the stability of the visual image is improved by the image acquisition device Sex.

Another advantage of the present invention is to provide a harvester and its automatic driving method, wherein a path planning system of the harvester automatically plans a route based on the current vehicle positioning information, the information recognized by the image processing system, and the information of the navigation system.

Another advantage of the present invention is to provide a harvester and its automatic driving method, wherein the harvester plans the driving path and working route of the harvester based on the area identified by the visual image.

Another advantage of the present invention is to provide a harvester and its automatic driving method, wherein an image acquisition device of the harvester acquires the visual image of the surrounding farmland in real time, and updates the path navigation information planned by the harvester in real time .

Another advantage of the present invention is to provide a harvester and its automatic driving method, wherein the harvester captures an image in real time through the image acquisition device, recognizes the area in the visual image, and changes according to the area Update or adjust the working route of the harvester in real time to improve the working quality of the harvester.

Another advantage of the present invention is to provide a harvester and its automatic driving method, wherein the image processing system of the harvester uses image segmentation technology to identify the unworked in the image based on the acquired visual image information Area, the operated area, and the field boundary area, and the boundary dividing the two adjacent areas.

Another advantage of the present invention is to provide a harvester and its automatic driving method, wherein the image processing system of the harvester uses image segmentation technology to identify the type and height of crops in the image based on the acquired visual image information Plant information, such as grain fullness, for the operating system of the harvester to adjust the operating parameters based on the crop information.

Another advantage of the present invention is to provide a harvester and its automatic driving method, wherein the image processing system of the harvester recognizes the area boundary in the image based on the acquired image information, so that the path planning system is based on The identified boundary of the area plans a driving path of the vehicle.

Another advantage of the present invention is to provide a harvester and its automatic driving method, wherein the harvester does not require high-precision satellite positioning, which reduces the difficulty of manufacturing the automatic driving equipment and reduces the maintenance cost of the equipment .

Another advantage of the present invention is to provide a harvester and its automatic driving method, wherein the harvester performs path planning based on the area division information output by the image processing system to realize automatic driving and automatic driving operations.

Other advantages and features of the present invention can be fully realized by the following detailed description and can be realized by the combination of means and devices specifically pointed out in the appended claims.

According to one aspect of the present invention, a harvester of the present invention that can achieve the foregoing objectives and other objectives and advantages includes:

One harvester host;

At least one image acquisition device, wherein the image acquisition device is provided on the harvester host, and the image acquisition device captures images around the harvester host, and

An image processing system, wherein the image processing system recognizes farmland information in the image based on an image captured by the image acquisition device, wherein the harvester host automatically determines the farmland information identified by the image processing system Control driving.

According to an embodiment of the present invention, the harvester further includes a path planning system, wherein the path planning system plans at least one travel planning path based on the farmland information identified by the image processing system, wherein the harvester The host computer controls driving according to the travel planning path planned by the path planning system.

According to an embodiment of the present invention, the image processing system uses image segmentation and recognition technology to identify the information of the farmland in the image, and plans the area of the farmland in the image based on the identified information.

According to an embodiment of the present invention, the image processing system uses image segmentation and recognition technology to identify crop information in the image for the harvester host to automatically adjust operation parameters based on the identified information.

According to an embodiment of the present invention, the image acquisition device is an anti-shake PTZ camera device, the image acquisition device is loaded on the harvester host, and the photo is taken in a photographic manner based on the position of the harvester host Images around the harvester mainframe.

According to an embodiment of the present invention, the image acquisition device is a mechanical anti-shake gimbal device, and the image acquisition device includes a gimbal and at least one camera, wherein the gimbal mounts the camera to the harvester The host, the camera is installed on the gimbal, and the camera is supported by the gimbal to maintain balance.

According to an embodiment of the present invention, the image acquisition device is an electronic pan/tilt device, and the image acquisition device controls the angle of view and zoom of the lens, thereby preventing the camera of the image acquisition device from shaking.

According to an embodiment of the present invention, the image acquisition device is disposed at the front of the harvester host, at the top of the harvester host, on the left or right side of the harvester host, or the harvester The rear of the host.

According to an embodiment of the invention, the image processing system further includes:

An image segmentation module, wherein the image segmentation module divides the image into a plurality of pixel regions, wherein each of the pixel regions includes at least one pixel unit;

A characterization module, wherein the characterization module extracts features corresponding to each pixel region based on the pixel unit of the pixel region; and

An area dividing module, wherein the area planning module identifies and divides the area of the image according to the characteristics of the pixel area.

According to an embodiment of the present invention, the harvester further includes a positioning device and a navigation system, the positioning device and the navigation system are provided on the harvester host, wherein the positioning device acquires the harvester Position information of the host, wherein the navigation system provides navigation information for the grain processing body.

According to an embodiment of the present invention, the path planning system further includes:

An operation area setting module, wherein the operation area setting module sets the operation area of the farmland and the operation boundary obtained from the boundary area of the farmland; and

A driving path planning module, wherein the positioning information based on the harvester host, the image processing system recognizes the area planning information of the image, and the navigation information of the navigation system to obtain at least one driving planning path.

According to an embodiment of the present invention, the harvester host includes a vehicle body, at least one operating system provided on the vehicle body, and a driving control system, the vehicle body drives the operation system to operate, wherein the The driving control system controls the operation of the vehicle body and the operating parameters of the operating system.

According to an embodiment of the present invention, the driving control system acquires the information of the image captured by the image acquisition device recognized by the image processing system, automatically controls the driving route of the vehicle body and controls the operation of the operating system Parameters to achieve unmanned automatic driving and harvesting operations.

According to another aspect of the present invention, the present invention further provides an automatic driving method of a harvester, wherein the automatic driving method includes the following steps:

(a) Acquire at least one image, and identify the area and boundary of the farmland in the image;

(b) based on the identification information, plan at least one travel planning path; and

(c) Control the main machine of the harvester to automatically travel according to the travel planning path.

According to an embodiment of the present invention, step (a) of the above-mentioned automatic driving method further includes: identifying information of corresponding crops in the farmland in the image, wherein the information of the crops includes types of crops, height of crops, and fullness of particles And other information.

According to an embodiment of the present invention, step (b) of the above-mentioned automatic driving method further includes steps:

(b.1) Identify the area and boundary of the farmland corresponding to the image; and

(b.2) Plan at least one travel planning route based on the identified area.

According to an embodiment of the present invention, the step (b.1) of the above-mentioned automatic driving method further includes the steps of segmenting the image by using an image segmentation technique, and identifying a region that divides the image.

According to an embodiment of the present invention, in step (b) of the above-mentioned automatic driving method, the image processing system uses image segmentation technology to segment the image information, and recognizes that the area dividing the image is the unworked area, The operated area and the field boundary area.

According to an embodiment of the present invention, step (b.1) of the automatic driving method further includes the following steps:

Dividing the image into a plurality of the pixel regions, and normalizing the pixel values of the pixel regions to an array;

Extract the features of the pixel region corresponding to each array; and

Based on the feature corresponding to the pixel region, the classification label of the image is output.

According to an embodiment of the present invention, the step (b.2) of the above-mentioned automatic driving method further includes the step of: based on the positioning information of the harvester host, the area planning information of the image, and the navigation information of the navigation system, Plan out the driving plan path.

According to an embodiment of the present invention, the above-mentioned automatic driving method further includes: step (b.3) comparing whether the area division and the area boundary range identified by the image processing system are consistent with the previous area boundary range, if not , The area division and the area boundary range corresponding to the image are adjusted, and if the consistency can be maintained, the area division and the boundary range are kept unchanged.

According to an embodiment of the present invention, the above-mentioned automatic driving method further includes the step of: (d) adjusting the operating parameters of the operating system of the harvester host based on the identification information of the image.

Through the understanding of the following description and drawings, further objects and advantages of the present invention will be fully realized.

These and other objects, features and advantages of the present invention are fully reflected in the following detailed description, drawings and claims.

BRIEF DESCRIPTION

FIG. 1 is a system schematic diagram of a harvester according to the first preferred embodiment of the present invention.

2 is a schematic diagram of image acquisition of the harvester according to the above-mentioned preferred embodiment of the present invention.

FIG. 3A is a schematic diagram of an image acquired by the harvester according to the above preferred embodiment of the present invention.

3B is a schematic diagram of another image acquired by the harvester according to the above-described preferred embodiment of the present invention.

3C is a schematic diagram of another image acquired by the harvester according to the above-described preferred embodiment of the present invention.

4 is a schematic diagram of an image processing system of the harvester according to the above preferred embodiment of the present invention for dividing and identifying the image area.

5A is a schematic diagram of the image processing system of the harvester according to the above preferred embodiment of the present invention dividing the image area.

5B is a system block diagram of the image processing system of the harvester according to the above-described preferred embodiment of the present invention.

6 is a schematic diagram of the image processing system of the harvester according to the above preferred embodiment of the present invention extracting the image region feature recognition.

7 is a schematic diagram of the area division of the image processing system of the harvester according to the above-mentioned preferred embodiment of the present invention outputting the image.

FIG. 8 is a schematic diagram of the change of the boundary division of the area division of the image output system of the harvester according to the above preferred embodiment of the present invention.

9 is a schematic diagram of an automatic driving scene of the harvester according to the above preferred embodiment of the present invention.

10 is a system schematic diagram of a path planning system of the harvester according to the above preferred embodiment of the present invention.

11A is a schematic diagram of farmland path planning generated by the path planning system of the harvester according to the above-described preferred embodiment of the present invention.

FIG. 11B is a schematic diagram of the driving path adjusted by the path planning system of the harvester according to the above preferred embodiment of the present invention.

12 is a schematic diagram of the overall structure of a harvester with an image acquisition device according to a second preferred embodiment of the present invention.

13 is a schematic diagram of an image captured by the image acquisition device of the harvester according to the above preferred embodiment of the present invention.

14 is a schematic structural view of the image acquisition device of the harvester according to the above-described preferred embodiment of the present invention, wherein the image acquisition device is implemented as a mechanical pan-tilt device.

15 is a schematic diagram of the installation position of the image acquisition device of the harvester according to the above-described preferred embodiment of the present invention.

16A is a schematic diagram of an image processing system of the harvester according to the above-mentioned preferred embodiment of the present invention identifying a farmland area in an image captured by the image acquisition device.

16B is a schematic diagram of an image processing system of the harvester according to the above preferred embodiment of the present invention identifying crops in an image captured by the image acquisition device.

FIG. 17 is a schematic diagram of another optional implementation manner of the image acquisition device of the harvester according to the above-described preferred embodiment of the present invention, wherein the image acquisition device is implemented as an electronic pan/tilt device.

detailed description

The following description is used to disclose the present invention to enable those skilled in the art to implement the present invention. The preferred embodiments in the following description are only examples, and those skilled in the art may think of other obvious modifications. The basic principles of the present invention defined in the following description can be applied to other embodiments, modifications, improvements, equivalent solutions, and other technical solutions without departing from the spirit and scope of the present invention.

Those skilled in the art should understand that in the disclosure of the present invention, the terms "portrait", "horizontal", "upper", "lower", "front", "rear", "left", "right", " The orientation or positional relationship indicated by "vertical", "horizontal", "top", "bottom", "inner", "outer", etc. is based on the orientation or positional relationship shown in the drawings, which is only for the convenience of describing the present invention And simplify the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operate in a specific orientation, and therefore the above terms should not be construed as limiting the invention.

It can be understood that the term "a" should be understood as "at least one" or "one or more", that is, in one embodiment, the number of an element can be one, and in other embodiments, the The number can be more than one, and the term "one" cannot be understood as a limitation on the number.

With reference to FIGS. 1 to 9 of the accompanying drawings of the description of the present invention, a harvester and its automatic driving method according to the first preferred embodiment of the present invention are disclosed and explained in the following description, wherein the harvester can be It is implemented as crop harvester equipment with grain processing functions, vegetable, fruit harvesting equipment, lawn mowing equipment, and other types of harvesting devices. It can be understood that the type of harvester described in the present invention is only of an exemplary nature, not a limitation. The harvester acquires at least one image in the surroundings, and processes the area type of the farmland in the image based on visual recognition, and divides various area types and boundaries of the farmland in the image.

The harvester divides the type and boundary of each area according to the division, wherein the area type of the farmland divided by the harvester includes at least one operated area 100, at least one unoperated area 200, and at least one field boundary area 300, and the harvest According to the type of the divided area, the aircraft plans the walking route of the vehicle by the navigation system to realize unmanned automatic driving and unmanned automatic driving.

It is worth mentioning that, in automatic driving mode, in order to identify and obtain the route of the vehicle walking in the automatic driving mode, accurate vehicle positioning information needs to be obtained, usually high-precision satellite positioning information is required, and the self-driving car needs to update the road at all times Information such as obstacles, road vehicles, and pedestrians on the road to realize the automatic driving function under high-speed operation. The image acquired by the harvester of the present invention is image data information corresponding to crop grains in farmland, wherein the image is an image of the periphery of the vehicle acquired based on the current position of the vehicle. The harvester does not need satellite positioning information with too high accuracy, and only needs satellite positioning with ordinary meter accuracy (GPS positioning or Beidou positioning, etc.). Correspondingly, the image acquired and processed by the harvester is different from the self-driving car, therefore, the path planning and driving manner formed by the harvester are also different. It can be understood that the harvester of the present invention recognizes the area of the farmland based on vision and the automatic driving function is different from the recognition mode of the automatic driving car.

As shown in FIGS. 1 and 2, the harvester acquires at least one image around the area, wherein the harvester divides the area type corresponding to the farmland and the boundary between the areas according to the acquired image recognition. The harvester obtains the images around the harvester by fixed-point photographing, video shooting, mobile photographing, and the like. It can be understood that the manner in which the harvester acquires the image is only exemplary in nature, and not limiting. The harvester includes a harvester host 10 and at least one image acquisition device 20, wherein the image acquisition device 20 acquires at least one image around the harvester host 10.

Preferably, the image acquisition device 20 is provided on the harvester host 10, wherein the image acquisition device 20 acquires the images around the harvester host 10 by way of photographing or video shooting. More preferably, the image acquisition device 20 is provided in front of the harvester host 10, wherein the image acquisition device 20 can acquire the image in front of the harvester host 10 in real time, wherein the harvester host 10 The travel route is set based on the area identified by the image information captured by the image acquisition device 20. It is worth mentioning that the image captured by the image acquisition device 20 is based on the image within the field of view of the harvester host 10. In other words, the image acquisition device 20 acquires an image based on the direction of the field of view of the harvester host 10, and adjusts the traveling direction of the harvester host 10 according to the position where the image acquisition device 20 is mounted to the harvester host 10.

Furthermore, the image acquisition device 20 captures the vision of the world in the driving direction of the harvester host 10, wherein the image may be a two-dimensional planar image or a three-dimensional stereoscopic image that is captured. It can be understood that the type of the image captured by the image acquisition device 20 is merely an example, not a limitation.

It is worth mentioning that in the first preferred embodiment of the present invention, the harvester host 10 is implemented as a grain harvester device, wherein the harvester host 10 is controlled to travel to an unworked area of farmland 200 performs a harvesting operation to harvest crops in the non-operational area 200, such as rice, wheat, corn, etc. The harvester host 10 performs automatic driving in the field according to the area divided by the image obtained by the image obtaining device 20, and unmanned automatic driving. It can be understood that the type of the harvester main machine 10 is merely an example, not a limitation.

As shown in FIGS. 3A to 3C, during the driving process of the harvester main unit 10, the image acquisition device 20 acquires images around the harvester main unit 10 in real time. FIG. 3A shows the image captured by the image acquisition device 20 when the harvester host 10 is used as a grain harvester. The area in the farmland is divided into at least one unharvested area 100a, at least one harvested area 200a, and at least one field boundary area 300a according to whether the cereal is harvested, wherein the harvested area 200a is an area where crops have been harvested, wherein The original crops in the harvested area 200a are harvested. The unharvested area 100a is an area where crops still exist, and there are still growing crops in the unharvested area 100a. The field boundary area 300a is a ridge in the farmland that separates the interval between crops, an outer boundary around the farmland, and an area where obstacles exist in the farmland, wherein the field boundary area 300a is not planted with crops.

FIG. 3B shows the image captured by the image acquisition device 20 when the harvester host 10 is used as a mowing device. The area in the farmland is divided into at least one unharvested area 100b, at least one harvested area 200b, and at least one field boundary area 300b according to whether the grain is cultivated, wherein the unharvested area 100b represents an area that has not been harvested crops, so The harvested area 200b represents an area where crops have been cultivated, and the field boundary 300b is an outer boundary that separates the ridges where crops are planted and the periphery of the farmland, and the area where obstacles exist in the farmland.

FIG. 3C shows the image captured by the image acquisition device 20 when the harvester host 10 is used as a stalk plant or fruit harvesting device, such as a vegetable harvester device. The area in the farmland is divided into at least one unharvested area 100c, at least one harvested area 200c, and at least one field boundary area 300c according to whether the grain is sprayed. The unharvested area 100c represents an area where crops have not yet been harvested, the harvested area 200c represents an area of crops that have been harvested, the field boundary 300b is an outer boundary that separates the ridges where crops are planted and the periphery of the farmland, and There are obstacles in the farmland.

As shown in FIGS. 1 and 4, the images acquired by the image acquisition device 20 are identified by the image segmentation recognition technology to the unoperated area 100, the operated area 200, and the field boundary area 300, and distinguished The boundary between the areas. The harvester further includes an image processing system 30, wherein the image processing system 30 recognizes the unworked from the image using image segmentation recognition technology based on the image of the farmland acquired by the image acquisition device 20 The area 100, the operated area 200, and the field boundary area 300.

It can be understood that the image processing system 30 uses image segmentation recognition technology to identify the areas and boundaries in the image that are used to represent the areas and boundaries of the farmland in front of the harvester main machine 10 traveling. Based on the regions and boundaries identified by the image processing system 30 using image segmentation recognition technology, the harvester main unit 10 is controlled to travel and perform operations in an unworked area of farmland. For example, a harvester device, the image acquisition device 20 provided at the front end of the harvester device acquires an image of farmland in front of the harvester device, wherein the image captured by the image acquisition device 20 is divided and recognized by the image processing system 30 To identify the unworked area 100, the worked area 200, and the field boundary area 300 of the farmland that divides the farmland in the traveling direction of the harvester device. The harvester host 10, which is the host of the harvester device, plans the vehicle travel path and harvesting operation based on the area and boundary recognized by the image processing system 30.

It can be understood that the image processing system 30 uses image segmentation and recognition technology to identify the types of crops in the image provided by the image acquisition device 20, the height of the crops, and the fullness of crop particles. The image processing system 30 can determine whether the crop has been harvested based on the identified type of crop in the image and the height of the crop, and can be used to adjust the job based on the fullness information of the identified crop particles in the image parameter. In other words, the image processing system 30 can identify the area type and boundary of the farmland according to the image provided by the image acquisition device 20, and can also identify the type, height, grain fullness, and crop maturity of the crops in the farmland.

It is worth mentioning that the image processing system 30 is selected from any of the segmentation recognition methods selected from threshold-based segmentation methods, area-based segmentation methods, edge-based segmentation methods, and specific theory-based segmentation methods. The image acquired by the acquiring device 20 performs segmentation recognition to identify regions and boundaries in the image. Preferably, the image processing system 30 utilizes a deep learning algorithm to recognize the image segmentation and perform area division and boundary definition on the image. In other words, the image processing system 30 uses a deep learning algorithm to identify the area and boundary of the corresponding farmland in the image for the harvester host to travel and perform operations based on the identified area and boundary. More preferably, the deep learning algorithm used by the image processing system 30 is a convolutional neural network algorithm image segmentation recognition technology to identify the unoperated area 100, the operated area 200, and the Described field boundary area 300.

It is worth noting that the processing algorithm utilized by the image processing system 30 is merely exemplary, not limiting. Therefore, the image processing system 30 can also use other algorithms to segment and identify the acquired image to identify the area and boundary of the farmland in the image.

As shown in FIGS. 5A and 6, the image processing system 30 divides the image acquired by the image acquisition device 20 into a plurality of pixel regions 301, each of which includes at least one pixel unit. It can be understood that the image corresponds to an area around the harvester main body 10, and accordingly, the pixel area 301 of the image corresponds to image information of a specific area of farmland or crops in the farmland being photographed . Each pixel region 301 formed by division is subjected to a normalization process, so that the pixel unit of the pixel region 301 is normalized into a numerical value or an array having a size corresponding to the pixel value. In other words, the image processing system 30 normalizes the divided pixel regions 301 into corresponding numerical values or arrays for the image processing system to extract the features of the image and divide the regions.

The image processing system 30 extracts image features corresponding to the pixel region 301 based on the array corresponding to each pixel region 301. The image processing system 30 obtains image features corresponding to the pixel region 301 according to the array corresponding to the pixel region 301. When the image processing system 30 uses a convolutional neural network algorithm, such as a two-dimensional convolutional neural network, the input layer of the convolutional neural network corresponds to the corresponding two-dimensional array or three-dimensional array in the pixel region 301. The hidden layer of the convolutional neural network performs feature extraction on the array of the input layer, and performs feature selection and information filtering after feature extraction. The convolutional neural network outputs a classification label of the pixel region 301 based on the features corresponding to the array, wherein the classification labels respectively correspond to the unoperated region 100, the operated region 200, and the field boundary region 300.

As shown in FIGS. 6 and 7, the image processing system 30 recognizes the region features corresponding to the pixel region 301 by extracting the features of the array corresponding to the pixel region 301, wherein the features corresponding to the pixel region 301 It mainly includes the height characteristics of crop plants, the interval of crop plants in farmland, the color of crops, the color of farmland land, the characteristics of crop types, the characteristics of farmland land, the fullness of crop particles, and the number of crop particles. The image processing system 30 outputs a classification label corresponding to the pixel area 301 according to the extracted features, wherein the classification label correspondingly identifies the area type and the boundary line corresponding to the pixel area 301 based on the feature information.

As shown in FIG. 5B, the image processing system 30 includes an image segmentation module 31, a characterization module 32, and an area division module 33. The image segmentation module 31 acquires an image captured by the image acquisition module 20, and performs image segmentation processing to form a plurality of the pixel regions 301, where each of the pixel regions 301 includes at least one pixel unit. The feature module 32 uses a deep learning algorithm to extract the feature type corresponding to the pixel region 301, and select features and filter information. The area dividing module 33 divides the image based on the features corresponding to the pixel area 301 extracted by the characterization module 32 to generate the corresponding unoperated area 100, the operated area 200, and the field The classification label of the boundary area 300.

Preferably, the image segmentation module 31 divides the image into a plurality of pixel regions 301, wherein each pixel region 301 has the same size, shape and range. It can be understood that the image segmentation module 31 may also perform segmentation according to the image pixel threshold size. In other words, the size, shape, and range of the pixel region 301 segmented by the degraded segmentation module 31 may be different. More preferably, when the characterization module 32 of the image processing system 30 adopts a convolutional neural network algorithm, the pixel region 301 divided by the image division module 31 is a single pixel unit.

The characterization module 32 includes a pixel processing module 321, a feature extraction module 322, and a feature output module 323, wherein the pixel processing module 321 processes the array corresponding to the pixel units in the pixel region 301. In other words, the pixel processing module 321 normalizes the pixel area 301 into an array suitable for processing. The feature extraction module 322 inputs the array of the pixel region 301 processed by the pixel processing module 321, extracts the feature type corresponding to the array, and selects the features, and filters the information to retain the available data and discharge Disturb the data, thus making the feature extraction results more prepared. The feature output module 323 outputs the features extracted by the feature extraction module 322 in combination with the feature extraction module 322, and combines the features output by the feature output module 323 by the area division module 33 to generate the Classification label.

The area dividing module 33 divides each area corresponding to the image and sets an area boundary based on the features corresponding to the pixel area 301 extracted by the characterization module 32. Correspondingly, the area dividing module 33 further includes an area dividing module 331 and a boundary dividing module 332, wherein the area dividing module 331 divides different areas according to the characteristics of the pixel area 301, wherein the boundary dividing module 332 Divide the boundary range corresponding to the area, so as to identify the range of the area.

During driving of the harvester host 10 of the harvester, the image acquisition device 20 acquires images in the field of view in front of the harvester host 10 in real time. Accordingly, the image processing system 30 acquires the image captured by the image acquisition device 20 in real time, and uses image segmentation and recognition technology to identify the area division and area boundary range of the image corresponding to the farmland. When the area division and area boundary range identified by the image processing system 30 cannot be consistent with the previous area boundary range, the area division and area boundary range corresponding to the image are adjusted.

As shown in FIG. 8, during the driving operation of the harvester main machine 10, problems such as vibration and deviation in the driving direction are unavoidable. When the driving direction of the harvester main body 10 is deviated, or the area division changes due to the vibration of the vehicle, the image processing system 30 updates the area division and area boundary range corresponding to the image in real time.

As shown in FIG. 1, the harvester further includes a positioning device 40 and a navigation system 50, wherein the positioning device 40 is disposed on the harvester host 10 to obtain the position of the harvester host 10 information. Preferably, the positioning device 40 uses satellite positioning information to obtain the position information of the harvester host 10, such as a GPS or a Beidou positioning device. The navigation system 50 is provided in the harvester host 10, wherein the navigation system 50 navigates the harvester host 10 for the positioning of the harvester host 10 based on the positioning device 40 The information, the area planning information obtained by the image processing system 30, and the navigation information of the navigation system 50 realize unmanned automatic driving and operation.

It can be understood that the image of the farmland area division and area boundary range obtained by the image processing system 30 based on the image is updated to the navigation system 50 in real time to update the navigation information of the navigation system 50. Preferably, the navigation system 50 is implemented as an inertial integrated navigation system. It can be understood that the type of the navigation system 50 is merely an example, not a limitation, and therefore, the navigation system 50 may also be implemented as other types of navigation devices.

Accordingly, the harvester main body 10 of the harvester includes a vehicle body 11, an operating system 12 provided on the vehicle body 11, and a driving control system 13, wherein the operating system 12 is controlled by the vehicle The main body 11 drives and implements grain processing operations, such as harvesting operations. The driving control system 13 controls the running of the vehicle body 11 and controls the operation of the working system 12. It is worth mentioning that the driving control system 13 has an unmanned driving mode and an operating driving mode. When the harvester is in the unmanned driving mode, the driving control system 13 controls the vehicle body 11 to automatically operate and the operation of the operation system 12. Accordingly, when the harvester is in the operation driving mode, the driving control system allows the driver to manually operate the vehicle body 11 and control the operation of the operation system by manual operation.

In the first preferred embodiment of the present invention, the harvester is a harvester device, and the operating system 12 is implemented as a harvesting device. The driving control system 13 controls the running of the vehicle body 11 and controls the operation of the working system 12. In other words, the driving control system 13 controls the adjustment of the operating parameters of the operating system 12 while the vehicle body 11 is traveling. The driving control system 13 acquires the information of the image processing system 30 to identify the types of crops, the height of the crops, the degree of grain fullness, the diameter of the crop stalks, etc. in the image, and adjusts the operation based on the acquired information The operating parameters of the system 12 are, for example, adjusting the operating speed of the operating system 12, the width of the operation, the height of the operation, and adjusting the parameters of the off-force processing.

Figure 9 of the accompanying drawings of the present specification shows an embodiment of the unmanned driving and harvesting operation of the harvester in farmland. When the driving control system 13 of the harvester host 10 is in the unmanned driving mode, the driving control system 13 acquires the positioning information of the vehicle body 11 provided by the positioning device 40, the navigation system 50 The provided navigation information and the area identification information provided by the image processing system 30 further control the vehicle body 11 to travel in the unoperated area 100 of the farmland to complete the grain harvesting operation. During the driving operation of the harvester host 10, the image acquisition device 20 acquires the image of the vehicle body 11 in front of the vehicle in real time, wherein the image is recognized by the image processing system 30 using image segmentation recognition technology Out of the area and boundary. When the area division and boundary range obtained by the image processing system 30 are inconsistent with the previous area division and boundary range, the image processing system 30 replaces the original area division and boundary range, and updates the navigation of the navigation system 50 Data to enable the driving control system 13 to acquire new navigation information to adjust the driving and working route.

As shown in FIGS. 10 to 11B, the harvester is based on the position information of the harvester host 10 acquired by the positioning device 40, the area planning information of the image recognized by the image processing system 30, and the The navigation information of the navigation system 50 generates at least one planned route. The driving control system 13 of the harvester host 10 controls the driving of the vehicle body 11 and the operation of the operation system 12 according to the generated planned route. Accordingly, the harvester further includes a path planning system 60, wherein the path planning system plans at least one vehicle's travel path for the harvester host 10. The path planning system 60 obtains the positioning information of the positioning device 40, obtains the area planning information of the image recognized by the image processing system 30, and obtains the navigation information of the navigation system 50, and plans based on the obtained information The travel path of the vehicle body 11.

As shown in FIGS. 11A and 11B, the path planning system 60 identifies or sets at least one operation area 601 and operation boundary 602 corresponding to the farmland, where the operation area 601 is the largest operation of the harvester Range, wherein the driving control system 13 controls the vehicle body 11 to travel within the range of the working boundary 602. It can be understood that the working area 601 and the working boundary 602 can be identified by the image processing system 30 by identifying the field boundary area 300 in the image by identifying the maximum area range and boundary of the working area 601. Alternatively, the path planning system 60 sets the working area 601 of the harvester in a setting manner.

The path planning system 60 plans at least one travel path based on the outermost working boundary 602 of the working area 601. When the width of the work area 601 is greater than the work width of the work system 12, the path planning system 60 plans a "back"-shaped travel route, or an "S"-shaped travel route. It can be understood that the manner in which the driving route planned by the path planning system 60 is merely exemplary, and not limiting. Therefore, other driving routes can also be applied here.

Preferably, when the vehicle body 11 travels to the far end boundary of the work area 601, the path planning system 60 replans at least one travel path based on the range of the current non-work area 100. In other words, when the vehicle body 11 travels to the far-end boundary of the work area 601, the path planning system 60 updates the work area 601 and the work boundary 602 for the vehicle body 11, and according to the update A new driving path is planned for the working area 601 of.

It can be understood that the driving control system 13 controls the vehicle body 11 to travel according to the travel path planned by the path planning system 60, wherein the driving control system 13 controls the working system 12 to harvest the working area 401 The outermost crop. In other words, the driving control system 13 controls the working system 12 to harvest crops in the unworked area 100 based on the working boundary 602.

As shown in FIG. 10, the path planning system 60 of the harvester includes a work area setting module 61, a driving path planning module 62, and a path adjustment module 63. The working area setting module 61 recognizes the working area 601 and the working boundary 602 of the farmland based on the image processing system 30 identifying the boundary area of the farmland in the image; or setting the harvesting by setting The main machine 10 operates the operation area 601 and the operation boundary 602 in the farmland. Since the operation of the harvester host 10 causes the unoperated area 100 and the operated area 200 to change, the operation area setting module 61 updates the range of the operation area 601 and the boundary of the operation boundary 602 in real time in order to A new said unworked area 100 and said already worked area 200 are generated.

The driving path planning module 62 obtains at least one driving planning path based on the positioning information of the harvester host 10, the image processing system 30 identifying the area planning information of the image, and the navigation information of the navigation system 50 603, wherein the driving control system 13 controls the vehicle body 11 to travel according to the travel planning path 603. The path adjustment module 63 adjusts the driving direction of the harvester body 10 based on the information of the image processing system 30 identifying the crops of the image to form a vehicle driving path 604, wherein the vehicle driving paths 604 substantially coincide or Parallel to the driving planning path 603. When the image processing system 30 recognizes that the crops in the image need to adjust the harvesting range, the vehicle travel path generated by the path adjustment module 63 deviates from the travel planning path 603.

Referring to FIGS. 12 to 16B of the drawings of the specification of the present invention, a harvester with an image acquisition device according to a second preferred embodiment of the present invention will be explained in the following description. The harvester includes a harvester host 10 and at least one image acquisition device 20, wherein the image acquisition device 20 is disposed on the harvester host 10, and the image acquisition device 20 photographs the farmland where the harvester host 10 is located Images or video images for the harvester host 10 to control the driving direction and/or operating parameters based on the image or image information captured by the image acquisition device 20. The image acquisition device 20 captures information on the farmland around the farmland where the harvester host 10 is located based on the position of the harvester host 10. It can be understood that the image acquisition device 20 captures an image in the field of view, such as an image in the field of vision of the driver, so as to adjust the operating parameters of the harvester host 10 according to the captured image, such as adjusting the driving route, Travel speed, operating parameters, etc.

It is worth mentioning that the image acquisition device 20 is carried to the harvester host 10, wherein the image and image information captured by the image acquisition device 20 is transmitted to the harvester host 10 for the The harvester host 10 adjusts the operating parameters based on the information. The image acquisition device 20 is mounted on the harvester main body 10, wherein the image acquisition device 20 shoots a clear image when the harvester main body 10 shakes. In other words, the image acquisition device 20 is an anti-shake camera device, which can avoid mechanical vibration of the harvester main body 10 itself and shaking caused by unevenness of the world during shooting. Based on the image information captured by the image acquisition device 20, the harvester host 10 controls the travel path and operation parameters under the operation of an operator or automatically to realize the operation of the harvester. In other words, the harvester host 10 adjusts the operation and operation parameters based on the image information captured by the image acquisition device 20 to achieve precise operation and/or unmanned autonomous driving operation.

Preferably, in the second preferred embodiment of the present invention, the image acquisition device 20 is implemented as a pan-tilt camera device, wherein the image acquisition device 20 takes stable quality images in the case of vibration or jitter Or image.

In a second preferred embodiment of the present invention, the image acquisition device 20 is a mechanical pan/tilt device, wherein the image acquisition device 20 is mounted on the harvester main body 10 by mechanical connection, and the image The acquiring device 20 realizes the anti-shake captured image through a mechanical anti-shake method. It can be understood that the type of the image acquisition device 20 is only exemplary, not limiting. Therefore, other types of structures and installation methods can also be applied here.

As shown in FIGS. 12 to 14, the image acquisition device 20 includes a pan-tilt head 21 and at least one camera 22, wherein the pan-tilt head 21 installs the camera 22 to the harvester host 10, and the pan-tilt head 21 The installation position of the camera 22 is fixed. The bottom end of the pan-tilt head 21 is loaded onto the harvester main body 10, and the pan-tilt head 21 is fixed by the harvester main body 10, wherein the upper end of the pan-tilt head 21 is set to be connected to the camera 22. The camera 22 is supported by the gimbal 21 to maintain relative balance, so as to stably capture images or videos. The camera 22 shoots images or videos around the harvester host 10 under the support of the pan/tilt head 21, wherein the camera 22 shoots the harvester host 10 based on the installation position of the pan/tilt head 21 The image within the field of view. It can be understood that the camera 22 of the image acquisition device 20 acquires at least one visual image by taking pictures based on the position of the harvester host 10. In other words, the camera 22 of the image acquisition device 20 acquires the image based on the field of view of the harvester host 10, thereby avoiding image data caused by changes in the position of the camera device 20 and the harvester host 10 Inaccurate question.

The gimbal 21 of the image acquisition device 20 further includes a gimbal fixing piece 211 and at least one gimbal moving piece 212, wherein the gimbal moving piece 212 is movably connected to the gimbal fixing piece 211 . The pan-tilt fixing piece 211 is fixedly installed on the harvester main body 10, wherein the camera 22 is mounted to the pan-tilt moving piece 212. The pan-tilt moving part 212 of the pan-tilt 21 movably supports the camera 22, so that the camera 22 maintains a stable relative position when the harvester main body 10 shakes, so that a clear image is set.

In other words, when the harvester main unit 10 is vibrating or shaking, such as mechanical vibration or vibration generated when performing harvesting operations in farmland, the pan-tilt fixing member 21 of the pan-tilt 21 and the harvester host 10 Simultaneous ground shaking, wherein the pan-tilt moving member 212 of the pan-tilt 21 moves relative to the pan-tilt fixing member 211, neutralizing the vibration generated by the pan-tilt fixing member 211, thereby maintaining the position of the camera 22 Stability. In detail, the pan/tilt moving member 212 shakes or vibrates in the up-down direction, left-right direction, and front-to-back direction of the motion fixing member 211 to keep the camera 22 at a stable photographing position, thereby capturing a stable image information.

As shown in FIG. 14, the camera 22 of the image acquisition device 20 is provided to the pan-tilt moving part 212 of the pan-tilt 21, wherein the camera 22 is fixedly or movably mounted to the The pan/tilt moving part 212 of the pan/tilt 21. Preferably, the camera 22 is movably disposed on the moving member 212, wherein the camera 22 can rotate based on the upper end of the pan/tilt moving member 212 to capture images in different directions of the field of view. Optionally, the camera 22 is fixedly mounted to the upper end of the pan-tilt moving member 212, wherein the camera 22 takes an image within a specified field of view under the fixed support of the pan-tilt 21, such as a shooting place The image in the field of view in front of the main body 10 of the harvester will be described.

The camera 22 includes a camera body 221 and at least one camera driving device 222, wherein the camera driving device 222 drives the movement of the camera body 221 to capture images in different directions. The camera body 221 is movably disposed on the pan/tilt moving member 212, wherein the camera body 221 can be rotated in the up-down direction under the driving action of the camera driving device 222 to photograph the harvester host 10 Images of farmland and crops at distant and nearby locations. It can be understood that when the camera body 221 is driven to rotate downward by the camera driving device 222, the camera body 221 captures an image of the harvester main body 10 in order to clearly identify the crop information in the image . When the camera body 221 is driven by the camera driving device 222 to rotate upward, the camera body 221 captures an image of the harvester main unit 10 at a distance, so as to identify the working area and field of the farmland through the image Border area.

The camera driving device 222 drives the camera body 221 to rotate in the left-right direction, so that the camera body 221 can capture left and right images of the harvester main body 10 in order to identify the unworked area 100 of the farmland and Worked area 200, and field border area 300.

According to FIG. 15 of the accompanying drawings of the specification of the present invention, several optional installation methods and installation positions of the image acquisition device 20 installed on the harvester host 10 are shown. In a second preferred embodiment of the present invention, the image acquisition device 20 of the harvester is disposed at the front side position, upper top end, left side, right side, and rear of the harvester main body 10 Etc. It can be understood that, the installation position of the image acquisition device 20 is different, the captured image is different, and the information recognized from the image is also different. It can be understood that the image acquisition device 20 provided on the front side of the harvester host 10 takes an image in front of the harvester host 10, and when the harvester travels forward, the harvester host The image acquisition device 20 on the front side of 10 captures the working condition of the harvester main machine 10 in order to adjust the traveling path, working parameters, etc. of the harvester main machine 10 according to the photographed working condition.

The image acquisition device 20 provided on the rear side of the harvester host 10 captures an image behind the harvester host 10, and when the harvester travels forward, the image capture device 20 captures the image An image of the work area 200. Identify whether the harvesting operation of the harvester host 10 is qualified by identifying the image of the operated area 200 taken by the image acquisition device 20 on the rear side of the harvester host 10, so as to adjust the harvester host 10 job parameters. It can be understood that, through the image captured by the image acquisition device 20 provided on the rear side of the harvester host 10, the harvester host 10 recognizes whether the crops in the working area 200 are completely harvested, and whether crop particles are left behind, etc. . The harvester host 10 is adjusted the operation parameters according to the information identified in the image, thereby improving the harvesting operation. It is worth mentioning that, when driving in reverse, the image captured by the image acquisition device 20 provides the driver with a reverse image.

The image acquisition device 20 provided at the top end of the harvester main body 10 takes a long-distance image of the harvester main body 10 so as to recognize the work area of the farmland, the field boundary area, etc. based on the image. Preferably, the image acquisition device 20 provided at the top end of the main unit 10 of the harvester is a rotatable pan/tilt camera.

Correspondingly, the image acquisition device 20 provided on the left or right side of the harvester main machine 10 takes an image on the left or right side of the harvester main machine 10. Based on the image on the left or right side of the harvester host 10, the crops in the farmland in the image are identified so as to identify the unoperated area 100, the operated area 200, and the field boundary area 300.

As shown in FIGS. 12 and 14, the harvester further includes an image processing system 30, a positioning device 40, and a navigation system 50, wherein the image processing system 30, the positioning device 40, and the navigation The system 50 is installed in the harvester main body 10. The positioning device 40 acquires the position information of the harvester host 10 and transmits the acquired position information to the harvester host 10. The navigation system 50 provides navigation information to the harvester host 10 based on the positioning information of the positioning device 40. Based on the image of the farmland acquired by the image acquisition device 20, the image processing system 30 recognizes the unworked area 100, the operated area 200, and the field boundary area 300 from the image.

Preferably, the image processing system 30 recognizes the unworked area 100, the worked area 200, and the field boundary area 300 from an image using image segmentation recognition technology. It can be understood that the image processing system 30 may also identify the area and boundary information in the image in other ways. Therefore, in the second preferred embodiment of the present invention, the manner in which the image processing system 30 recognizes the image is merely exemplary, not limiting.

As shown in FIGS. 16A and 16B, the image processing system 30 recognizes the area of the farmland in the image, the boundary of the field, and the recognition based on the image around the harvester main body 10 taken by the image acquisition device 20 Information on the types of crops in the farmland, the height of the crops, the fullness of the grains, the thickness of the stems, etc.

It is worth mentioning that the image processing system 30 is selected from any of the segmentation recognition methods selected from threshold-based segmentation methods, area-based segmentation methods, edge-based segmentation methods, and specific theory-based segmentation methods. The image acquired by the acquiring device 20 performs segmentation recognition to identify regions and boundaries in the image. Preferably, the image processing system 30 uses a deep learning algorithm to recognize the image segmentation and perform area division and boundary definition on the image. In other words, the image processing system 30 uses a deep learning algorithm to identify the area and boundary of the corresponding farmland in the image for the harvester host 10 to travel and perform operations based on the identified area and boundary. More preferably, the deep learning algorithm used by the image processing system 30 is a convolutional neural network algorithm image segmentation recognition technology to identify the unoperated area 100, the operated area 200, and the Described field boundary area 300.

It is worth mentioning that the processing algorithm utilized by the image processing system 30 is merely exemplary, not limiting. Therefore, the image processing system 30 may also use other algorithms to perform segmentation recognition on the acquired image to identify the area and boundary of the farmland in the image.

It can be understood that the image processing system 30 is an image processor provided in the harvester host 10, wherein the image processor receives the image or image captured by the image acquisition device 20, and recognizes the Information in an image or video. According to the information recognized by the image processing system 30, the harvester host 10 correspondingly operates the parameters for controlling the driving path and adjusting the work.

As shown in FIGS. 12 and 14, the harvester host 10 further includes a vehicle body 11, an operating system 12 provided on the vehicle body 11, and a driving control system 13, wherein the operating system 12 is driven Connected to the vehicle body 11, the vehicle body 11 drives the working system 12 to drive the working system 12 to harvest crops. The driving control system 13 controls the running of the vehicle body 11 and controls the operation of the working system 12. It is worth mentioning that the driving control system 13 has an unmanned driving mode and an operating driving mode. When the harvester is in the unmanned driving mode, the driving control system 13 controls the vehicle body 11 to automatically operate and the operation of the operation system 12. Accordingly, when the harvester is in the operation driving mode, the driving control system allows the driver to manually operate the vehicle body 11 and control the operation of the operation system by manual operation.

In the second preferred embodiment of the present invention, the driving control system 13 controls the driving of the vehicle body 11 and controls the harvesting operation of the working system 12. In other words, the driving control system 13 controls the adjustment of the operating parameters of the operating system 12 while the vehicle body 11 is traveling. The driving control system 13 acquires the information of the image processing system 30 to identify the types of crops, the height of the crops, the degree of grain fullness, the diameter of the crop stalks, etc. in the image, and adjusts the operation based on the acquired information The operating parameters of the system 12 are, for example, adjusting the operating speed of the operating system 12, the width of the operation, the height of the operation, and adjusting the parameters of post-processing.

The operating system 12 further includes at least one harvesting device 121, at least one conveying device 122, and at least one post-processing device 123, wherein the conveying device 122 is configured to receive the crops harvested by the harvesting device 121, and The crop is transported to the post-processing device 123 for the post-processing device 123 to post-process the crop. The harvesting device 121, the conveying device 122, and the post-processing device 123 of the working system are respectively drivingly connected to the vehicle body 11, and the working system 12 is driven by the vehicle body 11 And operation of the harvesting device 121, the conveying device 122, and the post-processing device 123. It can be understood that the post-processing device 123 is implemented as a post-processing subsequent processing device for crops, for example, a grain harvester, the post-processing device 123 is a threshing device, and the post-processing device 123 in the mowing equipment is Implemented as a packaging device, when the harvester is a vegetable and fruit harvesting device, the post-processing device 123 is implemented as a vegetable and fruit screening and storage device.

The driving control system 13 controls the width, height, and speed of the harvesting device 121 according to the image information recognized by the image processing system 30. It can be understood that when the density of crops in the farmland is large, the information of the crops in the farmland captured by the image acquisition device 20 is recognized by the image processing system 30, wherein the driving control system 13 processes the crops according to the image The image information recognized by the system 30 controls any operating parameters such as reducing the harvesting width of the harvesting device 121, increasing the harvesting height, and reducing the harvesting speed.

The driving control system 13 controls the conveying speed, conveying power, etc. of the conveying device 122 according to the image information recognized by the image processing system 30. It can be understood that when the stalks of the crops in the farmland are thick, the height of the crops is high, and the density is large, the information of the crops in the farmland captured by the image acquisition device 20 is recognized by the image processing system 30, wherein the The driving control system 13 controls the operation parameters such as increasing the conveying speed of the conveying device 122 and increasing the conveying power according to the image information recognized by the image processing system 30.

The driving control system 13 controls the post-processing parameters of the post-processing device 123 according to the image information recognized by the image processing system 30. It can be understood that when the grains of the crops in the farmland are full, the size of the grains, the moisture content, the degree of dryness and wetness, and the types of crop fruits. It can be understood that the image processing system 30 recognizes the crop information of the crops in the farmland, wherein the driving control system 13 adjusts the post-processing according to the image information recognized by the image processing system 30 The post-processing parameters of the device, such as the blowing power, the rotation speed of the post-processing chamber and other parameters.

Referring to FIG. 17 of the drawings of the specification of the present invention, another alternative embodiment of an image acquisition device 20A of the harvester according to the second preferred embodiment of the present invention will be explained in the following description. In this optional embodiment, the image acquisition device 20A controls the angle of view and zoom of the lens inside the camera to prevent the camera from taking pictures.

Accordingly, the image acquisition device 20A includes a camera mounting mechanism 21A and at least one camera 22A, wherein the camera mounting mechanism 21A loads the camera 22A to the harvester host 10. The bottom end of the camera mounting mechanism 21A is loaded to the harvester main body 10, and the camera mounting mechanism 21A is fixed by the harvester main body 10, wherein the upper end of the camera mounting mechanism 21A is set to be connected to the Camera 22A. The camera 22A is supported by the camera mounting mechanism 21A to maintain relative balance so as to stably capture images or videos. The camera 22A shoots images or videos around the harvester main body 10 under the support of the camera mounting mechanism 21A, wherein the camera 22A shoots the harvester based on the mounting position of the camera mounting mechanism 21A An image within the visual field of the host 10.

It can be understood that the camera 22A of the image acquisition device 20A acquires at least one visual image by photographing based on the position of the harvester host 10. In other words, the camera 22A of the image acquisition device 20A acquires the image based on the field of view of the harvester host 10, thereby avoiding image data caused by changes in the position of the camera device 20A and the harvester host 10 Inaccurate question.

According to another aspect of the present invention, the present invention further provides an automatic driving method of a harvester, wherein the automatic driving method includes the following method steps:

(b) based on the identification information, plan at least one travel planning path 603; and

(c) Control the harvester host 10 to automatically travel according to the travel planning path 603.

In the above automatic driving method step, the driving control system 13 controls the driving and operation of the harvester main body 10 based on the area information and the field boundary recognized by the image processing system 30.

Step (a) of the above-mentioned automatic driving method further includes: recognizing the information of the corresponding crops in the farmland in the image, wherein the information of the crops includes the types of crops, the height of the crops, the degree of grain fullness and the like.

Step (b) of the above-mentioned automatic driving method further includes steps:

(b.2) Plan at least one travel planning path 603 based on the identified area.

The step (b.1) of the above-mentioned automatic driving method further includes the steps of segmenting the image using an image segmentation technique, and identifying a region that divides the image.

In step (a) of the above-mentioned automatic driving method, based on the position and driving direction of the harvester main body 10, image information around the harvester main body 10 is photographed in real time. In other words, the image acquisition device 20 captures an image near the position of the harvester main body 10 in real time.

In step (b) of the above-mentioned automatic driving method, the image processing system divides the image information using an image segmentation technique, and recognizes that the area dividing the image is the unoperated area 100, the operated area 200,和此田界区区300。 The field boundary area 300. Accordingly, the step (b.1) of the automatic driving method further includes the following steps:

Dividing the image into a plurality of pixel regions 301, and normalizing the pixel values of the pixel regions 301 to an array;

Extract the features of the pixel region 301 corresponding to each array; and

Based on the feature corresponding to the pixel region 301, the classification label of the image is output.

It can be understood that the classification label corresponds to the unoperated area 100, the operated area 200, and the field boundary area 300.

The step (b.2) of the above-mentioned automatic driving method further includes the steps of: based on the positioning information of the harvester host 10, the image processing system 30 recognizes the area planning information of the image, and the navigation information of the navigation system 50 To get the driving plan path 603.

The step (b.2) of the above-mentioned automatic driving method further includes the step of adjusting the driving direction of the harvester body 10 based on the information of the image processing system 30 identifying the crops in the image to form a vehicle driving path 604.

The above-mentioned automatic driving method further includes: step (b.3) comparing whether the area division and the area boundary area identified by the image processing system 30 are consistent with the previous area boundary area, and if the consistency cannot be maintained, adjusting the image correspondence If the area division and area boundary range can be kept the same, the area division and boundary area will remain unchanged.

Correspondingly, in the above method step (c), the driving control system 13 according to the positioning information of the harvester host 10, the regional planning information of the farmland obtained by the image processing system 30, and the navigation information To control the vehicle body 11 of the harvester host 10 to travel.

In the above-mentioned automatic driving method, it further includes the step of: (d) adjusting the operating parameters of the operating system 12 of the harvester main machine 10 based on the identification information of the image.

Those skilled in the art should understand that the embodiments of the present invention shown in the above description and the accompanying drawings are only examples and do not limit the present invention. The object of the present invention has been completely and effectively achieved. The functions and structural principles of the present invention have been shown and described in the examples. Without departing from the principles, the embodiments of the present invention may have any variations or modifications.

Claims

A harvester, characterized by including:

One harvester host;

At least one image acquisition device, wherein the image acquisition device is provided on the harvester host, and the image acquisition device captures images around the harvester host, and

An image processing system, wherein the image processing system recognizes farmland information in the image based on an image captured by the image acquisition device, wherein the harvester host automatically determines the farmland information identified by the image processing system Control driving.
The harvester according to claim 1, wherein the harvester further comprises a path planning system, wherein the path planning system plans at least one driving planning path based on the farmland information identified by the image processing system, wherein The main engine of the harvester controls driving according to the travel planning path planned by the path planning system.
The harvester according to claim 1, wherein the image processing system uses image segmentation recognition technology to identify the information of the farmland in the image, and plans the area of the farmland in the image based on the identified information.
The harvester according to claim 3, wherein the image processing system uses image segmentation and recognition technology to identify the crop information in the image for the harvester host to automatically adjust operating parameters based on the identified information.
The harvester according to claim 1, wherein the image acquisition device is an anti-shake PTZ camera device, the image acquisition device is loaded on the harvester host, and a photograph is taken based on the position of the harvester host Take an image around the harvester mainframe.
The harvester according to claim 4, wherein the image acquisition device is an anti-shake PTZ camera device, the image acquisition device is loaded on the harvester host, and a photograph is taken based on the position of the harvester host Take an image around the harvester mainframe.
The harvester according to claim 6, wherein the image acquisition device is a mechanical anti-shake gimbal device, the image acquisition device includes a gimbal and at least one camera, wherein the gimbal mounts the camera to all In the harvester host, the camera is installed on the gimbal, and the camera is supported by the gimbal to maintain balance.
The harvester according to claim 6, wherein the image acquisition device is an electronic pan-tilt device, and the image acquisition device prevents the camera of the image acquisition device from shaking by controlling the angle of view and zoom of the lens.
The harvester according to claim 7, wherein the image acquisition device is provided at the front of the harvester main machine, at the top of the harvester main machine, on the left, right, or all sides of the harvester main machine Describe the rear of the harvester main unit.
The harvester according to claim 8, wherein the image acquisition device is provided at the front of the harvester host, at the top of the harvester host, on the left, right, or Describe the rear of the harvester main unit.
The harvester of claim 3, wherein the image processing system further comprises:

An image segmentation module, wherein the image segmentation module divides the image into a plurality of pixel regions, wherein each of the pixel regions includes at least one pixel unit;

A characterization module, wherein the characterization module extracts features corresponding to each pixel region based on the pixel unit of the pixel region; and

An area dividing module, wherein the area planning module identifies and divides the area of the image according to the characteristics of the pixel area.
The harvester of claim 4, wherein the image processing system further comprises:

An image segmentation module, wherein the image segmentation module divides the image into a plurality of pixel regions, wherein each of the pixel regions includes at least one pixel unit;

A characterization module, wherein the characterization module extracts features corresponding to each pixel region based on the pixel unit of the pixel region; and

An area dividing module, wherein the area planning module identifies and divides the area of the image according to the characteristics of the pixel area.
The harvester according to claim 3, wherein the harvester further includes a positioning device and a navigation system, the positioning device and the navigation system are provided on the harvester main body, wherein the positioning device acquires The position information of the main machine of the harvester is described, wherein the navigation system provides navigation information for the grain processing body.
The harvester of claim 13, wherein the path planning system further comprises:

An operation area setting module, wherein the operation area setting module sets the operation area of the farmland and the operation boundary obtained from the boundary area of the farmland; and

A driving path planning module, wherein the positioning information based on the harvester host, the image processing system recognizes the area planning information of the image, and the navigation information of the navigation system to obtain at least one driving planning path.
The harvester according to claim 1, wherein the harvester main body includes a vehicle body, at least one operating system provided on the vehicle body, and a driving control system, the vehicle body drives the operation system to operate, The driving control system controls the operation of the vehicle body and the operating parameters of the operating system.
The harvester according to claim 15, wherein the driving control system acquires information of the image captured by the image acquisition device recognized by the image processing system, automatically controls the driving route of the vehicle body and controls the operation Operating parameters of the system to achieve unmanned automatic driving and harvesting operations.
An automatic driving method of a harvester is characterized in that the automatic driving method includes the following steps:

(a) Acquire at least one image, and identify the area and boundary of the farmland in the image;

(b) based on the identification information, plan at least one travel planning path; and

(c) Control the main machine of the harvester to automatically travel according to the travel planning path.
The automatic driving method according to claim 17, wherein the step of the automatic driving method

(a) further includes: identifying the information of the corresponding crops in the farmland in the image, wherein the information of the crops includes information such as the type of crops, the height of the crops, and the degree of grain fullness.
The automatic driving method according to claim 17, wherein the step (b) of the automatic driving method further comprises the step of:

(b.1) Identify the area and boundary of the farmland corresponding to the image; and

(b.2) Plan at least one travel planning route based on the identified area.
The automatic driving method according to claim 19, wherein the step (b.1) of the automatic driving method further includes the steps of segmenting the image using an image segmentation technique, and identifying a region dividing the image.
The automatic driving method according to claim 17, wherein in step (b) of the automatic driving method, the image processing system divides the image information using an image segmentation technique, and recognizes that the area dividing the image is the The unworked area, the operated area, and the field boundary area.
The automatic driving method according to claim 20, wherein the step (b.1) of the automatic driving method further includes the following steps:

Dividing the image into a plurality of the pixel regions, and normalizing the pixel values of the pixel regions to an array;

Extract the features of the pixel region corresponding to each array; and

Based on the feature corresponding to the pixel region, the classification label of the image is output.
The automatic driving method according to claim 22, wherein the step (b.2) of the above-mentioned automatic driving method further includes the step of: based on positioning information of the harvester host, area planning information of the image, and the navigation system Navigation information to plan the driving plan path.
The automatic driving method according to claim 23, wherein the automatic driving method further comprises: step (b.3) comparing whether the area division and the area boundary area identified by the image processing system are consistent with the previous area boundary area, If the consistency cannot be maintained, the area division and the area boundary range corresponding to the image are adjusted, and if the consistency can be maintained, the area division and the boundary range are kept unchanged.
The automatic driving method according to claim 17, wherein the automatic driving method further comprises the step of: (d) adjusting the operating parameters of the operating system of the harvester host based on the identification information of the image.