CN114926532A - On-line detection method and system for height of regenerated rice ear layer and harvester - Google Patents
On-line detection method and system for height of regenerated rice ear layer and harvester Download PDFInfo
- Publication number
- CN114926532A CN114926532A CN202210443594.8A CN202210443594A CN114926532A CN 114926532 A CN114926532 A CN 114926532A CN 202210443594 A CN202210443594 A CN 202210443594A CN 114926532 A CN114926532 A CN 114926532A
- Authority
- CN
- China
- Prior art keywords
- image
- height
- coordinate system
- harvester
- spike
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 235000007164 Oryza sativa Nutrition 0.000 title claims abstract description 60
- 235000009566 rice Nutrition 0.000 title claims abstract description 60
- 238000001514 detection method Methods 0.000 title claims abstract description 24
- 240000007594 Oryza sativa Species 0.000 title 1
- 241000209094 Oryza Species 0.000 claims abstract description 60
- 238000012545 processing Methods 0.000 claims abstract description 43
- 238000000034 method Methods 0.000 claims abstract description 33
- 238000005259 measurement Methods 0.000 claims abstract description 17
- 230000008569 process Effects 0.000 claims abstract description 8
- 238000013507 mapping Methods 0.000 claims abstract description 7
- 238000006243 chemical reaction Methods 0.000 claims description 13
- 230000008859 change Effects 0.000 claims description 12
- 230000000877 morphologic effect Effects 0.000 claims description 9
- 238000012216 screening Methods 0.000 claims description 8
- 238000012937 correction Methods 0.000 claims description 6
- 230000011218 segmentation Effects 0.000 claims description 4
- 239000000284 extract Substances 0.000 claims description 3
- 238000001914 filtration Methods 0.000 claims description 3
- 230000003287 optical effect Effects 0.000 claims description 3
- 238000013519 translation Methods 0.000 claims description 3
- 210000005069 ears Anatomy 0.000 claims description 2
- 238000009434 installation Methods 0.000 claims 1
- 238000003306 harvesting Methods 0.000 abstract description 7
- 238000000605 extraction Methods 0.000 description 3
- 230000001131 transforming effect Effects 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000008602 contraction Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000005096 rolling process Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
- G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
- G06T7/74—Determining position or orientation of objects or cameras using feature-based methods involving reference images or patches
-
- A—HUMAN NECESSITIES
- A01—AGRICULTURE; FORESTRY; ANIMAL HUSBANDRY; HUNTING; TRAPPING; FISHING
- A01D—HARVESTING; MOWING
- A01D45/00—Harvesting of standing crops
- A01D45/04—Harvesting of standing crops of rice
-
- A—HUMAN NECESSITIES
- A01—AGRICULTURE; FORESTRY; ANIMAL HUSBANDRY; HUNTING; TRAPPING; FISHING
- A01D—HARVESTING; MOWING
- A01D75/00—Accessories for harvesters or mowers
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
- G06T17/05—Geographic models
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/90—Determination of colour characteristics
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10024—Color image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10028—Range image; Depth image; 3D point clouds
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30181—Earth observation
- G06T2207/30188—Vegetation; Agriculture
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30244—Camera pose
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
- Y02A40/00—Adaptation technologies in agriculture, forestry, livestock or agroalimentary production
- Y02A40/10—Adaptation technologies in agriculture, forestry, livestock or agroalimentary production in agriculture
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Life Sciences & Earth Sciences (AREA)
- Environmental Sciences (AREA)
- Geometry (AREA)
- Software Systems (AREA)
- Remote Sensing (AREA)
- Computer Graphics (AREA)
- Image Analysis (AREA)
Abstract
The invention provides a method and a system for on-line detection of height of a regenerated rice ear layer and a harvester, wherein a color image and a depth image of regenerated rice in a region to be harvested right in front of the harvester are obtained, the height image is reconstructed and corrected by combining with an IMU (inertial measurement unit) of the combine harvester, so that the influence of the position and posture of a camera on the detection of the height information of the ear layer is eliminated, and the accuracy of on-line detection of the height information of the regenerated rice ear layer can be improved; and performing HSV color processing on the color image, extracting the lower side line of the circumscribed rectangle of the spike head area, mapping the lower side line in the elevation image, and calculating to obtain the height of the regenerated rice spike layer so as to guide the real-time regulation and control of the height of a header of the harvester and provide more accurate data support for the control of the stubble height in the process of harvesting operation of the regenerated rice.
Description
Technical Field
The invention belongs to the field of intelligent harvesting equipment, and particularly relates to an on-line detection method and system for the height of a regenerated rice ear layer and a harvester.
Background
The ratoon rice is a double harvest ratoon rice, namely, after the rice in one season is ripe, only the upper 2/3 part of the rice plant is cut off, and the dominant dormant bud is reserved to ensure the yield of the rice in the second season of the ratoon rice. Compared with the common ratoon rice, the ratoon rice only harvests ears and has higher stubble remaining. The driver can continuously control the lifting handle of the header to adjust the working height through visual observation, the operation effect, the efficiency and the like are greatly influenced by human experience, the stubble height in the harvesting operation process is difficult to control, and the stubble height is uniform.
Disclosure of Invention
Aiming at the technical problem, the invention provides an online detection method and system for the height of a regenerated rice ear layer and a harvester, wherein a color image and a depth image of regenerated rice in a region to be harvested right in front of the harvester are obtained, an elevation image is reconstructed and corrected by combining an IMU (inertial measurement unit) of the combine harvester, the influence of the pose of a camera on the detection of the height information of the ear layer is eliminated, and the online detection precision of the height information of the regenerated rice ear layer can be improved; and performing HSV color processing on the color image, extracting the lower side line of the circumscribed rectangle of the ear head area, mapping the lower side line in the elevation image, and calculating to obtain the height of the regenerated rice ear layer so as to guide the real-time regulation and control of the height of the harvester cutting table, thereby providing more accurate data support for the control of the stubble height in the process of the regenerated rice harvesting operation.
The technical scheme of the invention is as follows: a method for detecting the height of a regenerated rice spike layer on line comprises the following steps:
s1, collecting color images and depth images of the ratoon rice in a region to be collected right in front of the harvester through a binocular camera, and transmitting the color images and the depth images to a signal processing terminal;
s2, reconstructing an elevation image through the signal processing terminal, and correcting the reconstruction of the elevation image by combining the pose change of the camera acquired by the IMU inertial measurement unit of the harvester;
step S3, converting the corrected elevation image into an HSV color space by the signal processing terminal, and removing a reel area in the HSV color space;
step S4, the signal processing terminal extracts the spike head region on the processed S channel image of the HSV color space by the maximum inter-class variance method, and draws the external rectangle of the spike head region after the morphological characteristic screening and the opening and closing operation processing, and takes the lower line of the external rectangle as the lower boundary of the spike layer;
and step S5, the signal processing terminal maps the lower line in the reconstructed elevation image, and the height information of the spike layer can be obtained through calculation.
In the above scheme, in step S2, the elevation image reconstruction includes the following steps:
step S2.1 the depth image acquired by the binocular camera is firstly converted from an image pixel coordinate system to an image physical coordinate system:
X L ＝(u L -C x )dx，X R ＝(u R -C x )dx
Y L ＝(v L -C y )dy，Y R ＝(v R -C y )dy
wherein (X) L ,Y L ) Is the coordinate point of the lower left graph of the image physical coordinate system, (X) R ,Y R ) Is a coordinate point of a right graph under the physical coordinate system of the image (u) L ,v L ) Is in the image pixel coordinate systemLeft graph coordinate point, (u) R ,v R ) Is a right graph coordinate point, C, under an image pixel coordinate system x 、C y As camera optical center coordinates, dx represents the physical size of each pixel on the horizontal axis x, and dy represents the physical size of each pixel on the vertical axis y;
step S2.2 performs conversion from the image physical coordinate system to the camera coordinate system:
X C ＝B×X/D，Y C ＝B×Y/D，Z C ＝B×f/D
wherein (X) C ,Y C ,Z C ) As coordinates under the camera coordinate system, the parallax D ═ X L -X R ＝(u L -u R ) dx, B is the base line distance of the binocular camera, and f is the focal length of the camera;
step S2.3 performs conversion of the camera coordinate system to the world coordinate system:
wherein (X) w ,Y w ,Z w ) The coordinate under the world coordinate system is shown as alpha, the rotation angle from the camera coordinate system to the world coordinate system around the x axis is the pitch angle, the rotation angle from the camera coordinate system to the world coordinate system around the z axis is the roll angle, and T is the translation distance from the camera coordinate system to the world coordinate system.
In the foregoing scheme, the correcting the reconstruction of the elevation image in step (2) includes the following steps:
taking an initial plane as a reference plane of the whole operation process, wherein the initial pitch angle of the camera is a mounting angle alpha 0 The initial roll angle is 0 degrees, and the IMU measures the initial pitch angle alpha of the harvester 1 And roll angle beta 1 ；
When the harvester works, the pitch angle alpha of the harvester is monitored in real time through the IMU 2 And roll angle beta 2 Calculating the increment alpha of the pitch angle and the roll angle Δ 、β Δ In which α is Δ ＝α 2 -α 1 ，β Δ ＝β 2 -β 1 ；
If the increment is smaller than the set error value, the inclination angle does not exist in the front-back or left-right direction, the direction is not corrected, and the acquired image is directly processed in the next step;
if the pitch angle and roll angle increment are larger than the set error value, the pitch angle alpha and roll angle beta are calculated by considering that the inclination angle exists in the front-back or left-right direction, wherein the alpha is alpha 0 +α Δ ，β＝0°+β Δ And when the coordinate conversion is carried out on the depth image, the attitude correction is carried out by combining alpha and beta to obtain a corrected elevation image.
In the above scheme, in the step S3, removing the reel area in the HSV color space includes the following steps:
s3.1, performing pixel value conversion on each pixel in the image in an HSV color space,
v＝max
wherein r, g and b are pixel channel values in RGB color space, max is the maximum of r, g and b, min is the minimum of r, g and b, and h, s and v are pixel channel values in HSV color space;
s3.2, the reel is the largest in difference with the rest areas in the H channel, the reel area can be obtained through threshold segmentation, and the difference between the head of the regenerated rice and the background is increased through median filtering and image enhancement processing in the S channel;
and S3.3, setting pixel values of the divided reel region in the S channel after the image enhancement processing, and distinguishing the spike heads and the reel.
In the above solution, in step S3.3, the pixel value of the reel area obtained by dividing is 255, at this time, the ear is black, and the reel is white.
In the foregoing scheme, in S4, drawing a circumscribed rectangle of the spike head region, and taking a lower edge of the circumscribed rectangle as a lower boundary of the spike layer includes the following steps:
dividing the image into a background part and a foreground part, namely a spike head part, by a maximum inter-class variance method, then carrying out binarization on the image, carrying out opening operation and closing operation on the binarized image to eliminate noise points, extracting a spike head region by screening morphological characteristics, drawing a circumscribed rectangle of the spike head region, taking an average value of the lowest points of the region as a height value of a regenerated rice spike layer, and taking the average value as a lower line of the circumscribed rectangle.
A system for realizing the on-line detection method of the height of the regenerated rice ear layer comprises an image acquisition device, an IMU inertial measurement unit and a signal processing terminal;
the image acquisition device is used for acquiring a color image and a depth image of the ratoon rice in a region to be acquired right in front of the harvester and transmitting the color image and the depth image to the signal processing terminal;
the IMU inertial measurement unit is used for acquiring the pose change of the harvester and transmitting the pose change to the signal processing terminal;
the signal processing terminal is used for reconstructing the elevation image and correcting the reconstruction of the elevation image by combining the pose change acquired by the IMU inertial measurement unit. Converting the color image into an HSV color space, removing a reel area in the HSV color space, extracting a spike head area on an S channel image of the processed HSV color space by a maximum inter-class variance method, screening morphological characteristics, performing opening operation and closing operation, drawing an external rectangle of the spike head area, taking a lower edge of the external rectangle as a lower boundary of a spike layer, mapping the lower edge into a reconstructed elevation image, and calculating to obtain spike layer height information.
In the above scheme, the image acquisition device is a binocular camera.
In the scheme, the binocular camera is installed above the cab of the harvester.
In the above scheme, the signal processing terminal is a cab controller of the harvester or a remote computer.
A harvester comprises the system of the regenerated rice ear layer height online detection method.
Compared with the prior art, the invention has the beneficial effects that:
according to the invention, the image acquisition device is used for acquiring the color image and the depth image of the ratoon rice in the area to be acquired right in front of the harvester. And transforming a coordinate system of the depth image, reconstructing an elevation image, and correcting by combining an Inertial Measurement Unit (IMU) of the combine harvester, so that the influence of the pose of the harvester on the height information detection is eliminated, and the detection precision is improved. And performing HSV color processing on the color image, extracting the lower edge line of the circumscribed rectangle of the heading area, mapping the lower edge line in the elevation image, and calculating to obtain the height of the regenerated rice heading layer. And according to the ear layer height information obtained by the regenerated rice ear layer height online detection method, data support is provided for controlling the expansion and contraction of the header hydraulic cylinder in real time and adjusting the header height.
Drawings
FIG. 1 is a flow chart of a method for online detection of the height of a regenerated rice ear layer according to an embodiment of the present invention;
FIG. 2 is a left side schematic view of a combine harvester and binocular camera pose according to one embodiment of the present invention;
FIG. 3 is a front view of a combine harvester and binocular camera pose in accordance with one embodiment of the present invention;
FIG. 4 is an image of an H channel after the image is converted into an HSV color space in accordance with one embodiment of the present invention;
FIG. 5 is an image of one embodiment of the present invention with the reel area removed;
FIG. 6 is an image of the spike head after extraction according to one embodiment of the present invention;
FIG. 7 is an image of a lower edge line of a circumscribed rectangle of the ear region of an embodiment of the present invention.
Detailed Description
Reference will now be made in detail to embodiments of the present invention, examples of which are illustrated in the accompanying drawings, wherein like reference numerals refer to the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the drawings are illustrative and intended to be illustrative of the invention and are not to be construed as limiting the invention.
In the description of the present invention, it is to be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "front", "rear", "left", "right", "upper", "lower", "axial", "radial", "vertical", "horizontal", "inner", "outer", etc. indicate orientations and positional relationships based on those shown in the drawings, and are used merely for convenience in describing the present invention and for simplicity in description, and do not indicate or imply that the device or element so referred to must have a particular orientation, be constructed and operated in a particular orientation, and thus should not be considered as limiting the present invention. Furthermore, the terms "first", "second" and "first" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or to implicitly indicate the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of the present invention, "a plurality" means two or more unless specifically defined otherwise.
In the present invention, unless otherwise expressly specified or limited, the terms "mounted," "connected," "secured," and the like are to be construed broadly and can, for example, be fixedly connected, detachably connected, or integrally connected; can be mechanically or electrically connected; they may be connected directly or indirectly through intervening media, or they may be interconnected between two elements. The specific meanings of the above terms in the present invention can be understood according to specific situations by those of ordinary skill in the art.
Example 1
A system of a regenerated rice spike layer height online detection method comprises an image acquisition device, an IMU inertia measurement unit and a signal processing terminal; the image acquisition device is used for acquiring a color image and a depth image of ratoon rice in a region to be acquired right in front of the harvester and transmitting the color image and the depth image to the signal processing terminal; the IMU inertial measurement unit is used for acquiring the pose change of the image acquisition device and transmitting the pose change to the signal processing terminal; the signal processing terminal is used for reconstructing the elevation image and correcting the reconstruction of the elevation image by combining the pose change acquired by the IMU inertial measurement unit. Converting the color image into an HSV color space, removing a reel area in the HSV color space, extracting a spike head area on an S channel image of the processed HSV color space by a maximum inter-class variance method, screening morphological characteristics, performing opening operation and closing operation, drawing an external rectangle of the spike head area, taking a lower edge of the external rectangle as a lower boundary of a spike layer, mapping the lower edge into a reconstructed elevation image, and calculating to obtain spike layer height information.
Preferably, the image acquisition device is a binocular camera. The binocular camera is installed above the cab of the harvester.
Preferably, the signal processing terminal is a cab controller of the harvester or a remote computer.
According to the method for detecting the height of the regenerated rice ear layer on line, a binocular camera is fixedly arranged above a cab of a harvester, and a pitch angle alpha and a roll angle beta of the camera are adjusted, so that the camera can acquire a color image and a depth image of the regenerated rice in a region to be acquired right in front of the harvester. And (4) converting a coordinate system of the depth image, reconstructing an elevation image, and correcting by combining an IMU inertial measurement unit of the combine harvester to eliminate the influence of the pose of the camera. And performing HSV color processing on the color image, extracting the lower edge line of the circumscribed rectangle of the heading area, mapping the lower edge line in the elevation image, and calculating to obtain the height of the regenerated rice heading layer. And (3) according to the ear layer height information obtained by the regenerated rice ear layer height online detection method, providing data support for controlling the expansion of the header hydraulic cylinder in real time and adjusting the header height, so that the stubble height is consistent as much as possible.
As shown in fig. 1, the method for detecting the height of the regenerated rice spike layer on line specifically comprises the following steps:
step S1, the binocular camera acquires an image: collecting a color image and a depth image of the ratoon rice in a region to be collected right in front of the harvester through a camera, and transmitting the color image and the depth image to a signal processing terminal;
specifically, as shown in fig. 2, a binocular camera is installed above a cab of the harvester, an initial plane is used as a reference plane in the whole operation process, and an initial pitch angle and a roll angle are adjusted to ensure that the camera can acquire a color image and a depth image of the ratoon rice in an area to be harvested right in front of the harvester.
Step S2, transforming the coordinate system to reconstruct an elevation image: the elevation image is reconstructed through the signal processing terminal, and the reconstruction of the elevation image is corrected by combining the pose change of the camera acquired by the IMU inertial measurement unit of the harvester, so that the influences of pitching and rolling of a vehicle body caused by uneven ground surface during the operation of the harvester are eliminated;
specifically, the elevation image reconstruction method comprises the following steps:
step S2.1 the depth image acquired by the binocular camera is firstly converted from an image pixel coordinate system to an image physical coordinate system:
X L ＝(u L -C x )dx，X R ＝(u R -C x )dx
Y L ＝(v L -C y )dy，Y R ＝(v R -C y )dy
wherein (X) L ,Y L ) Is the coordinate point of the lower left graph of the image physical coordinate system, (X) R ,Y R ) Is a coordinate point of a right graph under the physical coordinate system of the image (u) L ,v L ) Is the lower left graph coordinate point of the image pixel coordinate system, (u) R ,v R ) Is a right graph coordinate point, C, under an image pixel coordinate system x 、C y For the coordinates of the optical center of the camera, dx represents the physical size of each pixel on the horizontal axis x, and dy represents the physical size of each pixel on the vertical axis y;
step S2.2, the conversion from the image physical coordinate system to the camera coordinate system is carried out:
X C ＝B×X/D，Y C ＝B×Y/D，Z C ＝B×f/D
wherein (X) C ,Y C ,Z C ) As coordinates under the camera coordinate system, the parallax D ═ X L -X R ＝(u L -u R ) dx, B is the base line distance of the binocular camera, and f is the focal length of the camera;
step S2.3 performs conversion of the camera coordinate system to the world coordinate system:
wherein (X) w ,Y w ,Z w ) The coordinate under the world coordinate system is defined as alpha, the rotation angle of the camera coordinate system to the world coordinate system around the x axis is a pitch angle, beta is the rotation angle of the camera coordinate system to the world coordinate system around the z axis is a roll angle, and T is the translation distance of the camera coordinate system to the world coordinate system.
The pose correction is carried out on the reconstruction of the elevation image, and the pose correction method comprises the following steps:
taking an initial plane as a reference plane of the whole operation process, wherein the initial pitch angle of the camera is a mounting angle alpha 0 The initial roll angle is 0 degrees, and the IMU measures the initial pitch angle alpha of the harvester 1 And roll angle beta 1 ；
When the harvester works, the pitch angle alpha of the harvester is monitored in real time through the IMU 2 And roll angle beta 2 Calculating the increment alpha of the pitch angle and the roll angle Δ 、β Δ In which α is Δ ＝α 2 -α 1 ，β Δ ＝β 2 -β 1 ；
If the increment is smaller than the set error value, the inclination angle does not exist in the front-back or left-right direction, the direction is not corrected, and the acquired image is directly processed in the next step.
As shown in fig. 2 and 3, if the pitch angle and roll angle increment are larger than the set error values, the pitch angle α and roll angle β are calculated assuming that there is a tilt angle in the front-rear or right-left direction, where α ═ α 0 +α Δ ，β＝0°+β Δ And when the coordinate conversion is carried out on the depth image, the attitude correction is carried out by combining alpha and beta to obtain a corrected elevation image.
Step S3, converting HSV color space and removing reel areas: the signal processing terminal converts the color image into an HSV color space, and removes a reel area in the HSV color space so as to eliminate the shielding influence of the reel;
in the step S3, removing the reel region in the HSV color space includes the following steps:
s3.1, performing pixel value conversion on each pixel in the image in an HSV color space, specifically:
v＝max
wherein r, g and b are three pixel channel values of red, green and blue in an RGB color space, max is the maximum of r, g and b, min is the minimum of r, g and b, and h, s and v are three pixel channel values of hue, saturation and brightness in an HSV color space respectively;
s3.2, as shown in the figure 4, the difference between the reel and the rest areas in the H channel is the largest, the reel area can be obtained through threshold segmentation, and the difference between the head of the regenerated rice spike and the background is increased through median filtering and image enhancement processing in the S channel;
and S3.3, as shown in FIG. 5, setting pixel values of a reel region obtained by segmentation in the S channel after the image enhancement processing, and distinguishing the ear heads from the reel, wherein preferably, the pixel value of the reel region is 255, the ear heads are black at the moment, and the reel is white, so as to eliminate the shielding influence of the reel.
Step S4, extracting the spike head area and drawing a lower side line of a circumscribed rectangle: the signal processing terminal extracts the spike head region on the processed S channel image of the HSV color space by a maximum inter-class variance method, and draws an external rectangle of the spike head region after morphological characteristic screening and opening, operation and closing operation processing, and takes the lower line of the external rectangle as the lower boundary of the spike layer;
specifically, drawing a circumscribed rectangle of the spike head area, and taking a lower edge line of the circumscribed rectangle as a lower boundary of the spike layer comprises the following steps:
dividing the image into a background and a foreground, namely a spike head part, according to the gray characteristic of the image by a maximum inter-class variance method, when an optimal threshold value is obtained, the difference between the two parts is maximum, then carrying out binarization on the image, carrying out opening operation and closing operation on the binarized image to eliminate noise points, then removing a false extraction area and accurately extracting the spike head area by screening morphological characteristics, as shown in figure 6, wherein the non-extraction area comprises the background part and the stem part of the spike head;
and drawing a circumscribed rectangle of the heading region, taking the average value of the lowest points of the region as the height value of the regenerated rice heading layer, and drawing the lower line of the circumscribed rectangle as shown in fig. 7.
And step S5, the signal processing terminal maps the lower line in the reconstructed elevation image, and the height information of the spike bed can be calculated and obtained so as to guide the height control of the header.
According to the method, a color image and a depth image of the ratoon rice in the advancing direction of the combined harvester are obtained through a binocular camera, the depth image is subjected to coordinate system conversion, an elevation image is reconstructed, correction is carried out by combining an IMU (inertial measurement unit) of the combined harvester, the influence of the pose of the camera is eliminated, HSV (hue, saturation and value) color processing is carried out on the color image, the lower edge line of a circumscribed rectangle of a heading area is extracted, the lower edge line is mapped in the elevation image, and the height of the ratoon rice heading layer is obtained through calculation. According to the method based on the combination of binocular vision and IMU, the influence of the pose of the camera on the height information detection is used for improving the detection precision, the online detection of the height information of the regenerated rice ear layer is realized, and an accurate input signal is provided for stubble height control in the process of harvesting operation of the regenerated rice.
Example 2
A harvester including the system of the method for detecting the height of the ear layer of the regenerated rice in embodiment 1 on line, thereby having the advantages of embodiment 1, and will not be described herein again.
It should be understood that although the present description has been described in terms of various embodiments, not every embodiment includes only a single embodiment, and such description is for clarity purposes only, and those skilled in the art will recognize that the embodiments described herein may be combined as suitable to form other embodiments, as will be appreciated by those skilled in the art.
The above-listed detailed description is only a specific description of possible embodiments of the present invention, and they are not intended to limit the scope of the present invention, and equivalent embodiments or modifications made without departing from the technical spirit of the present invention should be included in the scope of the present invention.
Claims (10)
1. An on-line detection method for the height of a regenerated rice ear layer is characterized by comprising the following steps:
s1, collecting color images and depth images of the ratoon rice in a region to be collected right in front of the harvester through a binocular camera, and transmitting the color images and the depth images to a signal processing terminal;
s2, reconstructing an elevation image through the signal processing terminal, and correcting the reconstruction of the elevation image by combining the pose change of the camera acquired by the IMU inertial measurement unit of the harvester;
step S3, converting the corrected elevation image into HSV color space by the signal processing terminal, and removing a reel area in the HSV color space;
step S4, the signal processing terminal extracts the spike head area on the processed S channel image of the HSV color space by a maximum inter-class variance method, and draws the external rectangle of the spike head area after the morphological characteristics are screened and the opening and closing operation are carried out, and takes the lower edge line of the external rectangle as the lower boundary of the spike layer;
and step S5, the signal processing terminal maps the lower line in the reconstructed elevation image, and the height information of the spike layer can be obtained through calculation.
2. The method for detecting the height of a regenerated rice panicle layer on-line as claimed in claim 1, wherein in step S2, the elevation image reconstruction comprises the steps of:
step S2.1 the depth image acquired by the binocular camera is firstly converted from an image pixel coordinate system to an image physical coordinate system:
X L ＝(u L -C x )dx，X R ＝(u R -C x )dx
Y L ＝(v L -C y )dy，Y R ＝(v R -C y )dy
wherein (X) L ,Y L ) Is the coordinate point of the lower left graph of the image physical coordinate system, (X) R ,Y R ) Is a coordinate point of a right graph under an image physical coordinate system, (u) L ,v L ) Is the coordinate point of the lower left graph of the image pixel coordinate system (u) R ,v R ) Is a right graph coordinate point, C, under an image pixel coordinate system x 、C y As camera optical center coordinates, dx represents the physical size of each pixel on the horizontal axis x, and dy represents the physical size of each pixel on the vertical axis y;
step S2.2, the conversion from the image physical coordinate system to the camera coordinate system is carried out:
X C ＝B×X/D，Y C ＝B×Y/D，Z C ＝B×f/D
wherein (X) C ,Y C ,Z C ) As coordinates in the camera coordinate system, the parallax D ═ X L -X R ＝(u L -u R ) dx, B is the base line distance of the binocular camera, and f is the focal length of the camera;
step S2.3 performs conversion of the camera coordinate system to the world coordinate system:
wherein (X) w ,Y w ,Z w ) The coordinate under the world coordinate system is shown as alpha, the rotation angle from the camera coordinate system to the world coordinate system around the x axis is the pitch angle, the rotation angle from the camera coordinate system to the world coordinate system around the z axis is the roll angle, and T is the translation distance from the camera coordinate system to the world coordinate system.
3. The method for detecting the height of the regenerated rice ear layer on line as claimed in claim 1, wherein the step (2) of correcting the reconstruction of the elevation image comprises the steps of:
taking an initial plane as a reference plane of the whole operation process, wherein the initial pitch angle of the camera is an installation angle alpha 0 The initial roll angle is 0 degrees, and the IMU measures the initial pitch angle alpha of the harvester 1 And roll angle beta 1 ；
When the harvester works, the pitch angle alpha of the harvester is monitored in real time through the IMU 2 And roll angle beta 2 Calculating the increment alpha of the pitch angle and the roll angle Δ 、β Δ In which α is Δ ＝α 2 -α 1 ，β Δ ＝β 2 -β 1 ；
If the increment is smaller than the set error value, the inclination angle does not exist in the front-back or left-right direction, the direction is not corrected, and the obtained image is directly processed in the next step;
if the pitch angle and the roll angle increment are larger than the set error value, the inclination angle exists in the front-back or left-right direction, and the pitch angle alpha and the roll angle beta are calculated, wherein alpha is alpha 0 +α Δ ，β＝0°+β Δ And when the coordinate conversion is carried out on the depth image, the attitude correction is carried out by combining alpha and beta to obtain a corrected elevation image.
4. The method for detecting the height of a regenerated rice ear layer on-line as claimed in claim 1, wherein the step S3 of removing the reel area in HSV color space comprises the steps of:
s3.1, performing pixel value conversion on each pixel in the image in an HSV color space,
v＝max
wherein r, g and b are red, green and blue pixel channel values in an RGB color space, max is the maximum of r, g and b, min is the minimum of r, g and b, and h, s and v are hue, saturation and lightness pixel channel values in an HSV color space respectively;
s3.2, the difference between the reel and the rest areas in the H channel is the largest, the reel area can be obtained through threshold segmentation, and the difference between the head of the regenerated rice spike and the background is increased through median filtering and image enhancement processing in the S channel;
and S3.3, setting pixel values of the divided reel region in the S channel after the image enhancement processing, and distinguishing the spike heads and the reel.
5. The method as claimed in claim 4, wherein the pixel value of the reel area obtained by dividing in step S3.3 is 255, and the ears are black and the reel is white.
6. The method for detecting the height of the ear layer of the regenerated rice as claimed in claim 1, wherein in step S4, drawing a circumscribed rectangle of the ear region, and taking the lower line of the circumscribed rectangle as the lower boundary of the ear layer comprises the following steps:
dividing the image into a background part and a foreground part, namely a spike head part, by a maximum inter-class variance method, then carrying out binaryzation on the image, carrying out opening operation and closing operation on the binary image to eliminate noise points, extracting a spike head region by screening morphological characteristics, drawing a circumscribed rectangle of the spike head region, taking an average value of the lowest points of the region as a height value of a regenerated rice spike layer, and taking the lower edge of the circumscribed rectangle.
7. A system for realizing the on-line detection method of the height of the regenerated rice ear layer in any one of claims 1 to 6 is characterized by comprising an image acquisition device, an IMU inertial measurement unit and a signal processing terminal;
the image acquisition device is used for acquiring a color image and a depth image of the ratoon rice in a region to be acquired right in front of the harvester and transmitting the color image and the depth image to the signal processing terminal;
the IMU inertial measurement unit is used for acquiring the pose change of the image acquisition device and transmitting the pose change to the signal processing terminal;
the signal processing terminal is used for reconstructing the elevation image and correcting the reconstruction of the elevation image by combining the pose change acquired by the IMU inertial measurement unit. Converting the color image into an HSV color space, removing a reel area in the HSV color space, extracting a spike head area on an S channel image of the processed HSV color space by a maximum inter-class variance method, screening morphological characteristics, performing opening operation and closing operation, drawing an external rectangle of the spike head area, taking a lower edge of the external rectangle as a lower boundary of a spike layer, mapping the lower edge into a reconstructed elevation image, and calculating to obtain spike layer height information.
8. The system of the on-line detection method for the height of the regenerated rice ear layer as claimed in claim 7, wherein the image acquisition device is a binocular camera; the binocular camera is installed above the cab of the harvester.
9. The system for the on-line detection method of the height of the regenerated rice ear layer as claimed in claim 8, wherein the signal processing terminal is a cab controller of the harvester or a remote computer.
10. A harvester comprising the system for the method of on-line measuring the height of the ear layer of the regenerated rice according to claim 9.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210443594.8A CN114926532B (en) | 2022-04-26 | 2022-04-26 | On-line detection method and system for height of regenerated rice spike layer and harvester |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210443594.8A CN114926532B (en) | 2022-04-26 | 2022-04-26 | On-line detection method and system for height of regenerated rice spike layer and harvester |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114926532A true CN114926532A (en) | 2022-08-19 |
| CN114926532B CN114926532B (en) | 2024-06-11 |
Family
ID=82806941
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210443594.8A Active CN114926532B (en) | 2022-04-26 | 2022-04-26 | On-line detection method and system for height of regenerated rice spike layer and harvester |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114926532B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115719453A (en) * | 2023-01-09 | 2023-02-28 | 水利部交通运输部国家能源局南京水利科学研究院 | Rice planting structure remote sensing extraction method based on deep learning |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20150139533A1 (en) * | 2013-11-15 | 2015-05-21 | Htc Corporation | Method, electronic device and medium for adjusting depth values |
| CN111968074A (en) * | 2020-07-14 | 2020-11-20 | 北京理工大学 | Method for detecting and harvesting lodging crops of harvester by combining binocular camera and IMU |
| CN112990063A (en) * | 2021-03-30 | 2021-06-18 | 北京林业大学 | Banana maturity grading method based on shape and color information |
| CN113016331A (en) * | 2021-02-26 | 2021-06-25 | 江苏大学 | Wide-narrow row ratoon rice harvesting regulation and control system and method based on binocular vision |
| CN113545219A (en) * | 2021-07-13 | 2021-10-26 | 江苏大学 | Combine harvester field head steering system and method and combine harvester |
| CN113639643A (en) * | 2021-06-24 | 2021-11-12 | 河南农业大学 | Crop seedling-stage height detection method based on RGB-D depth camera |
| CN114187353A (en) * | 2021-10-18 | 2022-03-15 | 北京理工大学 | Vision-based intelligent rice and wheat harvester reel position measuring method |
-
2022
- 2022-04-26 CN CN202210443594.8A patent/CN114926532B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20150139533A1 (en) * | 2013-11-15 | 2015-05-21 | Htc Corporation | Method, electronic device and medium for adjusting depth values |
| CN111968074A (en) * | 2020-07-14 | 2020-11-20 | 北京理工大学 | Method for detecting and harvesting lodging crops of harvester by combining binocular camera and IMU |
| CN113016331A (en) * | 2021-02-26 | 2021-06-25 | 江苏大学 | Wide-narrow row ratoon rice harvesting regulation and control system and method based on binocular vision |
| CN112990063A (en) * | 2021-03-30 | 2021-06-18 | 北京林业大学 | Banana maturity grading method based on shape and color information |
| CN113639643A (en) * | 2021-06-24 | 2021-11-12 | 河南农业大学 | Crop seedling-stage height detection method based on RGB-D depth camera |
| CN113545219A (en) * | 2021-07-13 | 2021-10-26 | 江苏大学 | Combine harvester field head steering system and method and combine harvester |
| CN114187353A (en) * | 2021-10-18 | 2022-03-15 | 北京理工大学 | Vision-based intelligent rice and wheat harvester reel position measuring method |
Non-Patent Citations (2)
| Title |
|---|
| 曹英丽;刘亚帝;马殿荣;李昂;许童羽;: "基于最优子集选择的水稻穗无人机图像分割方法", 农业机械学报, no. 08, 31 December 2020 (2020-12-31) * |
| 郭翰林;林建;张翔;: "基于HSV空间再生稻植株与土壤背景图像分割", 农机化研究, no. 07, 1 July 2017 (2017-07-01) * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115719453A (en) * | 2023-01-09 | 2023-02-28 | 水利部交通运输部国家能源局南京水利科学研究院 | Rice planting structure remote sensing extraction method based on deep learning |
| CN115719453B (en) * | 2023-01-09 | 2023-08-04 | 水利部交通运输部国家能源局南京水利科学研究院 | Deep learning-based remote sensing extraction method for rice planting structure |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114926532B (en) | 2024-06-11 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109215071B (en) | Intelligent rice and wheat harvester swath measuring method based on vision | |
| EP1655620B1 (en) | Obstacle detection using stereo vision | |
| EP3552474B1 (en) | A system for controlling an operative parameter of a harvesting header | |
| CN113016331B (en) | Wide-narrow row ratoon rice harvesting regulation and control system and method based on binocular vision | |
| CN105989601B (en) | Agricultural AGV corn inter-row navigation datum line extraction method based on machine vision | |
| EP3299996A1 (en) | Agricultural machines with image processing system | |
| CN103336966B (en) | A kind of weed images discrimination method being applied to agricultural intelligent machine | |
| CN103646249A (en) | Greenhouse intelligent mobile robot vision navigation path identification method | |
| CN111460903B (en) | System and method for monitoring growth of field broccoli based on deep learning | |
| CN114926532B (en) | On-line detection method and system for height of regenerated rice spike layer and harvester | |
| CN111968074A (en) | Method for detecting and harvesting lodging crops of harvester by combining binocular camera and IMU | |
| CN110889846A (en) | Diabetes retina image optic disk segmentation method based on FCM | |
| CN115568332A (en) | Automatic following transportation platform for field environment and control method thereof | |
| CN105844264B (en) | It is a kind of based on the recognition methods of tree peony fruit image of the oil of stress | |
| CN115147638A (en) | Machine vision-based cherry picking and classifying method and device | |
| CN110414384B (en) | Intelligent rice and wheat harvester leading line tracking method | |
| CN113963255B (en) | On-ridge operation machine navigation line identification method based on field ridge color characteristics | |
| CN110097610B (en) | Speech synthesis system and method based on ultrasound and magnetic resonance imaging | |
| CN115953690A (en) | Lodging crop identification method for advancing calibration of unmanned harvester | |
| CN113545219B (en) | Combine harvester field head steering system and method and combine harvester | |
| Wang et al. | Design of crop yield estimation system for apple orchards using computer vision | |
| EP4042850A1 (en) | Systems and methods for determining residue coverage of a field | |
| Kaur et al. | Automatic crop furrow detection for precision agriculture | |
| CN114332626A (en) | Rice yield prediction method based on visual analysis | |
| CN111179303A (en) | Grain harvesting robot visual navigation method based on particle filtering and application thereof |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB03 | Change of inventor or designer information |
Inventor after: Pan Jiahui Inventor after: Xu Lizhang Inventor after: Hu Jinpeng Inventor after: Chai Xiaoyu Inventor after: Xing Shuaifeng Inventor after: Dai Buwang Inventor after: Tian Shichao Inventor after: Yu Xuan Inventor after: Luo Yusen Inventor before: Xu Lizhang Inventor before: Pan Jiahui Inventor before: Hu Jinpeng Inventor before: Chai Xiaoyu Inventor before: Xing Shuaifeng Inventor before: Dai Buwang Inventor before: Tian Shichao Inventor before: Yu Xuan Inventor before: Luo Yusen |
|
| CB03 | Change of inventor or designer information | ||
| GR01 | Patent grant |