CN112464759A - Pot seedling disease monitoring-seedling picking system and method of semi-automatic transplanter - Google Patents
Pot seedling disease monitoring-seedling picking system and method of semi-automatic transplanter Download PDFInfo
- Publication number
- CN112464759A CN112464759A CN202011280122.2A CN202011280122A CN112464759A CN 112464759 A CN112464759 A CN 112464759A CN 202011280122 A CN202011280122 A CN 202011280122A CN 112464759 A CN112464759 A CN 112464759A
- Authority
- CN
- China
- Prior art keywords
- seedling
- disease
- pot
- image
- point cloud
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 201000010099 disease Diseases 0.000 title claims abstract description 139
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 title claims abstract description 139
- 238000000034 method Methods 0.000 title claims abstract description 53
- 238000012549 training Methods 0.000 claims abstract description 31
- 238000001514 detection method Methods 0.000 claims abstract description 13
- 238000003745 diagnosis Methods 0.000 claims abstract description 12
- 230000008569 process Effects 0.000 claims abstract description 11
- 239000002131 composite material Substances 0.000 claims abstract description 8
- 238000013135 deep learning Methods 0.000 claims abstract description 5
- 239000013589 supplement Substances 0.000 claims description 16
- 230000003213 activating effect Effects 0.000 claims description 14
- 230000000007 visual effect Effects 0.000 claims description 10
- 240000008067 Cucumis sativus Species 0.000 claims description 7
- 241000227653 Lycopersicon Species 0.000 claims description 7
- 235000007688 Lycopersicon esculentum Nutrition 0.000 claims description 7
- 235000009849 Cucumis sativus Nutrition 0.000 claims description 6
- 230000004927 fusion Effects 0.000 claims description 6
- 230000008054 signal transmission Effects 0.000 claims description 5
- 230000005484 gravity Effects 0.000 claims description 4
- 230000011218 segmentation Effects 0.000 claims description 4
- 230000001960 triggered effect Effects 0.000 claims description 4
- 230000001133 acceleration Effects 0.000 claims description 3
- 238000007781 pre-processing Methods 0.000 claims description 3
- 238000012545 processing Methods 0.000 claims description 3
- 101100327917 Caenorhabditis elegans chup-1 gene Proteins 0.000 description 4
- 230000006835 compression Effects 0.000 description 2
- 238000007906 compression Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 208000015181 infectious disease Diseases 0.000 description 2
- 235000010799 Cucumis sativus var sativus Nutrition 0.000 description 1
- 230000009471 action Effects 0.000 description 1
- 244000052616 bacterial pathogen Species 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000008030 elimination Effects 0.000 description 1
- 238000003379 elimination reaction Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 230000036541 health Effects 0.000 description 1
- 238000007689 inspection Methods 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 239000002699 waste material Substances 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/35—Categorising the entire scene, e.g. birthday party or wedding scene
- G06V20/36—Indoor scenes
-
- A—HUMAN NECESSITIES
- A01—AGRICULTURE; FORESTRY; ANIMAL HUSBANDRY; HUNTING; TRAPPING; FISHING
- A01C—PLANTING; SOWING; FERTILISING
- A01C11/00—Transplanting machines
- A01C11/006—Other parts or details or planting machines
-
- A—HUMAN NECESSITIES
- A01—AGRICULTURE; FORESTRY; ANIMAL HUSBANDRY; HUNTING; TRAPPING; FISHING
- A01C—PLANTING; SOWING; FERTILISING
- A01C11/00—Transplanting machines
- A01C11/02—Transplanting machines for seedlings
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
- G06F18/2411—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on the proximity to a decision surface, e.g. support vector machines
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/25—Fusion techniques
- G06F18/253—Fusion techniques of extracted features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N20/00—Machine learning
- G06N20/20—Ensemble learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q50/00—Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
- G06Q50/02—Agriculture; Fishing; Forestry; Mining
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
- G06T17/20—Finite element generation, e.g. wire-frame surface description, tesselation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/60—Type of objects
- G06V20/68—Food, e.g. fruit or vegetables
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Life Sciences & Earth Sciences (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Computation (AREA)
- General Engineering & Computer Science (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Software Systems (AREA)
- Multimedia (AREA)
- Soil Sciences (AREA)
- Environmental Sciences (AREA)
- Business, Economics & Management (AREA)
- Computing Systems (AREA)
- Health & Medical Sciences (AREA)
- Computer Graphics (AREA)
- Geometry (AREA)
- Agronomy & Crop Science (AREA)
- Animal Husbandry (AREA)
- Marine Sciences & Fisheries (AREA)
- Mining & Mineral Resources (AREA)
- Medical Informatics (AREA)
- Mathematical Physics (AREA)
- Economics (AREA)
- General Health & Medical Sciences (AREA)
- Human Resources & Organizations (AREA)
- Marketing (AREA)
- Primary Health Care (AREA)
- Strategic Management (AREA)
- Tourism & Hospitality (AREA)
- General Business, Economics & Management (AREA)
- Image Analysis (AREA)
Abstract
The invention provides a pot seedling disease monitoring-seedling picking system and method of a semi-automatic transplanter, and belongs to the field of agricultural robots. The pot seedling disease monitoring-picking system comprises a pot seedling panoramic information head-up acquisition device and a light and simple retractable bad seedling removing device, dynamic and reliable capture of falling pot seedling panoramic information is achieved through the pot seedling panoramic information head-up acquisition device, in the pot seedling disease composite detection process, deep learning training is conducted layer by layer through multi-position and multi-disease complex characteristics, further, automatic input and layer by layer comparison of a pot seedling RGB image to a multi-layer classification linear image curing template are conducted, accurate and rapid diagnosis of free falling pot seedlings in the process is achieved, and therefore the intelligent operation level of the field semi-automatic transplanter is effectively improved.
Description
Technical Field
The invention relates to the field of agricultural robots, in particular to a pot seedling disease monitoring-picking system and method of a semi-automatic transplanter.
Background
In recent years, the planting machine used in China is mainly semi-automatic, and in the field pot seedling transplanting operation, bad growth conditions and mutual propagation and infection of germs occur after the seedlings are transplanted, so that the yield is influenced and waste is caused. In the existing field semi-automatic transplanter pot seedling detection technology, a photoelectric, electromagnetic and other on-off sensing feedback device is mainly additionally arranged in a seedling cup, whether pot seedlings exist in the seedling cup or not and a counting function are triggered and fed back through an on-off signal, but the detection index is single, and disease seedlings cannot be diagnosed; at present, a high-speed camera is additionally arranged right above a seedling cup at a seedling falling point to overlook to acquire image information of the pot seedlings, and the existence and the blade span of the pot seedlings are detected, so that the problems of space interference and visual field shielding and protrusion of seedling throwing operation exist, and complete measurement and disease spot detection of the seedlings in the seedling cup cannot be realized by overlooking. Because the semi-automatic field transplanting operation is fast and high in frequency, farmers cannot carry out plant-by-plant inspection on the quality of pot seedlings, an efficient and intelligent pot seedling information detection system is urgently needed to ensure the transplanting quality.
Disclosure of Invention
Aiming at the defects in the prior art, the invention provides a pot seedling disease monitoring-seedling picking system and method of a semi-automatic transplanter, which effectively solve the key problems of effective detection of pot seedling health information, faulty seedling false transplanting and the like in field transplanting operation.
The present invention achieves the above-described object by the following technical means.
A pot seedling disease monitoring-picking system of a semi-automatic transplanter comprises a pot seedling panoramic information head-up acquisition device and a light and simple retractable bad seedling picking device;
the pot seedling panoramic information head-up acquisition device comprises laser correlation sensors and RGB-D cameras, the laser correlation sensors are symmetrically arranged on the inner side of the funnel, the RGB-D cameras are arranged on one side of the seedling guide pipe, and an opening II is formed in the seedling guide pipe opposite to the RGB-D cameras;
the light and simple retractable bad seedling removing device is obliquely fixed on the other side of the seedling guide pipe and comprises a driving device, a telescopic rod and double-side seedling picking plates which are sequentially connected, the driving device is also connected with a direct current motor, and pressure sensors are fixed at the bottoms of the double-side seedling picking plates; the tail end of the light and simple retractable bad seedling removing device is opposite to the opening I, the opening I is arranged on the seedling guide pipe, and the height of the opening I is lower than that of the opening II;
and the laser correlation sensor, the RGB-D camera, the direct current motor and the pressure sensor are in signal transmission with the controller.
In the technical scheme, the opening I and the opening II are square and have the same size and the height
Width of opening
Wherein HmThe maximum height of the pot seedlings counted in the suitable planting period is d is the diameter of the seedling guide pipe S1The horizontal distance between the RGB-D camera and the vertical central line of the seedling guide pipe is shown.
In the technical scheme, the device for horizontally acquiring the panoramic information of the pot seedlings further comprises a light supplement lamp which is in signal transmission with the controller, and the light supplement lamp is obliquely and downwards fixed on the inner wall of the seedling guide pipe.
In the technical scheme, the RGB-D camera is fixed at the tail end of the L-shaped camera support, and the L-shaped camera support is fixed on one side of the seedling guide pipe.
A pot seedling disease monitoring-seedling picking method of a semi-automatic transplanter comprises a falling pot seedling RGB-D image panoramic dynamic capturing method, a pot seedling disease composite detection method and a diseased seedling on-the-way falling removing method;
the falling pot seedling RGB-D image panoramic dynamic capturing method specifically comprises the following steps:
1) the completely opened cup bottom cover blocks the light beam of the laser correlation sensor in the falling process, a field scanning countdown signal of the controller is triggered, and the controller starts to synchronously time the interval time T1And T2Simultaneously, the pot seedlings start to fall freely;
2) interval time T1Then, the light supplement lamp carries out single in-tube light supplement operation;
3) interval time T2Then, the RGB-D camera carries out single pot seedling visual field scanning operation, and the RGB-D camera has a set vertical visual angle [0, beta ]]And horizontal viewing angle [ - α, + α]Performing pot seedling image grabbing to obtain a pot seedling panoramic three-dimensional depth point cloud set A and a two-dimensional RGB image B;
the pot seedling disease composite detection method comprises a disease model training sub-method and an RGB image disease area positioning and recognizing sub-method;
1) disease model training sub-method
Firstly, constructing a training library of high-incidence diseases of stem-leaf parts of pot seedlings of tomatoes and cucumbers in a suitable planting period;
secondly, manually marking the stem-leaf part high-disease-incidence picture data sets of the tomatoes and the cucumbers according to the types of the seedlings, the disease parts and the disease types and making training sets;
thirdly, classifying the seedling types in the training set into high-level characteristics, classifying the disease parts into middle-level characteristics and classifying the disease types into low-level characteristics, and sequentially implanting the training set into a deep learning training network according to the high-medium-low sequence to perform layer-by-layer training;
fourthly, performing weighted fusion on the high-medium-low characteristic parameters obtained by training the pot seedling type, the disease part and the disease type layer by layer to obtain a multilayer classified linear image curing template with pot seedling type-occurrence part-disease type fusion characteristics;
2) RGB image disease area positioning and identifying sub-method
Preprocessing the two-dimensional RGB image B to obtain a two-dimensional RGB image B with enhanced disease-health part characteristics2；
② the image B2Automatically inputting the pot seedling type image classification templates in the multi-layer classification linear image curing template for comparing one by one to determine the pot seedling type;
thirdly, automatically calling image classification templates of disease parts of the pot seedlings according to the types of the pot seedlings, and dividing the image B through stem-leaf division points2Autonomous segmentation into stem part images B3And blade part position image B4(ii) a The image B is processed3、B4Automatically inputting a disease part classification template in a multi-layer classification linear image curing template for automatic comparison, and positioning a disease occurrence part;
fourthly, positioning knots according to diseasesAutomatically calling a disease type image classification template of the corresponding part of the pot seedling; self-processing the disease area, cutting and amplifying the disease part of the stem to obtain a two-dimensional image B5Cutting and amplifying the leaf part to obtain a two-dimensional image B6(ii) a The image B is processed5Or B6Automatically inputting disease type classification templates of a multi-layer classification linear image curing template to compare one by one, and determining disease types;
the method for removing the diseased seedlings in the falling process specifically comprises the following steps:
dividing the panoramic three-dimensional depth point cloud set A (alpha, beta, D (alpha, beta)) of the pot seedlings into depth point cloud sets A only containing the leaves of the pot seedlings through stem-leaf division points1(α1，β1，D1(α1，β1) ) and a deep-point cloud collection A containing only pot seedlings (16) stalks2(α2，β2，D2(α2，β2) ); if the diseased part is detected as a blade, automatically calling a depth point cloud set A of the blade1(ii) a If the disease part is detected as the stem, automatically calling a deep point cloud set A of the stem2(ii) a If the leaves and the stems are detected to be diseased, the depth point cloud sets A of the leaves1Deep point cloud collection A of stalks2All automatically invoked.
Further, the diseased seedling falling on-the-way removing method is divided into single-part disease degree diagnosis and multi-part disease degree diagnosis according to disease parts.
Further, the diagnosis of the single-part disease degree specifically comprises:
stalk: two-dimensional image B5Coordinate automatic alignment to depth point cloud set A1In the method, the point cloud occupation ratio of the stalk disease area at the stalk part is calculated
If N is present1More than or equal to 10 percent, judging the stalk diseases are serious, and activating a seedling picking signal; wherein DA11The data amount of point cloud of the stem disease occurrence area, DA1The data quantity of the point cloud of the whole part depth of the stem is obtained;
a blade: two-dimensional image B6Coordinate auto-alignment to depthPoint cloud collection A2In the middle, calculating the point cloud ratio of the leaf disease area at the leaf part
If N is present2Not less than 15%, judging the leaf disease is serious, and activating a seedling picking signal; wherein DA22The amount of point cloud data of the area where the leaf disease occurs, DA2Depth point cloud data quantity of the integral part of the blade;
the controller controls the simple retractable bad seedling removing device by activating the seedling removing signal to remove the sick seedlings.
Further, the multi-part disease degree diagnosis specifically comprises: the two-dimensional image B5And B6Coordinate automatic alignment to depth point cloud set A1And A2Respectively calculating the point cloud occupation ratio of each disease area at each part, and activating a seedling picking signal if the point cloud occupation ratio at any part is greater than a set value; the controller controls the simple retractable bad seedling removing device by activating the seedling removing signal to remove the sick seedlings.
Further, the interval time
Said interval time
Wherein L is2Is the vertical distance from the bottom of the seedling cup to the inlet of the seedling guide pipe, L3The vertical distance from the RGB-D camera to the inlet of the seedling guide pipe, g is the gravity acceleration of the free falling body, and delta t is the set time difference between light supplement and scanning operation.
Further, the vertical viewing angle
The horizontal viewing angle
Wherein HmThe maximum height of the pot seedlings counted in the suitable planting period is d is the diameter of the seedling guide pipe S1The horizontal distance between the RGB-D camera and the vertical central line of the seedling guide pipe is shown.
The invention has the beneficial effects that: according to the method, firstly, the dynamic state of the panoramic information of the falling pot seedlings is reliably captured through an RGB-D camera, in the process of pot seedling disease composite detection, the deep learning and training of multiple parts and multiple diseases with complex characteristics are carried out layer by layer, further, the self-input and layer by layer comparison of the RGB images of the pot seedlings to a multi-layer classification linear image curing template are carried out, the disease area is accurately positioned and identified, and the accurate and rapid diagnosis and elimination of the diseased seedlings in the free falling process of the pot seedlings are realized. The invention effectively solves the outstanding problems of disease propagation infection and low transplanting quality caused by the semi-automatic transplanting operation of the field pot seedlings after diseased seedling planting, and effectively improves the intelligent operation level of the field semi-automatic transplanter.
Drawings
FIG. 1 is a schematic structural diagram of a panoramic bowl seedling information head-up acquisition device according to the invention;
FIG. 2 is a schematic structural diagram of the retractable bad seedling removing device of the present invention;
FIG. 3 is a schematic view of a double-opening structure of the wall of the seedling guide tube according to the present invention;
FIG. 4 is a schematic view of the vertical field of view compression of the RGB-D camera of the present invention;
FIG. 5 is a schematic view of the horizontal field of view compression of the RGB-D camera of the present invention;
in the figure, 1, a seedling cup, 2, a cup bottom cover, 3, a laser correlation sensor, 4, a funnel, 5, a seedling guide pipe, 6, an opening I, 7, an L-shaped camera support, 8, an RGB-D camera, 9, an opening II, 10, a light filling lamp, 11, a direct current motor, 12, a driving device, 13, a telescopic rod, 14, a pressure sensor, 15, pot picking seedling plates on two sides, and 16, seedlings are picked.
Detailed Description
The invention will be further described with reference to the following figures and specific examples, but the scope of the invention is not limited thereto.
A pot seedling disease monitoring-picking system of a semi-automatic transplanter comprises a pot seedling panoramic information head-up acquisition device, a light and simple retractable bad seedling picking device and a double-opening structure of a seedling guide pipe wall.
As shown in figure 1, the pot seedling panoramic information head-up acquisition device comprises a laser correlation sensor 3, an L-shaped camera support 7 and an RA GB-D camera 8 and a fill light 10; the laser correlation sensors 3 are symmetrically arranged at the inner side of the funnel 4, and the nearest vertical distance between the laser correlation sensors 3 and the seedling cup 1 is L1(ii) a The L-shaped camera support 7 is fixed on the right side of the seedling guide pipe 5, and the distance from the L-shaped camera support 7 to the vertical central line of the seedling guide pipe 5 is S3(ii) a The RGB-D camera 8 is fixed at the tail end of the L-shaped camera support 7, the horizontal visual angle of the RGB-D camera 8 is symmetrical along the vertical central line of the seedling guide pipe 5, and the horizontal distance between the RGB-D camera 8 and the vertical central line of the seedling guide pipe 5 is S1(ii) a The opening II 9 is positioned at the right lower part of the inlet of the seedling guide pipe 5; the light supplement lamp 10 is at an angle theta1The oblique left lower part is fixed on the right inner wall of the seedling guide pipe 5, and the light supplement lamp 10 is positioned at the upper part of the opening II 9; in this example L1＝10cm、S1＝20cm、S325 cm. The RGB-D camera 8 shoots the depth-color information of the panoramic view of the pot seedlings 16 through the opening II 9. The funnel 4 is arranged at the upper end of the seedling guide pipe 5.
As shown in fig. 1, 2 and 3, the light and simple retractable bad seedling removing device comprises a direct current motor 11, a driving device 12, a telescopic rod 13, a pressure sensor 14 and double-side seedling removing plates 15; one end of a driving device 12 is connected with one end of a telescopic rod 13, the other end of the driving device is connected with a direct current motor 11, a double-side seedling picking plate 15 is horizontally arranged above the other end of the telescopic rod 13, a pressure sensor 14 is fixed at the bottom of the double-side seedling picking plate 15, an opening I6 is positioned on the left side of the pipe wall below a seedling guide pipe 5, and the height of the opening I6 is lower than that of an opening II 9; the light and simple retractable bad seedling removing device is obliquely and upwards fixed on the bottom plate on the left side of the seedling guide pipe 5, and the distance between the light and simple retractable bad seedling removing device and the vertical central line of the seedling guide pipe 5 is S2The tail end of the light and simple retractable bad seedling removing device is opposite to the opening I6; in this example S2＝13cm。
As shown in fig. 3, opening i 6 and opening ii 9 constitute the two open structure of seedling guide pipe wall, and opening i 6 and opening ii 9 are square and the same size, and height (H) and width (W) of opening are:
in the formula: hmIs suitable for planting periodThe maximum height of the seedling statistics, d is the diameter of the seedling guide pipe, H in the embodimentm＝12cm、d＝15cm。
The laser correlation sensor 3, the RGB-D camera 8, the light supplement lamp 10, the direct current motor 11 and the pressure sensor 14 are in signal transmission with the controller.
A pot seedling disease monitoring-seedling picking method of a semi-automatic transplanter comprises a falling pot seedling RGB-D image panoramic dynamic capturing method, a pot seedling disease composite detection method and a diseased seedling on-the-way falling removing method.
(1) As shown in fig. 4 and 5, the falling pot seedling RGB-D image panoramic dynamic capturing method specifically includes the following steps:
step one, the seedling cup 1 rotates to a position right above the center of the funnel 4, the cup bottom cover 2 at the bottom of the seedling cup 1 automatically opens under the action of gravity, the completely opened cup bottom cover 2 blocks the light beam of the laser correlation sensor 3 in the falling process, so that a field-of-view scanning countdown signal of the controller is triggered, and the controller starts to synchronously time T1And T2Simultaneously, the pot seedlings 16 start to fall freely;
step two, interval time T1The rear light supplement lamp 10 performs single in-tube light supplement operation under the control of the controller, and after the light supplement operation is finished, the light supplement lamp 10 is in a dormant state and waits for the next trigger; wherein time T1Comprises the following steps:
in the formula L2Is the vertical distance from the bottom of the seedling cup 1 to the inlet of the seedling guide pipe 5, L3The vertical distance from the RGB-D camera 8 to the entrance of the seedling guide tube 5, g is the gravity acceleration of the free falling body, and delta t is the set time difference between the light supplement and the scanning operation, in this embodiment, L2＝20cm、L3＝30cm、g＝9.8m/s2、Δt＝0.05；
Step three, spacing time T2Then, the RGB-D camera 8 carries out single pot seedling visual field scanning operation under the control of the controller, and after the scanning operation is finished, the RGB-D camera 8 is set to be in a dormant state and waits for the next triggering; wherein time T2Comprises the following steps:
meanwhile, during the visual field scanning operation, the RGB-D camera 8 captures the 16 images of the pot seedlings at the vertical visual angle [0, beta ] and the horizontal visual angle [ -alpha, + alpha ] set by the controller; wherein the vertical viewing angle β and the horizontal viewing angle α are respectively:
and step four, the RGB-D camera 8 obtains a pot seedling 16 panoramic three-dimensional depth point cloud set A and a two-dimensional RGB image B.
(2) The pot seedling disease composite detection method comprises a disease model training sub-method and an RGB image disease area positioning and recognizing sub-method;
the disease model training sub-method specifically comprises the following steps:
step one, constructing a training library of high-incidence diseases of stem-leaf parts of pot seedlings of tomatoes and cucumbers in a suitable planting period, wherein the training library is shown in table 1:
TABLE 1 training library for high disease damage of stem-leaf part of potted seedlings of tomato and cucumber in proper planting period
Step two, carrying out artificial marking on the stem-leaf part high disease damage picture data sets (obtained in advance) of the tomatoes and the cucumbers according to the seedling types, the disease parts and the disease types and preparing training sets;
thirdly, classifying the seedling types in the training set into high-level characteristics, classifying the disease parts into middle-level characteristics and classifying the disease types into low-level characteristics; sequentially implanting a training set into a deep learning training network according to a high-medium-low sequence to perform layer-by-layer training by utilizing an open-source Resnet50 network and a Pythrch open-source model algorithm;
and step four, performing weighted fusion on the high-medium-low characteristic parameters obtained by training the pot seedling type, the disease part and the disease type layer by layer to obtain a multi-layer classification linear image curing template with the pot seedling type-generating part-disease type fusion characteristics, wherein the multi-layer classification linear image curing template comprises a pot seedling type image classification template, a generating part image classification template and a disease type image classification template.
A RGB image disease area positioning and identifying sub-method specifically comprises the following steps:
step one, preprocessing a pot seedling panoramic two-dimensional RGB image B in a controller to obtain a two-dimensional RGB image B with enhanced disease-health part characteristics2；
Step two, image B2Automatically inputting the bowl seedling type (high-level feature) image classification templates in the multi-layer classification linear image curing template to compare one by one, and obtaining an image B2The template type with the highest similarity with the layer is the pot seedling type;
step three, according to the variety result of the pot seedlings, the controller automatically calls an image classification template of disease parts (middle-layer characteristics) of the pot seedlings, and images B are obtained through stem-leaf segmentation points2Autonomous segmentation into stem part images B3And blade part position image B4(ii) a Image B3、B4Automatically inputting a disease part (middle layer characteristic) classification template in a multi-layer classification linear image curing template for automatic comparison, and positioning a disease generation part;
step four, automatically calling a disease type (low-level characteristic) image classification template of the corresponding part of the pot seedlings according to the disease positioning result in the step three; self-processing the disease area, cutting and amplifying the disease part of the stem to obtain an image B5Cutting and enlarging the leaf part to obtain an image B6(ii) a Image B5Or B6And automatically inputting the disease type (low-level characteristics) classification templates of the multi-layer classification linear image curing template to compare one by one, wherein the class with the highest similarity is the disease type.
(3) The method for removing the diseased seedlings in the falling process comprises the following steps: dividing the pot seedling 16 panoramic depth point cloud set A (alpha, beta, D (alpha, beta)) into depth point cloud sets A only containing 16 leaves of the pot seedling through stem-leaf division points1(α1，β1，D1(α1，β1) ) and a deep point cloud collection A containing only 16 stalks of pot seedlings2(α2，β2，D2(α2，β2))；
If the diseased part is detected as a blade, automatically calling a depth point cloud set A of the blade1(α1，β1，D1(α1，β1) ); if the disease part is detected as the stem, automatically calling a deep point cloud set A of the stem2(α2，β2，D2(α2，β2) ); if the leaves and the stems are detected to be diseased, the depth point cloud sets A of the leaves1(α1，β1，D1(α1，β1) ) and depth point clouds of stalks A2(α2，β2，D2(α2，β2) All automatic calls;
1) single-site disease degree diagnosis
Stalk: two-dimensional image B5Coordinate automatic alignment to depth point cloud set A1(α1，β1，D1(α1，β1) In), calculate:
in the formula N1The point cloud of the stem disease area at the stem part is taken as the ratio DA11The data amount of point cloud of the stem disease occurrence area, DA1The data quantity of the point cloud of the whole part depth of the stem is obtained; if N is present1More than or equal to 10 percent, judging the stalk diseases are serious, and activating a seedling picking signal;
a blade: two-dimensional image B6Coordinate automatic alignment to depth point cloud set A2(α2，β2，D2(α2，β2) In), calculate:
in the formula N2The point cloud of the diseased region of the leaf at the leaf part is calculated, DA22The amount of point cloud data of the area where the leaf disease occurs, DA2Depth point cloud data quantity of the integral part of the blade; if N is present2Not less than 15%, judging the leaf disease is serious, and activating a seedling picking signal;
2) multi-site disease degree diagnosis
Two-dimensional image B5And B6Coordinate automatic alignment to depth point cloud set A1And A2And respectively calculating the point cloud occupation ratios of each disease and each part area, wherein the occupation ratio value of any part is greater than a set value, and activating a seedling picking signal.
The controller immediately drives the direct current motor 11 to rotate forwards by activating the seedling removing signal, so that the telescopic rod 13 enters the seedling guide pipe 5 through the opening I6, and when the controller detects that the weight data of the pressure sensor 14 changes, the controller immediately sends the direct current motor 11 to rotate backwards, so that the telescopic rod 13 is withdrawn, diseased seedlings are effectively taken out of the seedling guide pipe 5, and the seedling removing is completed.
In the description of the present invention, it is to be understood that the terms "left", "right", "vertical", "horizontal", and the like indicate orientations or positional relationships based on those shown in the drawings, and are only for convenience in describing the present invention and simplifying the description, but do not indicate or imply that the referred device or element must have a specific orientation, be constructed in a specific orientation, and be operated, and thus, should not be construed as limiting the present invention.
The present invention is not limited to the above-described embodiments, and any obvious improvements, substitutions or modifications can be made by those skilled in the art without departing from the spirit of the present invention.
Claims (10)
1. A pot seedling disease monitoring-picking system of a semi-automatic transplanter is characterized by comprising a pot seedling panoramic information head-up acquisition device and a light and simple retractable bad seedling picking device;
the pot seedling panoramic information head-up acquisition device comprises laser correlation sensors (3) and RGB-D cameras (8), wherein the laser correlation sensors (3) are symmetrically arranged on the inner side of a funnel (4), the RGB-D cameras (8) are arranged on one side of a seedling guide pipe (5), and openings II (9) are formed in the seedling guide pipe (5) opposite to the RGB-D cameras (8);
the other side of the seedling guide pipe (5) is obliquely fixed with a light and simple retractable bad seedling removing device, the light and simple retractable bad seedling removing device comprises a driving device (12), a telescopic rod (13) and double-side seedling picking plates (15) which are sequentially connected, the driving device (12) is also connected with a direct current motor (11), and pressure sensors (14) are fixed at the bottoms of the double-side seedling picking plates (15); the tail end of the light and simple retractable bad seedling removing device is opposite to the opening I (6), the opening I (6) is arranged on the seedling guide pipe (5), and the height of the opening I (6) is lower than that of the opening II (9);
the laser correlation sensor (3), the RGB-D camera (8), the direct current motor (11) and the pressure sensor (14) are in signal transmission with the controller.
2. The pot seedling disease monitoring-picking system as claimed in claim 1, wherein the opening I (6) and the opening II (9) are square and have the same size, and the height of the opening is equal to that of the opening
Width of opening
Wherein HmThe maximum height of the pot seedlings counted in the suitable planting period is d is the diameter of the seedling guide pipe S1Is the horizontal distance between the RGB-D camera (8) and the vertical central line of the seedling guide tube (5).
3. The pot seedling disease monitoring-picking system according to claim 1, characterized in that the head-up acquisition device for panoramic pot seedling information further comprises a supplementary lighting lamp (10) in signal transmission with the controller, and the supplementary lighting lamp (10) is obliquely and downwardly fixed on the inner wall of the seedling guide pipe (5).
4. The pot seedling disease monitoring-picking system according to claim 1, wherein the RGB-D camera (8) is fixed at the tail end of the L-shaped camera support (7), and the L-shaped camera support (7) is fixed at one side of the seedling guide pipe (5).
5. A pot seedling disease monitoring-seedling picking method of a semi-automatic transplanter is characterized by comprising a falling pot seedling RGB-D image panoramic dynamic capturing method, a pot seedling disease composite detection method and a diseased seedling picking method in the falling process;
the falling pot seedling RGB-D image panoramic dynamic capturing method specifically comprises the following steps:
1) the completely opened cup bottom cover (2) blocks the light beam of the laser correlation sensor (3) in the falling process, a field scanning countdown signal of the controller is triggered, and the controller starts to synchronously time the interval time T1And T2Simultaneously, the pot seedlings (16) start to fall freely;
2) interval time T1Then, the light supplement lamp (10) performs single in-tube light supplement operation;
3) interval time T2Then, the RGB-D camera (8) carries out single pot seedling visual field scanning operation, and the RGB-D camera (8) carries out vertical visual angle [0, beta ] with set value]And horizontal viewing angle [ - α, + α]Performing pot seedling (16) image capture to obtain a pot seedling (16) panoramic three-dimensional depth point cloud set A and a two-dimensional RGB image B;
the pot seedling disease composite detection method comprises a disease model training sub-method and an RGB image disease area positioning and recognizing sub-method;
1) disease model training sub-method
Firstly, constructing a training library of high-incidence diseases of stem-leaf parts of pot seedlings of tomatoes and cucumbers in a suitable planting period;
secondly, manually marking the stem-leaf part high-disease-incidence picture data sets of the tomatoes and the cucumbers according to the types of the seedlings, the disease parts and the disease types and making training sets;
thirdly, classifying the seedling types in the training set into high-level characteristics, classifying the disease parts into middle-level characteristics and classifying the disease types into low-level characteristics, and sequentially implanting the training set into a deep learning training network according to the high-medium-low sequence to perform layer-by-layer training;
fourthly, performing weighted fusion on the high-medium-low characteristic parameters obtained by training the pot seedling type, the disease part and the disease type layer by layer to obtain a multilayer classified linear image curing template with pot seedling type-occurrence part-disease type fusion characteristics;
2) RGB image disease area positioning and identifying sub-method
Preprocessing the two-dimensional RGB image B to obtain a two-dimensional RGB image B with enhanced disease-health part characteristics2；
② the image B2Automatically inputting the pot seedling type image classification templates in the multi-layer classification linear image curing template for comparing one by one to determine the pot seedling type;
thirdly, automatically calling image classification templates of disease parts of the pot seedlings according to the types of the pot seedlings, and dividing the image B through stem-leaf division points2Autonomous segmentation into stem part images B3And blade part position image B4(ii) a The image B is processed3、B4Automatically inputting a disease part classification template in a multi-layer classification linear image curing template for automatic comparison, and positioning a disease occurrence part;
fourthly, automatically calling a disease type image classification template of the corresponding part of the pot seedling according to the disease positioning result; self-processing the disease area, cutting and amplifying the disease part of the stem to obtain a two-dimensional image B5Cutting and amplifying the leaf part to obtain a two-dimensional image B6(ii) a The image B is processed5Or B6Automatically inputting disease type classification templates of a multi-layer classification linear image curing template to compare one by one, and determining disease types;
the method for removing the diseased seedlings in the falling process specifically comprises the following steps:
through the stem-leaf division points, the panoramic three-dimensional depth point cloud set A (alpha, beta, D (alpha, beta)) of the pot seedlings (16) is divided into depth point cloud sets A only containing the leaf blades of the pot seedlings (16)1(α1，β1，D1(α1，β1) ) and a deep-point cloud collection A containing only pot seedlings (16) stalks2(α2，β2，D2(α2，β2) ); if the diseased part is detected as a blade, automatically calling a depth point cloud set A of the blade1(ii) a If the disease part is detected as the stem, automatically calling a deep point cloud set A of the stem2(ii) a If the leaves and the stems are detected to be diseased, the depth point cloud sets A of the leaves1Deep point cloud collection A of stalks2All automatically invoked.
6. The pot seedling disease monitoring-seedling picking method according to claim 5, characterized in that the method for picking seedlings falling on the way is divided into single-part disease degree diagnosis and multi-part disease degree diagnosis according to disease parts.
7. The pot seedling disease monitoring-picking method according to claim 6, wherein the single-part disease degree diagnosis specifically comprises:
stalk: two-dimensional image B5Coordinate automatic alignment to depth point cloud set A1In the method, the point cloud occupation ratio of the stalk disease area at the stalk part is calculated
If N is present1More than or equal to 10 percent, judging the stalk diseases are serious, and activating a seedling picking signal; wherein DA11The data amount of point cloud of the stem disease occurrence area, DA1The data quantity of the point cloud of the whole part depth of the stem is obtained;
a blade: two-dimensional image B6Coordinate automatic alignment to depth point cloud set A2In the middle, calculating the point cloud ratio of the leaf disease area at the leaf part
If N is present2Not less than 15%, judging the leaf disease is serious, and activating a seedling picking signal; wherein DA22The amount of point cloud data of the area where the leaf disease occurs, DA2Depth point cloud data quantity of the integral part of the blade;
the controller controls the simple retractable bad seedling removing device by activating the seedling removing signal to remove the sick seedlings.
8. The pot seedling disease monitoring-picking method according to claim 7, wherein the multi-part disease degree diagnosis specifically comprises: the two-dimensional image B5And B6Coordinate automatic alignment to depth point cloud set A1And A2Respectively calculating the point cloud occupation ratio of each disease area at each part, and activating a seedling picking signal if the point cloud occupation ratio at any part is greater than a set value; the controller controls the simple retractable bad seedling removing device by activating the seedling removing signal to remove the sick seedlings.
9. The pot seedling disease monitoring-picking method as claimed in claim 5, wherein the interval time is set
Said interval time
Wherein L is2Is the vertical distance from the bottom of the seedling cup (1) to the inlet of the seedling guide pipe (5), L3The vertical distance from the RGB-D camera (8) to the inlet of the seedling guide tube (5), g is the gravity acceleration of the free falling body, and delta t is the set time difference between light supplement and scanning operation.
10. The pot seedling disease monitoring-picking method as claimed in claim 5, wherein the vertical viewing angle
The horizontal viewing angle
Wherein HmThe maximum height of the pot seedlings counted in the suitable planting period is d is the diameter of the seedling guide pipe S1Is the horizontal distance between the RGB-D camera (8) and the vertical central line of the seedling guide tube (5).
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011280122.2A CN112464759A (en) | 2020-11-16 | 2020-11-16 | Pot seedling disease monitoring-seedling picking system and method of semi-automatic transplanter |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011280122.2A CN112464759A (en) | 2020-11-16 | 2020-11-16 | Pot seedling disease monitoring-seedling picking system and method of semi-automatic transplanter |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112464759A true CN112464759A (en) | 2021-03-09 |
Family
ID=74836970
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011280122.2A Pending CN112464759A (en) | 2020-11-16 | 2020-11-16 | Pot seedling disease monitoring-seedling picking system and method of semi-automatic transplanter |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112464759A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112930809A (en) * | 2021-04-23 | 2021-06-11 | 石河子大学 | Automatic seedling device is got to whole row |
| CN112970714A (en) * | 2021-03-16 | 2021-06-18 | 亳州学院 | Chinese-medicinal material plant diseases and insect pests location and counting assembly |
| CN113950914A (en) * | 2021-09-16 | 2022-01-21 | 江苏大学 | Selective seedling feeding device, control method thereof and transplanter |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JPH05308804A (en) * | 1992-05-01 | 1993-11-22 | Norin Suisansyo Yasai Chiyagiyou Shikenjo | Thinning machine |
| CN104737686A (en) * | 2015-03-26 | 2015-07-01 | 江苏大学 | Motion coordination control system and motion coordination control method for automatic plug seedling transplanter |
| CN107690906A (en) * | 2017-11-22 | 2018-02-16 | 河南科技大学 | A kind of detection of pot seedling disk leakage seedling and positioning seek seedling and change seedling device and method |
| CN207427789U (en) * | 2017-11-22 | 2018-06-01 | 河南科技大学 | Detection and more changing device are planted in a kind of pot seedling disk leakproof |
| CN109682326A (en) * | 2018-12-19 | 2019-04-26 | 河南科技大学 | Pot seedling upright degree detection device and detection method based on depth image |
-
2020
- 2020-11-16 CN CN202011280122.2A patent/CN112464759A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JPH05308804A (en) * | 1992-05-01 | 1993-11-22 | Norin Suisansyo Yasai Chiyagiyou Shikenjo | Thinning machine |
| CN104737686A (en) * | 2015-03-26 | 2015-07-01 | 江苏大学 | Motion coordination control system and motion coordination control method for automatic plug seedling transplanter |
| CN107690906A (en) * | 2017-11-22 | 2018-02-16 | 河南科技大学 | A kind of detection of pot seedling disk leakage seedling and positioning seek seedling and change seedling device and method |
| CN207427789U (en) * | 2017-11-22 | 2018-06-01 | 河南科技大学 | Detection and more changing device are planted in a kind of pot seedling disk leakproof |
| CN109682326A (en) * | 2018-12-19 | 2019-04-26 | 河南科技大学 | Pot seedling upright degree detection device and detection method based on depth image |
Non-Patent Citations (1)
| Title |
|---|
| 杨启志;孙梦涛;蔡静;石新异;毛罕平;顾俊;: "温室穴盘苗并联高速移栽机器人运动误差分析与试验", 农业机械学报, no. 03, 8 December 2017 (2017-12-08) * |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112970714A (en) * | 2021-03-16 | 2021-06-18 | 亳州学院 | Chinese-medicinal material plant diseases and insect pests location and counting assembly |
| CN112970714B (en) * | 2021-03-16 | 2023-01-24 | 亳州学院 | Chinese-medicinal material plant diseases and insect pests location and counting assembly |
| CN112930809A (en) * | 2021-04-23 | 2021-06-11 | 石河子大学 | Automatic seedling device is got to whole row |
| CN113950914A (en) * | 2021-09-16 | 2022-01-21 | 江苏大学 | Selective seedling feeding device, control method thereof and transplanter |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112464759A (en) | Pot seedling disease monitoring-seedling picking system and method of semi-automatic transplanter | |
| Nielsen et al. | Vision-based 3D peach tree reconstruction for automated blossom thinning | |
| CN201600330U (en) | System for recognizing and locating mature pineapples | |
| WO2021208407A1 (en) | Target object detection method and apparatus, and image collection method and apparatus | |
| CN111462058B (en) | Method for rapidly detecting effective rice ears | |
| KR20140077513A (en) | System and method for crops information management of greenhouse using image | |
| CN107610122B (en) | Micro-CT-based single-grain cereal internal insect pest detection method | |
| CN109211198A (en) | A kind of intelligent Target detection and measuring system and method based on trinocular vision | |
| CN110286092A (en) | A kind of plant growth trend analysis system | |
| CN112580671A (en) | Automatic detection method and system for multiple development stages of rice ears based on deep learning | |
| CN101940096B (en) | System for sorting corn seed haploid | |
| CN114818909A (en) | Weed detection method and device based on crop growth characteristics | |
| CN110741790B (en) | Multi-claw transplanting-sorting processing method for plug seedlings based on depth camera | |
| CN107464232B (en) | Image detection method for planting quality of unmanned rice transplanter | |
| CN109598215A (en) | Orchard modeling analysis system and method based on positioning shooting of unmanned aerial vehicle | |
| CN110689022B (en) | Method for extracting images of crops of each plant based on blade matching | |
| Li et al. | Design and experiment of intelligent sorting and transplanting system for healthy vegetable seedlings | |
| CN105319166A (en) | Real-time soybean detection instrument | |
| CN201789739U (en) | Haploid corn seed sorting system | |
| CN114140675B (en) | Sugarcane seed screening system and method based on deep learning | |
| CN112907516A (en) | Sweet corn seed identification method and device for plug seedling | |
| CN208766708U (en) | A kind of beanpod quantity statistics device | |
| CN109063816B (en) | Bean pod number statistics device and method | |
| CN111950773A (en) | System and method for predicting tea yield | |
| CN110866972A (en) | In-situ observation device for sugarcane root system configuration and analysis method thereof |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |