CN116465887A - Phenotype collection method based on facility crop phenotype collection device - Google Patents

Phenotype collection method based on facility crop phenotype collection device Download PDF

Info

Publication number
CN116465887A
CN116465887A CN202310260996.9A CN202310260996A CN116465887A CN 116465887 A CN116465887 A CN 116465887A CN 202310260996 A CN202310260996 A CN 202310260996A CN 116465887 A CN116465887 A CN 116465887A
Authority
CN
China
Prior art keywords
camera
crop
rgb
phenotype
light source
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202310260996.9A
Other languages
Chinese (zh)
Inventor
张倩
郭建鑫
于峰
王明
刘新
平阳
王一罡
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Academy of Agriculture and Forestry Sciences
Original Assignee
Beijing Academy of Agriculture and Forestry Sciences
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Academy of Agriculture and Forestry Sciences filed Critical Beijing Academy of Agriculture and Forestry Sciences
Priority to CN202310260996.9A priority Critical patent/CN116465887A/en
Publication of CN116465887A publication Critical patent/CN116465887A/en
Pending legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01J—MEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY
    • G01J3/00—Spectrometry; Spectrophotometry; Monochromators; Measuring colours
    • G01J3/46—Measurement of colour; Colour measuring devices, e.g. colorimeters
    • G01J3/52—Measurement of colour; Colour measuring devices, e.g. colorimeters using colour charts
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/01—Arrangements or apparatus for facilitating the optical investigation
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/84—Systems specially adapted for particular applications
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
    • G06T7/33—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods
    • G06T7/344—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods involving models
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/40—Analysis of texture
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/60—Analysis of geometric attributes
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/90—Determination of colour characteristics
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/695—Control of camera direction for changing a field of view, e.g. pan, tilt or based on tracking of objects
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/90—Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/01—Arrangements or apparatus for facilitating the optical investigation
    • G01N2021/0106—General arrangement of respective parts
    • G01N2021/0112—Apparatus in one mechanical, optical or electronic block
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/84—Systems specially adapted for particular applications
    • G01N2021/8466—Investigation of vegetal material, e.g. leaves, plants, fruits
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
    • Y02A40/00—Adaptation technologies in agriculture, forestry, livestock or agroalimentary production
    • Y02A40/10—Adaptation technologies in agriculture, forestry, livestock or agroalimentary production in agriculture
    • Y02A40/25—Greenhouse technology, e.g. cooling systems therefor

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Health & Medical Sciences (AREA)
  • Chemical & Material Sciences (AREA)
  • Geometry (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Spectroscopy & Molecular Physics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Pathology (AREA)
  • Analytical Chemistry (AREA)
  • Biochemistry (AREA)
  • General Health & Medical Sciences (AREA)
  • Immunology (AREA)
  • Software Systems (AREA)
  • Computer Graphics (AREA)
  • Image Processing (AREA)

Abstract

The invention relates to the technical field of crop phenotype acquisition, and provides a phenotype acquisition method based on a facility crop phenotype acquisition device, wherein the phenotype acquisition device comprises a mobile platform, a mounting bracket and a camera shooting assembly; the mounting bracket is arranged on the mobile platform, the camera shooting assembly is arranged on the mounting bracket, and the camera shooting assembly comprises a first RGB-D camera and a second RGB-D camera; the phenotype acquisition method comprises the following steps: controlling the mobile platform to move along the left side and the right side of the same crop row respectively, and acquiring video acquisition information of the crop row; acquiring left-view images, right-view images and overlook images of each plant of crops in crop rows according to the video acquisition information; establishing a 3D space model of the crop according to the left-view image, the right-view image and the overlook image; and extracting the characteristics of the 3D space model to obtain the phenotype parameters of the crops. The invention can rapidly, accurately and nondestructively perform large-scale phenotype measurement on facility crops.

Description

Phenotype collection method based on facility crop phenotype collection device
Technical Field
The invention relates to the technical field of crop phenotype acquisition, in particular to a phenotype acquisition method based on a facility crop phenotype acquisition device.
Background
The digital acquisition of crop phenotypes is the basis of research on crop growth and development. The phenotype acquisition and analysis technology is combined with genome information, so that crops with higher yield, higher quality and higher stress tolerance can be cultivated, and the breeding improvement efficiency is greatly improved. The development of image processing has led to a growing search for image-based crop phenotypes.
Crop phenotype image acquisition devices applied in the current facility environment have certain disadvantages to a greater or lesser extent: the hand-held acquisition equipment needs manual operation and is greatly interfered by human factors; the fixed acquisition device cannot continuously acquire images, and is small in coverage and single in visual angle; the hanging device needs to deploy hanging points in the greenhouse, so that the construction difficulty is high, and the hanging device is influenced by the specification of the greenhouse.
The traditional crop phenotype acquisition method has the defects of less measurement characters, small sample size, low manual measurement efficiency, large subjective serious error, poor adaptability to single plants and the like, and cannot meet the requirements of large-scale, rapid, accurate and nondestructive phenotype measurement.
Disclosure of Invention
The invention provides a phenotype acquisition method based on a facility crop phenotype acquisition device, which is used for solving the problem that the current method is difficult to rapidly, accurately and nondestructively perform large-scale phenotype measurement on facility crops.
The invention provides a phenotype acquisition method based on a facility crop phenotype acquisition device, wherein the phenotype acquisition device comprises a mobile platform, a mounting bracket and a camera shooting assembly; the mounting support is arranged on the mobile platform, the camera shooting assembly is arranged on the mounting support, the camera shooting assembly comprises a first RGB-D camera and a second RGB-D camera, the first RGB-D camera is used for being arranged on the left side or the right side of a crop row, and the second RGB-D camera is used for being arranged on the top of the crop row; the phenotype acquisition method comprises the following steps:
controlling the mobile platform to move along the left side and the right side of the same crop row respectively, and acquiring video acquisition information of the crop row;
acquiring left-view images, right-view images and overlook images of each plant of crops in the crop rows according to the video acquisition information;
establishing a 3D space model of the crop according to the left-view image, the right-view image and the overlook image;
and extracting the characteristics of the 3D space model to obtain the phenotype parameters of the crops.
According to the phenotype acquisition method based on the facility crop phenotype acquisition device provided by the invention, the steps of controlling the mobile platform to move along the left side and the right side of the same crop row respectively and acquiring video acquisition information of the crop row comprise the following steps:
controlling the mobile platform to move along the extending direction of the crop row at the left side of the crop row, and acquiring first video information acquired by the first RGB-D camera and second video information acquired by the second RGB-D camera;
controlling the mobile platform to move along the extending direction of the crop row on the right side of the crop row, and acquiring third video information acquired by the first RGB-D camera and fourth video information acquired by the second RGB-D camera;
correspondingly, the step of acquiring left-view images, right-view images and top-view images of each plant of crops in the crop rows according to the video acquisition information comprises the following steps:
and calibrating and matching the first video information and the third video information with the second video information and the fourth video information to obtain the left-view image, the right-view image and the top-view image of each crop in the crop rows.
According to the phenotype acquisition method based on the facility crop phenotype acquisition device, the moving speeds of the moving platforms along the extending direction of the crop row are the same at the left side and the right side of the crop row;
wherein the moving speed of the moving platform is determined according to the type of the crop.
According to the phenotype acquisition method based on the facility crop phenotype acquisition device, the phenotype acquisition device further comprises a colorimetric plate; the colorimetric plate is arranged on the mounting bracket, and the colorimetric plate is arranged opposite to the lens of at least one of the first RGB-D camera and the second RGB-D camera;
accordingly, the phenotype acquisition method further comprises: and correcting the colors of the left-view image, the right-view image and the overlook image by adopting the colorimetric plate.
According to the phenotype acquisition method based on the facility crop phenotype acquisition device, the mounting bracket comprises a vertical expansion bracket and a horizontal expansion bracket;
the bottom end of the vertical expansion bracket is connected with the mobile platform, and the top end of the vertical expansion bracket is vertically connected with the horizontal expansion bracket;
the first RGB-D camera is arranged on the vertical expansion bracket, and the second RGB-D camera is arranged on the horizontal expansion bracket.
According to the phenotype acquisition method based on the facility crop phenotype acquisition device, the optical axis of the first RGB-D camera is horizontally arranged, and the vertical expansion bracket is used for adjusting the height of the optical axis of the first RGB-D camera relative to the horizontal plane;
the optical axis of the second RGB-D camera is arranged vertically, and the horizontal expansion bracket is used for adjusting the distance between the optical axis of the second RGB-D camera and the vertical expansion bracket.
According to the phenotype acquisition method based on the facility crop phenotype acquisition device provided by the invention, the camera shooting assembly further comprises a first light source and a second light source; the first light source is arranged on one side of the first RGB-D camera, and the second light source is arranged on one side of the second RGB-D camera;
accordingly, the phenotype acquisition method further comprises: in the process of acquiring the video acquisition information, controlling the first light source and the second light source to be turned on;
and acquiring the ambient illumination intensity, and adjusting the luminous power of the first light source and the luminous power of the second light source according to the ambient illumination intensity.
According to the phenotype acquisition method based on the facility crop phenotype acquisition device, at least one of the first light source and the second light source comprises an LED lamp.
According to the phenotype acquisition method based on the facility crop phenotype acquisition device provided by the invention, the phenotype acquisition device further comprises control equipment; the control equipment comprises a power supply module, a control module and a communication module;
the power supply module is respectively and electrically connected with the control module, the communication module and the mobile platform; the mobile platform and the camera shooting assembly are respectively connected with the control module, the control module is connected with the communication module, and the communication module is used for communicating with a remote control terminal.
According to the phenotype acquisition method based on the facility crop phenotype acquisition device, the phenotype parameters comprise plant height, leaf number, canopy coverage, leaf area index and leaf angle of crops.
According to the phenotype acquisition method based on the facility crop phenotype acquisition device, the mobile platform is controlled to move along the left side and the right side of the same crop row respectively, the first RGB-D camera and the second RGB-D camera are used for acquiring videos of crops containing RGB features and depth features, left-view images, right-view images and overlook images of the crops are obtained through data calibration and matching, a 3D space model of the crops is built according to the phenotype images, the colors, the shapes and the texture features of the crops are extracted from the 3D space model, and further the phenotype parameters of the crops are obtained.
Therefore, the phenotype collection method based on the facility crop phenotype collection device provided by the invention is convenient to operate, and can realize rapid, accurate and nondestructive large-scale phenotype measurement of facility crops.
Drawings
In order to more clearly illustrate the invention or the technical solutions of the prior art, the following description will briefly explain the drawings used in the embodiments or the description of the prior art, and it is obvious that the drawings in the following description are some embodiments of the invention, and other drawings can be obtained according to the drawings without inventive effort for a person skilled in the art.
FIG. 1 is a schematic diagram of a facility crop phenotype acquisition device provided by the invention;
FIG. 2 is a schematic flow chart of a phenotype collection method based on a facility crop phenotype collection device provided by the invention;
fig. 3 is a schematic flow chart of acquiring video acquisition information of a crop row according to the present invention;
FIG. 4 is a control block diagram of the facility crop phenotype acquisition device provided by the invention;
fig. 5 is a schematic workflow diagram of a phenotype collection method based on a facility crop phenotype collection device provided by the invention.
Reference numerals:
1. a mobile platform;
2. a mounting bracket; 21. a vertical expansion bracket; 22. a horizontal expansion bracket;
3. a camera assembly; 31. a first RGB-D camera; 32. a second RGB-D camera; 33. a first light source; 34. a second light source;
4. a color comparison plate;
5. a control device; 51. a power supply module; 52. a control module; 53. a communication module; 54. a storage module;
6. crop rows; 7. and (5) ridging.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the present invention more apparent, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings, and it is apparent that the described embodiments are some embodiments of the present invention, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
The following describes in detail, with reference to fig. 1 to fig. 5, a phenotype acquisition method based on a facility crop phenotype acquisition device according to an embodiment of the present invention through specific embodiments and application scenarios thereof.
In some embodiments, as shown in fig. 1, the present embodiment provides a facility crop phenotype acquisition device, including a mobile platform 1, a mounting bracket 2, and a camera assembly 3; the moving platform 1 is located to the installing support 2, and the installing support 2 is located to the subassembly 3 that makes a video recording, and the subassembly 3 that makes a video recording includes first RGB-D camera 31 and second RGB-D camera 32, and first RGB-D camera 31 is used for locating the left side or the right side of crop row 6, and second RGB-D camera 32 is used for locating the top of crop row 6.
It is understood that the mobile platform 1 is used for walking between two adjacent ridges 7; the mobile platform 1 comprises an objective table and a travelling mechanism, wherein the objective table is used for carrying equipment of the phenotype acquisition device, the travelling mechanism is connected with the objective table, and the travelling mechanism is used for driving the objective table to travel. The travelling mechanism comprises a driving motor and travelling wheels, and the driving motor is used for driving travelling wheels to travel and steer.
In some examples, the mounting bracket 2 may be configured as a fixed bracket in the event that the height and length ranges of the mounting bracket 2 meet the image acquisition requirements for the crop row 6.
Optionally, the fixed support comprises a vertical support and a horizontal support, the vertical support is arranged on the mobile platform 1, and the vertical support is vertically connected with the horizontal support; the first RGB-D camera 31 is arranged on the vertical support, and the lens of the first RGB-D camera 31 faces the side face of the crop row 6; the second RGB-D camera 32 is arranged on a horizontal stand with the lens of the second RGB-D camera 32 facing the top of the crop row 6.
In some examples, to facilitate phenotypic harvesting of different types of crops, the mounting bracket 2 may be configured as an adjustable bracket.
Optionally, the adjustable bracket comprises a vertical bracket and a horizontal bracket, the vertical bracket is arranged on the mobile platform 1, and the vertical bracket is vertically connected with the horizontal bracket; wherein the length of at least one of the vertical support and the horizontal support is adjustable; the first RGB-D camera 31 is arranged on the vertical bracket, and the lens of the first RGB-D camera 31 faces the side face of the crop row 6; the second RGB-D camera 32 is arranged on a horizontal stand with the lens of the second RGB-D camera 32 facing the top of the crop row 6.
The first RGB-D camera 31 and the second RGB-D camera 32 in the camera assembly 3 can collect RGB images and depth images of crops at the same time, the RGB images obtain images with various colors through the changes of three color channels of red (R), green (G) and blue (B) and the superposition of the three color channels, and the depth images can reflect the geometric shape of the visible surface of the crops, so that depth information is provided for the construction of the 3D space model of the follow-up crops.
Wherein, the first RGB-D camera 31 is used for collecting the left or right video of the crops, and the second RGB-D camera 32 is used for collecting the overlook video of the crops.
As shown in fig. 2, the present embodiment further provides a phenotype collection method based on a facility crop phenotype collection device, where the execution subject of the phenotype collection method may be a control device disposed on the mobile platform 1, or may be a server communicatively connected to the control device, and the phenotype collection method includes the following steps:
step 211, the mobile platform 1 is controlled to move along the left side and the right side of the same crop row 6 respectively, so as to acquire the video acquisition information of the crop row 6.
It is understood that the invention is mainly used for facility greenhouses or multi-span greenhouses, and the crop object used can be any one of vegetables, green plants, flowers and grain crops.
As shown in fig. 1, when crops or plants are planted in a greenhouse, in order to facilitate inspection and management, the plants are generally uniformly distributed according to the ridges 7, and furrows are arranged between two adjacent ridges 7. The ridge 7 is a soil post which is separated from the surface of the land by a certain width, the ridge is called a ridge, and a ditch formed between two adjacent ridges is called a furrow which can be used for irrigation or walking according to planting requirements.
The moving platform 1 moves along the left side of a certain crop row 6, the first RGB-D camera 31 arranged on the moving platform 1 collects video of the left side of the crop, the second RGB-D camera 32 collects video of the top of the crop, then the moving platform 1 moves along the right side of the certain crop row 6, the first RGB-D camera 31 arranged on the moving platform 1 collects video of the right side of the crop, and the second RGB-D camera 32 collects video of the top of the crop.
Meanwhile, after the phenotype acquisition device completes one-time detection, the information of each crop of the crop row 6 is acquired, so that not only can the large-batch acquisition of crop videos be realized, but also the rapid acquisition of the crops of the crop row 6 can be ensured as independent acquisition operation of each crop is not needed, and the acquisition efficiency is greatly improved.
Step 212, obtaining left-view images, right-view images and top-view images of each crop in the crop row 6 according to the video acquisition information.
It can be understood that after acquiring the collected video of all the crops in the crop row 6, the video information of the left side and the top of the crop row 6 is calibrated with the video information of the right side and the top, the left side video and the right side video of each crop are matched, the crop video is decomposed into a plurality of images of the crops by adopting a video frame extraction mode, and the left view image, the right view image and the top view image of the same crop are acquired.
The left view image, the right view image and the overlook image of the obtained crop comprise RGB images and depth images, so that the phenotype images of the crop with multiple view angles and depth features can be obtained, and the canopy structure of the crop can be reflected in a three-dimensional mode.
According to the embodiment, the information on the left side and the right side of the same plant is respectively acquired, and after calibration, the accuracy of the acquired information of the plant is improved, surrounding environment information is effectively filtered, and the acquired data can accurately restore the phenotype image of each plant of plant.
And step 213, building a 3D space model of the crop according to the left view image, the right view image and the overlook image.
It can be understood that, based on the crop video acquired by the first RGB-D camera 31 and the second RGB-D camera 32, a 3D space model of the crop is established by using the acquired left-view image, right-view image and top view image of the same crop, and the 3D space model contains color, depth and space image features at the same time.
And step 214, extracting features of the 3D space model to obtain the phenotype parameters of the crops.
It can be understood that the phenotypic parameters such as plant height, leaf number, canopy coverage, leaf area index, leaf angle and the like of crops are estimated by extracting color, shape and texture characteristics through a 3D space model of the crops by image segmentation.
According to the embodiment, the crop phenotype parameters including the plant height, the leaf number, the canopy coverage rate, the leaf area index and the leaf included angle of the crops are obtained, a good foundation is laid for the research on the growth and development of the crops, and data support is provided for subsequent cultivation of crops with higher yield, higher quality and higher stress resistance, so that the breeding improvement efficiency is improved.
Meanwhile, all crops in the same crop row 6 can realize batch processing of images, so that rapid, automatic and nondestructive acquisition of phenotype parameters of all crops in a facility environment is realized.
According to the phenotype acquisition method based on the facility crop phenotype acquisition device, the mobile platform 1 is controlled to move along the left side and the right side of the same crop row 6 respectively, videos of crops containing RGB features and depth features are acquired by using the first RGB-D camera 31 and the second RGB-D camera 32, left-view images, right-view images and overlook images of the crops are obtained through data calibration and matching, a 3D space model of the crops is established according to the phenotype images, and the color, the shape and the texture features of the crops are extracted from the 3D space model, so that the phenotype parameters of the crops are obtained.
Therefore, the phenotype collection method based on the facility crop phenotype collection device provided by the invention is convenient to operate, and can realize rapid, accurate and nondestructive large-scale phenotype measurement of facility crops.
In some embodiments, as shown in fig. 1 and 3, the step of controlling the mobile platform 1 to move along the left side and the right side of the same crop row 6 to obtain video acquisition information of the crop row 6 in this embodiment includes:
in step 311, the mobile platform 1 is controlled to move along the extending direction of the crop row 6 on the left side of the crop row 6, so as to obtain the first video information collected by the first RGB-D camera 31 and the second video information collected by the second RGB-D camera 32.
In step 312, the mobile platform 1 is controlled to move along the extending direction of the crop row 6 on the right side of the crop row 6, so as to acquire the third video information acquired by the first RGB-D camera 31 and the fourth video information acquired by the second RGB-D camera 32.
Correspondingly, the step of acquiring left-view images, right-view images and top-view images of each crop in the crop row 6 according to the video acquisition information comprises the following steps:
and calibrating and matching the first video information and the third video information with the second video information and the fourth video information to obtain left-view images, right-view images and overlooking images of each plant of crops in the crop row 6.
It will be appreciated that the mobile platform 1 moves first along the left side of the crop row 6, the first video information collected by the first RGB-D camera 31 is the left side video of the crop, the second video information collected by the second RGB-D camera 32 is the top video of the crop, then the mobile platform 1 moves along the right side of the crop row 6, the third video information collected by the first RGB-D camera 31 is the right side video of the crop, and the fourth video information collected by the second RGB-D camera 32 is the top video of the crop.
And calibrating the first video information and the third video information, matching the same crop, calibrating the second video information and the fourth video information, matching the same crop to obtain videos on the left side, the right side and the top of each crop, decomposing the crop video into images of a plurality of crops by adopting a video frame extraction mode, and obtaining left-view images, right-view images and overlooking images of the same crop.
In some embodiments, as shown in fig. 1, the moving speed of the moving platform 1 of the present embodiment along the extending direction of the crop line 6 is the same on the left and right sides of the crop line 6.
Wherein the moving speed of the moving platform 1 is determined according to the type of crop.
In the left side and the right side of the crop row 6, the moving speed of the moving platform 1 along the extending direction of the crop row 6 is the same, so that videos acquired by the camera module on the left side and the right side of the same crop row 6 are matched, and the later calibration and matching work of the same crop are facilitated.
Meanwhile, the method is set according to the specific type of crops, the moving speed of the moving platform 1 is set to be a first speed for crops with larger canopy structures, and the moving speed of the moving platform 1 is set to be a second speed for crops with smaller canopy structures, and the first speed is set to be smaller than the second speed in the embodiment, so that video information collected by the camera assembly 3 can be ensured to cover each crop under the condition that the crop canopy is larger, a foundation is made for matching video frames of each crop with each view in the later stage, and under the condition that the crop canopy is smaller, the accuracy of the video information collected by the camera assembly 3 can be ensured, and the collection efficiency can be improved.
In some embodiments, as shown in fig. 1, the phenotype acquisition device of the present embodiment further comprises a colorimetric plate 4; the colorimetric plate 4 is disposed on the mounting bracket 2, and the colorimetric plate 4 is disposed opposite to the lens of at least one of the first RGB-D camera 31 and the second RGB-D camera 32.
Accordingly, the phenotype acquisition method further comprises: the color of the left-view image, the right-view image, and the top-view image is corrected by using the color chart 4.
It is understood that the color plate 4 may be disposed one, the single color plate 4 may be disposed opposite the first RGB-D camera 31, and the single color plate 4 may be disposed opposite the second RGB-D camera 32.
The two colorimetric plates 4 can be further arranged, wherein one colorimetric plate 4 is arranged opposite to the first RGB-D camera 31, and the other colorimetric plate 4 is arranged opposite to the second RGB-D camera 32, so that the colorimetric plate 4 can be used as a color background of crops and can be collected simultaneously with the crops when the first RGB-D camera 31 and the second RGB-D camera 32 collect videos.
Accordingly, in the later image processing process, the colorimetric plate 4 can provide color correction for the collected left-view image, right-view image and overlook image of the crops, so that color differences caused by environmental factors such as illumination, reflection and the like are reduced to the greatest extent, the color characteristics of the crops such as the color of the leaves of the crops, the maturity of the fruits and the like are subjected to standard color processing, the standardized restoration of the crop images is realized, and the extraction of the color characteristics of the crops is ensured to be more accurate.
In some embodiments, as shown in fig. 1, the mounting bracket 2 of the present embodiment includes a vertical expansion bracket 21 and a horizontal expansion bracket 22.
The bottom of the vertical expansion bracket 21 is connected with the mobile platform 1, and the top of the vertical expansion bracket 21 is vertically connected with the horizontal expansion bracket 22.
The first RGB-D camera 31 is provided on the vertical expansion bracket 21, and the second RGB-D camera 32 is provided on the horizontal expansion bracket 22.
It will be appreciated that the vertical expansion bracket 21 may be a telescopic rod, which is a sleeved structure of a round rod and a sleeve rod, and the expansion of the vertical expansion bracket 21 is realized by the relative sliding of the round rod and the sleeve rod. The vertical expansion bracket 21 can also be an adjustable expansion bracket, the two sections of bar frames can slide relatively, and after the positions of the two sections of bar frames are adjusted properly, the vertical expansion bracket 21 is fixed through a locking piece. When the length of the vertical expansion bracket 21 is changed, the height of the first RGB-D camera 31 relative to the horizontal plane is changed, so that the position of the first RGB-D camera 31 can be adjusted to a position facing the crops.
Meanwhile, the horizontal expansion bracket 22 may be an expansion link, and the horizontal expansion bracket 22 may also be an adjustable expansion bracket, which will not be described in detail herein. When the length of the horizontal telescopic frame 22 is changed, the distance between the second RGB-D camera 32 and the vertical telescopic frame 21 is changed, so that the position of the second RGB-D camera 32 can be adjusted to the top of crops.
Meanwhile, the colorimetric plate 4 may be provided on the vertical expansion bracket 21 or the horizontal expansion bracket 22. Preferably, as shown in fig. 1, the color plate 4 is located on a vertical expansion bracket 21.
When the phenotype acquisition operation is carried out, the vertical telescopic frame 21 is positioned on the left side or the right side of the crop row 6, and the horizontal telescopic frame 22 is positioned on the top of the crop row 6, so that the phenotype acquisition device can carry out video acquisition on crops in a short distance.
According to the embodiment, the vertical telescopic frame 21 and the horizontal telescopic frame 22 are arranged, so that the phenotype acquisition device can be differentially adjusted according to the plant height and the volume of different crops through the telescopic operation of the vertical telescopic frame and the horizontal telescopic frame, different crops can be better adapted, crop damage caused by the limitation of the height and the width when the video is acquired by the crops is avoided, and the wide adaptability of the phenotype acquisition device to different types of crops is ensured.
In some embodiments, as shown in fig. 1, the optical axis of the first RGB-D camera 31 of the present embodiment is disposed horizontally, and the vertical telescopic frame 21 is used to adjust the height of the optical axis of the first RGB-D camera 31 relative to the horizontal plane.
The optical axis of the second RGB-D camera 32 is vertically disposed, and the horizontal expansion bracket 22 is used for adjusting the distance between the optical axis of the second RGB-D camera 32 and the vertical expansion bracket 21.
It can be understood that the present embodiment changes the position of the first RGB-D camera 31 disposed on the vertical expansion bracket 21 by adjusting the length of the vertical expansion bracket 21, so that the height of the optical axis of the first RGB-D camera 31 with respect to the horizontal plane changes, and the first RGB-D camera 31 can face the left side or the right side of the crop canopy.
Meanwhile, in this embodiment, by adjusting the length of the horizontal expansion bracket 22, the position of the second RGB-D camera 32 disposed on the horizontal expansion bracket 22 is changed, so that the distance between the optical axis of the second RGB-D camera 32 and the vertical expansion bracket 21 is changed, and the second RGB-D camera 32 can be opposite to the top of the crop canopy.
According to the embodiment, the acquisition ranges of the first RGB-D camera 31 and the second RGB-D camera 32 are adjusted at the same time, so that the phenotype acquisition device can adapt to the height and the volume of different crops, acquire the video of the close-range acquisition crops, and ensure the acquisition accuracy.
In some embodiments, as shown in fig. 1, the image capturing assembly 3 of the present embodiment further includes a first light source 33 and a second light source 34; the first light source 33 is disposed on one side of the first RGB-D camera 31, and the second light source 34 is disposed on one side of the second RGB-D camera 32.
Accordingly, the phenotype acquisition method further comprises: in the process of acquiring the video acquisition information, the first light source 33 and the second light source 34 are controlled to be turned on.
The ambient light intensity is obtained, and the light emitting powers of the first light source 33 and the second light source 34 are adjusted according to the ambient light intensity.
It will be appreciated that the first light source 33 is configured to provide a lighting environment for image acquisition by the first RGB-D camera 31 and the second light source 34 is configured to provide a lighting environment for image acquisition by the second RGB-D camera 32 to ensure that there is sufficient light source for the first RGB-D camera 31 and the second RGB-D camera 32 to acquire crop video.
The first light source 33 and the second light source 34 may be LED lamps or incandescent lamps.
Meanwhile, in this embodiment, a light intensity sensor is provided on the mounting bracket 2 to obtain the ambient light intensity at any time. Under the condition of insufficient ambient illumination intensity, the current intensity or the voltage intensity of the first light source 33 and the second light source 34 is adjusted to adjust the luminous power of the first light source 33 and the second light source 34, so that the first light source 33 and the second light source 34 can automatically adjust the brightness intensity according to the ambient illumination intensity, shadows of crop canopy views are reduced to a large extent, the influence of illumination and shadows of collected videos is reduced, the accuracy of video collection is ensured, and a sufficient light source following the change of ambient illumination intensity is provided when the video is collected by the first RGB-D camera 31 and the second RGB-D camera 32.
In some embodiments, as shown in fig. 1, in order to facilitate adjustment of the light emission luminance of the first light source 33 and the second light source 34, at least one of the first light source 33 and the second light source 34 of the present embodiment includes an LED lamp.
Alternatively, both the first light source 33 and the second light source 34 may be provided as LED lamps.
In some embodiments, as shown in fig. 1 and 4, the phenotype acquisition apparatus of the present embodiment further comprises a control device 5; the control device 5 comprises a power supply module 51, a control module 52 and a communication module 53.
The power supply module 51 is electrically connected with the control module 52, the communication module 53 and the mobile platform 1 respectively; the mobile platform 1 and the camera assembly 3 are respectively connected with a control module 52, the control module 52 is connected with a communication module 53, and the communication module 53 is used for communicating with a remote control terminal.
The control device 5 comprises a power supply module 51, a control module 52, a communication module 53 and a storage module 54. The power supply module 51 is electrically connected with the control module 52, the communication module 53 and the storage module 54, and the control module 52 is connected with the communication module 53 and the storage module 54.
It is understood that the power supply module 51 includes a power module, an early warning module and a power management module, the power management module is electrically connected to the power module, and the early warning module and the power management module are electrically connected to the control module 52 respectively.
The power module may be a rechargeable lead-acid battery or a lithium battery. When the power management module detects that the electric quantity of the power module is lower than a preset value, the control module 52 can control the early warning module to send out early warning instructions so as to prompt an operator to charge the power module or replace the power module.
The control module 52 includes a PLC controller or a single chip microcomputer. The control module 52 connects the power supply module 51, the communication module 53, and the storage module 54. The storage module 54 is used for storing video information collected by the first RGB-D camera 31 and the second RGB-D camera 32.
The control module 52 not only can control the first RGB-D camera 31 and the second RGB-D camera 32 to collect images of crops, but also can adjust the on-off states and the luminous power of the first light source 33 and the second light source 34 in the process.
Meanwhile, the operator can also send an image acquisition control instruction to the control module 52 through the communication module 53, and the control module 52 responds to the image acquisition control instruction to control the travelling direction and speed of the mobile platform 1, so as to complete the acquisition operation of crop videos.
In addition, the control module 52 also controls the communication module 53 to establish a communication connection with the remote control terminal, so that, for the video information collected by the first RGB-D camera 31 and the second RGB-D camera 32 and the related processing data of the control module 52, the control module 52 can also upload the video information to the remote control terminal through the communication module 53.
Wherein the communication module 53 includes at least one of a bluetooth module and a GSM module.
In some examples, the control device 5 may be configured with a waterproof, moisture-proof housing within which the above-described power module 51, control module 52, and communication module 53 are disposed, thereby achieving an integrated packaging of the power module 51, control module 52, and communication module 53 based on the housing.
As shown in fig. 5, the present embodiment can collect the superficial parameters of the crop by referring to the following steps:
(1) Hardware deployment and initialization
The phenotype collecting device is placed between two adjacent ridges 7 of the facility environment, the lengths of the vertical telescopic frame 21 and the horizontal telescopic frame 22 are adjusted, and the adaptation of the sizes and the volumes of the vertical telescopic frame 21 and the horizontal telescopic frame 22 and crops in the shed is ensured. At this time, the first RGB-D camera 31, the second RGB-D camera 32, the first light source 33, the second light source 34, and the driving motor of the moving platform 1 are in the off state. And (3) starting a power supply of the phenotype acquisition device, and testing whether the communication between the phenotype acquisition device and the remote control terminal is normal or not.
(2) Facility environmental crop phenotype rapid harvesting
By manually or remotely starting the first RGB-D camera 31, the second RGB-D camera 32, the first light source 33 and the second light source 34 of the phenotype acquisition device, the system automatically detects whether the first RGB-D camera 31, the second RGB-D camera 32, the first light source 33 and the second light source 34 are in normal operation or not, and if not in normal operation, an early warning signal is sent. If the detection result is normal, the movement speed of the mobile platform 1 is manually or remotely preset, the driving motor of the mobile platform 1 is started, the mobile platform 1 carries the camera assembly 3 to move forward along the ridge 7 at a certain speed under the driving of the driving motor of the mobile platform 1, the first RGB-D camera 31 and the second RGB-D camera 32 continuously shoot the video of crops, and the acquired video data is transmitted and stored to the control equipment 5.
When the end of the ridge 7 is reached, the mobile platform 1 is manually or remotely controlled to turn to move to the next ridge 7 for collection. The whole device realizes the bidirectional two-side video acquisition of each ridge 7 by an s-shaped motion track in a facility environment.
(3) Intelligent early warning
The control equipment 5 monitors whether each component of the phenotype acquisition device operates normally, and if the phenotype acquisition device fails, early warning information is sent out. The use of the power supply module 51 and the storage module 54 is monitored, and early warning information is sent out in time when the power supply of the power supply module 51 is insufficient, the storage space of the storage module 54 is insufficient and the like.
(4) Crop phenotype image processing
After the phenotype acquisition device finishes acquisition, the power supply is turned off. The video data collected in the storage module 54 is exported through an interface using a data line, or uploaded to a remote control terminal through the communication module 53.
(1) Data calibration: the videos shot twice on the left side and the right side of the same crop row 6 are calibrated and matched. After calibration, the left-view image, the right-view image and the overlook image of the same crop are matched one by one in a video frame extraction mode. And meanwhile, the colorimetric plate 4 is used for realizing the standardized reduction of the crop images and eliminating chromatic aberration caused by environmental conditions such as illumination and the like.
(2) Constructing a crop space model: and constructing a 3D space model of the crop by using the model by using the left view image, the right view image and the depth image of the overlook image of the same crop.
(3) Crop phenotype parameter extraction: through a 3D space model of the crop, the phenotypic parameters such as plant height, leaf number, canopy coverage, leaf area index, leaf included angle and the like of the crop are estimated by utilizing image segmentation, extraction of color, shape and texture features.
(4) Through the batch processing of the images, the rapid, automatic and nondestructive collection of the phenotype parameters of all crops in the facility environment is realized.
Finally, it should be noted that: the above embodiments are only for illustrating the technical solution of the present invention, and are not limiting; while the invention has been described in detail with reference to the foregoing embodiments, it will be appreciated by those skilled in the art that variations may be made in the techniques described in the foregoing embodiments, or equivalents may be substituted for elements thereof; such modifications and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims (10)

1. The phenotype collection method based on the facility crop phenotype collection device is characterized in that the phenotype collection device comprises a mobile platform, a mounting bracket and a camera assembly; the mounting support is arranged on the mobile platform, the camera shooting assembly is arranged on the mounting support, the camera shooting assembly comprises a first RGB-D camera and a second RGB-D camera, the first RGB-D camera is used for being arranged on the left side or the right side of a crop row, and the second RGB-D camera is used for being arranged on the top of the crop row; the phenotype acquisition method comprises the following steps:
controlling the mobile platform to move along the left side and the right side of the same crop row respectively, and acquiring video acquisition information of the crop row;
acquiring left-view images, right-view images and overlook images of each plant of crops in the crop rows according to the video acquisition information;
establishing a 3D space model of the crop according to the left-view image, the right-view image and the overlook image;
and extracting the characteristics of the 3D space model to obtain the phenotype parameters of the crops.
2. The method for harvesting phenotypes based on facility crop phenotypes according to claim 1, wherein the step of controlling the moving platform to move along the left and right sides of the same crop row, respectively, to obtain video harvesting information of the crop row comprises:
controlling the mobile platform to move along the extending direction of the crop row at the left side of the crop row, and acquiring first video information acquired by the first RGB-D camera and second video information acquired by the second RGB-D camera;
controlling the mobile platform to move along the extending direction of the crop row on the right side of the crop row, and acquiring third video information acquired by the first RGB-D camera and fourth video information acquired by the second RGB-D camera;
correspondingly, the step of acquiring left-view images, right-view images and top-view images of each plant of crops in the crop rows according to the video acquisition information comprises the following steps:
and calibrating and matching the first video information and the third video information with the second video information and the fourth video information to obtain the left-view image, the right-view image and the top-view image of each crop in the crop rows.
3. The phenotype acquisition method based on facility crop phenotype acquisition apparatus according to claim 2, wherein the moving speed of the moving platform along the extending direction of the crop row is the same at the left side and the right side of the crop row;
wherein the moving speed of the moving platform is determined according to the type of the crop.
4. The method of claim 1, wherein the phenotype acquisition device further comprises a colorimetric plate; the colorimetric plate is arranged on the mounting bracket, and the colorimetric plate is arranged opposite to the lens of at least one of the first RGB-D camera and the second RGB-D camera;
accordingly, the phenotype acquisition method further comprises: and correcting the colors of the left-view image, the right-view image and the overlook image by adopting the colorimetric plate.
5. The method of claim 1, wherein the mounting bracket comprises a vertical telescoping bracket and a horizontal telescoping bracket;
the bottom end of the vertical expansion bracket is connected with the mobile platform, and the top end of the vertical expansion bracket is vertically connected with the horizontal expansion bracket;
the first RGB-D camera is arranged on the vertical expansion bracket, and the second RGB-D camera is arranged on the horizontal expansion bracket.
6. The phenotype acquisition method based on the facility crop phenotype acquisition device according to claim 5, wherein the optical axis of the first RGB-D camera is horizontally arranged, and the vertical telescopic frame is used for adjusting the height of the optical axis of the first RGB-D camera relative to the horizontal plane;
the optical axis of the second RGB-D camera is arranged vertically, and the horizontal expansion bracket is used for adjusting the distance between the optical axis of the second RGB-D camera and the vertical expansion bracket.
7. The method of claim 1, wherein the camera assembly further comprises a first light source and a second light source; the first light source is arranged on one side of the first RGB-D camera, and the second light source is arranged on one side of the second RGB-D camera;
accordingly, the phenotype acquisition method further comprises: in the process of acquiring the video acquisition information, controlling the first light source and the second light source to be turned on;
and acquiring the ambient illumination intensity, and adjusting the luminous power of the first light source and the luminous power of the second light source according to the ambient illumination intensity.
8. The facility crop phenotype acquisition apparatus based phenotype acquisition method of claim 7 wherein at least one of the first light source and the second light source comprises an LED lamp.
9. The method of phenotype acquisition based on a facility crop phenotype acquisition apparatus according to claim 1, wherein the phenotype acquisition apparatus further comprises a control device; the control equipment comprises a power supply module, a control module and a communication module;
the power supply module is respectively and electrically connected with the control module, the communication module and the mobile platform; the mobile platform and the camera shooting assembly are respectively connected with the control module, the control module is connected with the communication module, and the communication module is used for communicating with a remote control terminal.
10. The method of claim 1, wherein the phenotypic parameters include plant height, leaf number, canopy coverage, leaf area index and leaf angle of the crop.
CN202310260996.9A 2023-03-13 2023-03-13 Phenotype collection method based on facility crop phenotype collection device Pending CN116465887A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202310260996.9A CN116465887A (en) 2023-03-13 2023-03-13 Phenotype collection method based on facility crop phenotype collection device

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202310260996.9A CN116465887A (en) 2023-03-13 2023-03-13 Phenotype collection method based on facility crop phenotype collection device

Publications (1)

Publication Number Publication Date
CN116465887A true CN116465887A (en) 2023-07-21

Family

ID=87172561

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202310260996.9A Pending CN116465887A (en) 2023-03-13 2023-03-13 Phenotype collection method based on facility crop phenotype collection device

Country Status (1)

Country Link
CN (1) CN116465887A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117314716A (en) * 2023-09-06 2023-12-29 深圳市农业科技促进中心 Seed characteristic acquisition equipment and seed characteristic acquisition method

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117314716A (en) * 2023-09-06 2023-12-29 深圳市农业科技促进中心 Seed characteristic acquisition equipment and seed characteristic acquisition method

Similar Documents

Publication Publication Date Title
CN106441442B (en) A kind of high-throughput reciprocity monitoring device of field crop phenotypic information and monitoring method
JP5020444B2 (en) Crop growth measuring device, crop growth measuring method, crop growth measuring program, and computer-readable recording medium recording the crop growth measuring program
CN105717115B (en) High-throughput Plant phenotypic analysis device and method based on optical image technology
CN106596412A (en) Method for monitoring plant growth by using unmanned aerial vehicle with multispectral light source
CN101356877B (en) Cucumber picking robot system and picking method in greenhouse
CN110046613B (en) Crop canopy in-situ growth phenotype monitoring device and three-dimensional reconstruction method
CN105547252B (en) Crop canopies image collecting device based on context aware
KR102301658B1 (en) Apparatus for growing plant and control method of the same
JP5593255B2 (en) Vegetation control device, plant breeding system
CN111561873A (en) Self-propelled seedling tree trunk phenotype information acquisition system and acquisition method thereof
CN105547360B (en) Crop canopies image-pickup method based on context aware
CN112507768A (en) Target detection method and device and image acquisition method and device
CN110121264A (en) Grow the image-taking system and method for cabin assembly line
CN116465887A (en) Phenotype collection method based on facility crop phenotype collection device
CN112036407B (en) Plant population three-dimensional phenotype in-situ acquisition method and device
CN105510335A (en) Growth condition detection method for plants in home-use planting machine
CN113016506A (en) Automatic planting box for acquiring three-dimensional phenotype parameters in edible fungus growth period
CN114279490A (en) Device and method for monitoring diseases and insect pests of field crops
CN106683092B (en) Device and method for measuring and calculating crown canopy density of blueberries
CN115061168A (en) Mobile inspection type crop growth monitoring system and method
CN112816441B (en) Method and device for detecting growth condition of facility horticultural crop
CN110064601B (en) Seedling detection and classification system and classification method for vegetable grafting
CN103499313A (en) Plug seedling detecting system and method
CN103575917A (en) Multispectral image collecting method and device
CN108834667A (en) A kind of greenhouse system based on Internet of Things

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination