WO2023231408A1 - Automatic fruit harvesting apparatus mounted on unmanned aerial vehicle, and control method therefor - Google Patents

Automatic fruit harvesting apparatus mounted on unmanned aerial vehicle, and control method therefor Download PDF

Info

Publication number
WO2023231408A1
WO2023231408A1 PCT/CN2022/143449 CN2022143449W WO2023231408A1 WO 2023231408 A1 WO2023231408 A1 WO 2023231408A1 CN 2022143449 W CN2022143449 W CN 2022143449W WO 2023231408 A1 WO2023231408 A1 WO 2023231408A1
Authority
WO
WIPO (PCT)
Prior art keywords
fruit
scissors
microprocessor
rgb
camera
Prior art date
Application number
PCT/CN2022/143449
Other languages
French (fr)
Chinese (zh)
Inventor
李君�
李灯辉
周峥琦
周浩波
黄光文
林佩怡
贾宇航
姚中威
李钊
伍源水
陈盈宜
Original Assignee
华南农业大学
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 华南农业大学 filed Critical 华南农业大学
Publication of WO2023231408A1 publication Critical patent/WO2023231408A1/en

Links

Classifications

    • AHUMAN NECESSITIES
    • A01AGRICULTURE; FORESTRY; ANIMAL HUSBANDRY; HUNTING; TRAPPING; FISHING
    • A01DHARVESTING; MOWING
    • A01D46/00Picking of fruits, vegetables, hops, or the like; Devices for shaking trees or shrubs
    • A01D46/30Robotic devices for individually picking crops
    • AHUMAN NECESSITIES
    • A01AGRICULTURE; FORESTRY; ANIMAL HUSBANDRY; HUNTING; TRAPPING; FISHING
    • A01DHARVESTING; MOWING
    • A01D91/00Methods for harvesting agricultural products
    • A01D91/04Products growing above the soil
    • BPERFORMING OPERATIONS; TRANSPORTING
    • B64AIRCRAFT; AVIATION; COSMONAUTICS
    • B64CAEROPLANES; HELICOPTERS
    • B64C39/00Aircraft not otherwise provided for
    • B64C39/02Aircraft not otherwise provided for characterised by special use
    • BPERFORMING OPERATIONS; TRANSPORTING
    • B64AIRCRAFT; AVIATION; COSMONAUTICS
    • B64UUNMANNED AERIAL VEHICLES [UAV]; EQUIPMENT THEREFOR
    • B64U10/00Type of UAV
    • B64U10/10Rotorcrafts
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/70Determining position or orientation of objects or cameras
    • G06T7/73Determining position or orientation of objects or cameras using feature-based methods
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/45Cameras or camera modules comprising electronic image sensors; Control thereof for generating image signals from two or more image sensors being of different type or operating in different modes, e.g. with a CMOS sensor for moving images in combination with a charge-coupled device [CCD] for still images
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/57Mechanical or electrical details of cameras or camera modules specially adapted for being embedded in other devices
    • BPERFORMING OPERATIONS; TRANSPORTING
    • B64AIRCRAFT; AVIATION; COSMONAUTICS
    • B64UUNMANNED AERIAL VEHICLES [UAV]; EQUIPMENT THEREFOR
    • B64U2101/00UAVs specially adapted for particular uses or applications
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/10Image acquisition modality
    • G06T2207/10024Color image
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/10Image acquisition modality
    • G06T2207/10028Range image; Depth image; 3D point clouds
    • YGENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02PCLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
    • Y02P60/00Technologies relating to agriculture, livestock or agroalimentary industries
    • Y02P60/12Technologies relating to agriculture, livestock or agroalimentary industries using renewable energies, e.g. solar water pumping

Definitions

  • the invention belongs to the technical field of agricultural machinery, and in particular relates to an automatic fruit harvesting device mounted on a drone and a control method thereof.
  • the main purpose of the present invention is to overcome the shortcomings and deficiencies of the existing technology.
  • the present invention provides an automatic fruit harvesting device mounted on a drone and a control method thereof to improve the automation level of bunch-type fruit picking such as lychees and longans.
  • the present invention provides an automatic fruit harvesting device mounted on a drone, including a fruit positioning component, control and information transmission equipment and a picking mechanism;
  • the fruit positioning component includes an RGB-D camera, a microprocessor and a photoelectric sensor; the RGB-D camera is installed in the middle of the support rod of the picking mechanism, and the RGB-D camera simultaneously acquires the color and outline of the target. and location characteristics; the microprocessor is installed at the top of the drone, and the photoelectric sensor includes a diffuse reflection infrared transmitter and an infrared receiver. The infrared rays emitted by the infrared transmitter have the result that the mother branch enters the middle of the scissors. When the infrared receiver receives the diffusely reflected infrared ray, this information is fed back to the microprocessor in the form of a photoelectric signal;
  • the control and information transmission equipment includes an electrician development board, a relay, a motor control board and a USB-TypeB data line; the relay is connected to the iOS development board, and the motor control boards are a power control board and a drive control board respectively. ;
  • the power control board is composed of an electromagnetic switch and a circuit board and is used to control the start and stop of the motor; the drive control board is used to control the input voltage and current of the motor; the USB-TypeB data line is used to Connecting the chicken development board and the microprocessor enables program downloading and data communication;
  • the picking mechanism includes a battery, a motor protective shell, a motor, a screw rod, a sleeve, a support rod, a steel bar, and scissors with a clamping mechanism;
  • the battery provides electrical energy for the automatic fruit harvesting device;
  • the motor protection The shell and the support rod are fixedly connected with screws;
  • the motor is connected to one end of the screw rod;
  • the other end of the screw rod is connected to the sleeve;
  • the other end of the sleeve is connected to the steel bar with screws;
  • the other end of the steel bar is connected to one blade of the scissors through screws;
  • the other blade of the scissors is fixedly connected to the support rod through screws;
  • the two clamping pieces of the clamping mechanism on the scissors are respectively connected with the scissors.
  • the two blades are secured with screws.
  • the RGB-D camera includes a color camera and an infrared camera; the color camera provides information on three channels of red, green and blue for collecting RGB images; the infrared camera provides A depth information channel is used to collect depth images.
  • the microprocessor has built-in memory, NVIDIA Jetson TX2 GPU and 8GB RAM; the memory is a computer-readable storage device, which stores the relevant information of the fruit positioning algorithm written in the Ubuntu18.04 operating system and Python programming language. Program; NVIDIA Jetson TX2 GPU and 8GB RAM are used to execute programs related to the Python programming language.
  • the UICC development board includes an AVR microcontroller, a crystal oscillator or oscillator and a DC power supply, and contains digital input/output pins, analog inputs, crystal oscillator clock, power jack, ICSP connector and reset button.
  • the processing core of the AVR microcontroller is ATMEGA328P.
  • the relay is connected to the iOS development board through three DuPont lines; the three DuPont lines are respectively connected to the positive and negative poles and signal output ports on the Engineering development board to provide power and input signals to the relay.
  • the motor protective shell is made of 3D printed plastic parts; the motor is a DC motor, used to control the movement of a blade on the scissors; the support rod is made of carbon fiber material
  • the present invention provides a control method for an automatic fruit harvesting device mounted on a drone, which includes the following steps:
  • the drone drives the automatic fruit harvesting device to fly to the first fruit tree
  • RGB-D camera uses an RGB-D camera to collect RGB images and depth images of the fruit and input them to the microprocessor;
  • the blurred image judgment program runs on the microprocessor and only clear images are retained
  • the fruit positioning program runs on the microprocessor, and the positioning information is processed and converted into the flight path information of the UAV, which is transmitted to the flight controller to control the UAV to fly to the destination location;
  • the photoelectric sensor When the photoelectric sensor detects that the fruit-bearing mother branch enters the middle of the scissors, it emits a signal to the microprocessor;
  • the microprocessor After receiving the signal from the photoelectric sensor, the microprocessor runs the fruit positioning program, locates the fruit again and determines whether the drone has reached the destination location;
  • the microprocessor sends a running program signal to the Engineering development board. After the Engineering development board runs the program, the start relay switch is closed;
  • the DC motor starts and links the sleeve through the screw rod.
  • the sleeve drives the steel bar, and the steel bar drives a blade of the scissors to move and cut off the fruit-bearing mother branches;
  • the clamping device on the scissors clamps the fruit-bearing mother branch and sends a signal to the flight controller;
  • the drone flies above the fruit collection basket, opens the scissors, and the fruit falls into the fruit collection basket to complete fruit picking.
  • a fuzzy image judgment program is run on the microprocessor and only clear images are retained, specifically as follows:
  • the RGB image collected in real time by the RGB-D camera is processed by the LoG operator and the gray variance of the RGB image is calculated. If the gray variance value is lower than the preset threshold k, it can be automatically determined to be a blurred image and needs to be collected again. The image is then judged until the grayscale variance value of the collected RGB image is higher than the threshold, and then this clear RGB image and depth image can be used as the input image.
  • a fruit positioning program is run on the microprocessor to process the positioning information and convert it into UAV flight path information, specifically as follows:
  • the maximum circumscribed rectangular boxes of multiple candidate fruits are output and the corresponding confidence values are given, and the areas marked by all rectangular boxes with confidence levels greater than 0.75 are saved to obtain the RGB location of the fruit.
  • Pixel coordinates in the image map this coordinate to the optimized depth image to extract the spatial information of the fruit.
  • the RGB-D shape of the fruit is obtained.
  • the spatial coordinates in the camera coordinate system; through the positional relationship between the RGB-D camera and the center of the drone, the coordinates in the RGB-D camera coordinate system are converted into coordinates in the drone coordinate system, and the drone can be calculated The flight destination coordinates of the aircraft.
  • the DC motor is started, and the sleeve is linked through the screw rod.
  • the sleeve drives the steel bar, and the steel bar drives the blade on one side of the scissors to move to cut off the fruit-bearing mother branches, specifically:
  • the DC motor starts to rotate after starting, driving the screw rod to move in a circle.
  • the screw rod and the sleeve are connected through threads.
  • the sleeve moves horizontally on the screw rod.
  • the sleeve and the steel bar are fixed together.
  • the steel bar and the scissors are connected.
  • the blade on one side is fixed. At this time, the blade on one side of the scissors makes an arc-like motion and closes with the fixed blade on the other side of the scissors to cut off the fruit-bearing mother branch.
  • the present invention has the following advantages and beneficial effects:
  • the present invention is mounted on a drone platform and can adapt to the complex terrain conditions of hilly orchards, effectively solving the problem that picking robots cannot work when facing fruits on tall fruit trees.
  • the present invention combines an RGB-D camera, a microprocessor, an UPU development board and a picking mechanism to realize automatic positioning and automatic control of picking of fruits, providing a foundation for the development of automatic picking drones.
  • the picking mechanism proposed by the present invention is equipped with a clamping device, which can cut off the fruit branches while preventing the fruits from falling, and can effectively avoid damage caused by the fruits falling after being picked.
  • Figure 1 is an overall schematic diagram of the automatic fruit harvesting device according to Embodiment 1 of the present invention.
  • FIG. 2 is a schematic structural perspective view of the automatic fruit harvesting device according to Embodiment 1 of the present invention.
  • Figure 3 is an overall schematic diagram of the automatic fruit harvesting device mounted on a drone according to Embodiment 1 of the present invention
  • Figure 4 is a flow chart of the control method of the automatic fruit harvesting device in Embodiment 2 of the present invention.
  • the terms “mounted”, “connected”, “connected”, “fixed” and other terms should be interpreted broadly. For example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection, or it can be an electrical connection; it can be a direct connection, or it can be an indirect connection through an intermediate medium, or it can be an internal connection between two components. Connectivity can also be surface contact only. For those of ordinary skill in the art, the specific meanings of the above terms in this application can be understood according to specific circumstances.
  • this embodiment provides an automatic fruit harvesting device mounted on a drone, including a fruit positioning component 01, a control and information transmission device 02, and a picking mechanism 03.
  • the fruit positioning component 01 includes an RGB-D camera 0101, a microprocessor (host computer) 0102 and a photoelectric sensor 0103; the RGB-D camera 0101 is installed on the support of the picking mechanism.
  • the middle position of pole 0306 is composed of a color camera and an infrared camera; the color camera provides information on three channels of red, green and blue, used to collect RGB images; the infrared camera provides a depth information channel, which is used to collect RGB images.
  • the RGB-D camera 0101 can simultaneously obtain the color, contour and position characteristics of the target;
  • the microprocessor 0102 is installed at the top of the drone, with built-in memory, NVIDIA Jetson TX2 GPU and 8GB RAM;
  • the memory is a computer-readable storage device, which stores the relevant programs of the fruit positioning algorithm written in the Ubuntu18.04 operating system and Python programming language;
  • the NVIDIA Jetson TX2 GPU and 8GB RAM are used to execute related programs in Python programming language;
  • the photoelectric sensor 0103 contains a diffuse reflection infrared transmitter and an infrared receiver.
  • the infrared rays emitted by the infrared transmitter have results when the mother branch enters the middle of the scissors 0309 , the infrared receiver will receive the diffusely reflected infrared rays and feedback this information to the microprocessor 0102 in the form of photoelectric signals.
  • the control and information transmission device 02 includes an chicken development board 0201, a relay 0202, a motor control board 0203, a USB-TypeB data line 0204 and related circuit lines;
  • the PC development board 0201 It consists of an AVR microcontroller, a crystal oscillator or oscillator and a DC power supply. It contains digital input/output pins (6 of which can be used as PWM output), analog input, crystal oscillator clock, power jack, ICSP connector and reset button;
  • the processing core of the AVR microcontroller is ATMEGA328P;
  • the relay 0202 is an electrical control device, an automatic switch that uses small current to control large current operation.
  • the power control board is composed of an electromagnetic switch and a circuit board and is used to control the start and stop of the motor 0303; the drive control board is used to control the input voltage and current of the motor 0303; the USB-TypeB data line 0204 Used to connect the chicken development board 0201 and the microprocessor 0102 to enable program downloading and data communication; the related circuit lines provide power and data transmission for the entire control device.
  • the picking mechanism 03 includes a battery 0301, a motor protective shell 0302, a motor 0303, a screw rod 0304, a sleeve 0305, a support rod 0306, a steel bar 0307, and scissors with a clamping mechanism 0308.
  • the battery 0301 provides electric energy for the automatic fruit harvesting device; the motor protective shell 0302 and the support rod 0306 are fixedly connected by screws; the motor 0303 is connected to one end of the screw 0304; the wire The other end of the rod 0304 is connected to the sleeve 0305; the other end of the sleeve 0305 is connected to the steel bar 0307 with screws; the other end of the steel bar 0307 is connected to a blade of the scissors 0309 through screws; The other blade of the scissors 0309 is fixedly connected to the support rod 0306 through screws; the two clamping pieces of the clamping mechanism 0308 on the scissors 0309 are respectively fixed with the two blades of the scissors 0309 with screws; the motor protective shell 0302 is made of 3D printed plastic parts; the motor 0303 is a DC motor, used to control the movement of the blade on the scissors 0309; the support rod 0306 is made of carbon fiber material.
  • This embodiment provides a control method for an automatic fruit harvesting device mounted on a drone, as shown in Figure 4, including the following steps:
  • the drone drives the automatic fruit harvesting device to fly to the first fruit tree, uses the RGB-D camera 0101 to collect the RGB image and depth image of the fruit, and inputs them to the microprocessor 0102;
  • the RGB image collected in real time by the RGB-D camera 0101 is processed by the LoG operator and the grayscale variance of the RGB image is calculated. If the grayscale variance value is lower than the preset threshold k, it can be automatically determined as a blurred image and needs to be re- Collect the image and then make a judgment. Until the grayscale variance value of the collected RGB image is higher than the threshold, the clear RGB image and depth image can be used as the input image.
  • the fruit positioning program runs on the microprocessor 0102, which processes the positioning information and converts it into the flight path information of the UAV, and transmits it to the flight controller to control the UAV to fly to the destination location.
  • the specific method is:
  • the maximum circumscribed rectangular boxes of multiple candidate fruits are output and the corresponding confidence values are given, and the areas marked by all rectangular boxes with confidence levels greater than 0.75 are saved to obtain the location of the fruit.
  • Pixel coordinates in the RGB image map this coordinate to the optimized depth image to extract the spatial information of the fruit.
  • the RGB- The spatial coordinates in the coordinate system of the D camera 0101 through the positional relationship between the RGB-D camera 0101 and the center of the drone, the coordinates in the coordinate system of the RGB-D camera 0101 are converted into coordinates in the coordinate system of the drone, which can be calculated Get the flight destination coordinates of the drone.
  • the microprocessor 0102 After receiving the signal from the photoelectric sensor 0103, the microprocessor 0102 runs the fruit positioning program, locates the fruit again and determines whether the drone has reached the destination location;
  • the microprocessor 0102 sends a running program signal to the Engineering development board 0201. After the Engineering development board 0201 runs the program, the start relay 0202 switch is closed;
  • the DC motor 0303 starts and links the sleeve 0305 through the screw rod 0304.
  • the sleeve 0305 drives the steel bar 0307, and the steel bar 0307 drives a blade of the scissors 0309 to move to cut off the fruit-bearing mother branches.
  • the specific method is:
  • DC motor 0303 starts to rotate and drives the screw rod 0304 to move in a circle.
  • the screw rod 0304 and the sleeve 0305 are connected through threads.
  • the sleeve 0305 moves horizontally on the screw rod 0304.
  • the sleeve 0305 is fixed to the steel bar 0307. Together, the steel bar 0307 and the blade on one side of the scissors 0309 are fixed. At this time, the blade on one side of the scissors 0309 makes an arc-like motion and closes with the fixed blade on the other side of the scissors 0309 to cut off the fruit-bearing mother branch.
  • the clamping mechanism 0308 on the scissors 0309 clamps the fruit-bearing mother branch and sends a signal to the flight controller;

Landscapes

  • Engineering & Computer Science (AREA)
  • Aviation & Aerospace Engineering (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Environmental Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Human Computer Interaction (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Robotics (AREA)
  • Remote Sensing (AREA)
  • Mechanical Engineering (AREA)
  • Harvesting Machines For Specific Crops (AREA)

Abstract

An automatic fruit harvesting apparatus mounted on an unmanned aerial vehicle, the apparatus comprising: a fruit positioning assembly (01), a control and information transmission device (02) and a picking mechanism (03), wherein the fruit positioning assembly (01) comprises an RGB-D camera (0101), a microprocessor (0102) and a photoelectric sensor (0103); the control and information transmission device (02) comprises an Arduino development board (0201), a relay (0202), an electric motor control board (0203) and a USB-TypeB data cable (0204); and the picking mechanism (03) comprises a battery (0301), an electric motor protective housing (0302), an electric motor (0303), a lead screw (0304), a sleeve (0305), a support rod (0306), a steel bar (0307), and scissors (0309) having a clamping mechanism (0308). When in use, the photoelectric sensor (0103) transmits a signal to the microprocessor (0102) upon detecting that a fruiting mother branch has entered the middle of the scissors (0309); and after receiving the signal from the photoelectric sensor (0103), the microprocessor (0102) runs a fruit positioning program to position fruits and determine whether an unmanned aerial vehicle has reached a destination position. The apparatus is mounted on an unmanned aerial vehicle platform, can adapt to the complex terrain conditions of hilly orchards, and thus effectively solves the problem whereby a picking robot fails to operate in dealing with fruits on tall fruit trees. The present invention further relates to a control method for an automatic fruit harvesting apparatus mounted on an unmanned aerial vehicle.

Description

一种搭载在无人机上的水果自动采收装置及其控制方法An automatic fruit harvesting device mounted on a drone and its control method 技术领域Technical field
本发明属于农业机械的技术领域,特别涉及一种搭载在无人机上的水果自动采收装置及其控制方法。The invention belongs to the technical field of agricultural machinery, and in particular relates to an automatic fruit harvesting device mounted on a drone and a control method thereof.
背景技术Background technique
荔枝、龙眼等串型水果是热带亚热带地区的特产名果,具有重要的经济价值,在中国南部的丘陵地区广泛种植。由于荔枝、龙眼都是簇状生长,果实分布较为分散,因此,此类水果采摘时需要大量的劳动力,成本较高。当前,荔枝、龙眼等串型水果的采摘主要采用人工操作,不仅费力费时,而且随着农村劳动力向非农产业转移,农村富余劳动力逐渐减少。在成熟季节,遇到炎热天气,很容易因为采摘不及时导致果实品质变差。因此,为降低荔枝、龙眼等串型水果的采摘成本,开发能够自动采摘此类水果的农业机器人是具有经济价值的。已经有相关研究者开发了水果收获机器人,在此基础上,根据山地果园的地形条件和高大荔枝、龙眼树上的串果生长特征,需要研发更加合适的收获机器人。Litchi, longan and other bunch-shaped fruits are specialty fruits in tropical and subtropical regions. They have important economic value and are widely planted in the hilly areas of southern China. Since lychees and longans grow in clusters and the fruits are scattered, picking such fruits requires a lot of labor and is costly. Currently, picking bunch-shaped fruits such as lychees and longans is mainly done manually, which is not only laborious and time-consuming, but also as rural labor shifts to non-agricultural industries, the surplus rural labor force is gradually decreasing. During the ripening season, if you encounter hot weather, it is easy for the fruit quality to deteriorate due to delayed picking. Therefore, in order to reduce the cost of picking bunch-shaped fruits such as lychees and longans, it is of economic value to develop agricultural robots that can automatically pick such fruits. Relevant researchers have developed fruit harvesting robots. On this basis, based on the terrain conditions of mountain orchards and the fruit cluster growth characteristics of tall lychee and longan trees, more suitable harvesting robots need to be developed.
近几年来,无人机在农业生产中得到广泛应用,包括植物保护、作物监测、农作物产量评估。与地面行走机械设备相比,无人机具有良好的地形适应性和高效率的优势。因此,无人机可用于在非结构化的果园环境中执行采摘任务,研究在无人机上搭载轻便的采收装置及其控制方法,对水果自动采摘无人机的发展具有重要意义。In recent years, drones have been widely used in agricultural production, including plant protection, crop monitoring, and crop yield assessment. Compared with ground walking mechanical equipment, UAVs have the advantages of good terrain adaptability and high efficiency. Therefore, UAVs can be used to perform picking tasks in unstructured orchard environments. Research on installing lightweight harvesting devices and their control methods on UAVs is of great significance to the development of automatic fruit picking UAVs.
技术问题technical problem
本发明的主要目的在于克服现有技术的缺点与不足,本发明提供了一种搭载在无人机上的水果自动采收装置及其控制方法,提高荔枝、龙眼等串型水果采摘的自动化水平。The main purpose of the present invention is to overcome the shortcomings and deficiencies of the existing technology. The present invention provides an automatic fruit harvesting device mounted on a drone and a control method thereof to improve the automation level of bunch-type fruit picking such as lychees and longans.
技术解决方案Technical solutions
为了达到上述目的,本发明采用以下技术方案:In order to achieve the above objects, the present invention adopts the following technical solutions:
本发明一方面提供了一种搭载在无人机上的水果自动采收装置,包括水果定位组件、控制及信息传输设备和采摘机构;On the one hand, the present invention provides an automatic fruit harvesting device mounted on a drone, including a fruit positioning component, control and information transmission equipment and a picking mechanism;
所述的水果定位组件包括RGB-D相机、微型处理器和光电传感器;所述的RGB-D相机安装在采摘机构的支撑杆中间位置,所述的RGB-D相机同时获取目标的颜色、轮廓和位置特征;所述的微型处理器安装在无人机的最上方,所述的光电传感器包含漫反射红外发射器和红外接收器,红外发射器发射的红外线,有结果母枝进入到剪刀中间时,红外接收器会接收到漫反射回来的红外线,将此信息以光电信号的形式反馈给微型处理器;The fruit positioning component includes an RGB-D camera, a microprocessor and a photoelectric sensor; the RGB-D camera is installed in the middle of the support rod of the picking mechanism, and the RGB-D camera simultaneously acquires the color and outline of the target. and location characteristics; the microprocessor is installed at the top of the drone, and the photoelectric sensor includes a diffuse reflection infrared transmitter and an infrared receiver. The infrared rays emitted by the infrared transmitter have the result that the mother branch enters the middle of the scissors. When the infrared receiver receives the diffusely reflected infrared ray, this information is fed back to the microprocessor in the form of a photoelectric signal;
所述的控制及信息传输设备包括Arduino开发板、继电器、电机控制板和USB-TypeB数据线;所述继电器与Arduino开发板相连,所述的电机控制板是分别是电源控制板和驱动控制板;所述的电源控制板由一个电磁开关和电路板组成,用于控制电机的启停;所述的驱动控制板用于控制电机的输入电压和电流;所述的USB-TypeB数据线用于连接Arduino开发板与微型处理器,能够实现程序下载和数据通讯;The control and information transmission equipment includes an Arduino development board, a relay, a motor control board and a USB-TypeB data line; the relay is connected to the Arduino development board, and the motor control boards are a power control board and a drive control board respectively. ; The power control board is composed of an electromagnetic switch and a circuit board and is used to control the start and stop of the motor; the drive control board is used to control the input voltage and current of the motor; the USB-TypeB data line is used to Connecting the Arduino development board and the microprocessor enables program downloading and data communication;
所述采摘机构包括电池、电机保护壳、电机、丝杆、套筒、支撑杆、钢条、带夹持机构的剪刀;所述的电池为水果自动采收装置提供电能;所述的电机保护壳与支撑杆通过螺钉固定连接;所述的电机与丝杆的一端连接;所述的丝杆的另一端与套筒连接;所述的套筒的另一端与钢条用螺钉连接;所述的钢条的另一端与剪刀的一个刀片通过螺钉连接;所述的剪刀的另一个刀片与支撑杆通过螺钉固定连接;所述的剪刀上的夹持机构的两片夹持片分别与剪刀的两个刀片用螺丝固定。The picking mechanism includes a battery, a motor protective shell, a motor, a screw rod, a sleeve, a support rod, a steel bar, and scissors with a clamping mechanism; the battery provides electrical energy for the automatic fruit harvesting device; the motor protection The shell and the support rod are fixedly connected with screws; the motor is connected to one end of the screw rod; the other end of the screw rod is connected to the sleeve; the other end of the sleeve is connected to the steel bar with screws; The other end of the steel bar is connected to one blade of the scissors through screws; the other blade of the scissors is fixedly connected to the support rod through screws; the two clamping pieces of the clamping mechanism on the scissors are respectively connected with the scissors. The two blades are secured with screws.
作为优选的技术方案,所述的RGB-D相机包括彩色相机和红外相机;所述的彩色相机提供了红、绿、蓝三个通道的信息,用于采集RGB图像;所述的红外相机提供了一个深度信息通道,用于采集深度图像。As a preferred technical solution, the RGB-D camera includes a color camera and an infrared camera; the color camera provides information on three channels of red, green and blue for collecting RGB images; the infrared camera provides A depth information channel is used to collect depth images.
作为优选的技术方案,微型处理器内置存储器、NVIDIA Jetson TX2 GPU和8GB RAM;所述的存储器为计算机可读存储设备,其存储有Ubuntu18.04操作系统、Python编程语言编写的水果定位算法的相关程序;所述的NVIDIA Jetson TX2 GPU和8GB RAM用于执行Python编程语言的相关程序。As the preferred technical solution, the microprocessor has built-in memory, NVIDIA Jetson TX2 GPU and 8GB RAM; the memory is a computer-readable storage device, which stores the relevant information of the fruit positioning algorithm written in the Ubuntu18.04 operating system and Python programming language. Program; NVIDIA Jetson TX2 GPU and 8GB RAM are used to execute programs related to the Python programming language.
作为优选的技术方案,所述的Arduino开发板包括一块AVR单片机、一个晶振或振荡器和一个直流电源,并含有数字输入/输出引脚、模拟输入、晶振时钟、电源插孔、ICSP接头和复位按钮。As a preferred technical solution, the Arduino development board includes an AVR microcontroller, a crystal oscillator or oscillator and a DC power supply, and contains digital input/output pins, analog inputs, crystal oscillator clock, power jack, ICSP connector and reset button.
作为优选的技术方案,所述的AVR单片机的处理核心是ATMEGA328P。As a preferred technical solution, the processing core of the AVR microcontroller is ATMEGA328P.
所述继电器通过三条杜邦线与Arduino开发板相连接;所述的三条杜邦线分别连接到Arduino开发板上的正负极和信号输出端口上,用于为继电器提供电源和输入信号。The relay is connected to the Arduino development board through three DuPont lines; the three DuPont lines are respectively connected to the positive and negative poles and signal output ports on the Arduino development board to provide power and input signals to the relay.
作为优选的技术方案,所述的电机保护壳由3D打印的塑料件;所述的电机为直流电机,用于控制剪刀上一刀片的运动;所述的支撑杆采用碳纤维材料As a preferred technical solution, the motor protective shell is made of 3D printed plastic parts; the motor is a DC motor, used to control the movement of a blade on the scissors; the support rod is made of carbon fiber material
本发明另一方面提供了一种搭载在无人机上的水果自动采收装置的控制方法,包括下述步骤:On the other hand, the present invention provides a control method for an automatic fruit harvesting device mounted on a drone, which includes the following steps:
无人机带动水果自动采收装置飞行到第一棵果树前;The drone drives the automatic fruit harvesting device to fly to the first fruit tree;
采用RGB-D相机采集果实的RGB图像和深度图像,输入给微型处理器;Use an RGB-D camera to collect RGB images and depth images of the fruit and input them to the microprocessor;
微型处理器上运行模糊图像判断程序,只保留清晰图像;The blurred image judgment program runs on the microprocessor and only clear images are retained;
微型处理器上运行水果定位程序,将定位信息处理后转换为无人机飞行的路径信息,传输给飞行控制器,控制无人机飞行到目的地位置;The fruit positioning program runs on the microprocessor, and the positioning information is processed and converted into the flight path information of the UAV, which is transmitted to the flight controller to control the UAV to fly to the destination location;
光电传感器检测到水果的结果母枝进入剪刀中间时,向微型处理器发射信号;When the photoelectric sensor detects that the fruit-bearing mother branch enters the middle of the scissors, it emits a signal to the microprocessor;
微型处理器收到光电传感器的信号后运行水果定位程序,再次定位水果并判断无人机是否到达目的地位置;After receiving the signal from the photoelectric sensor, the microprocessor runs the fruit positioning program, locates the fruit again and determines whether the drone has reached the destination location;
微型处理器给Arduino开发板发送运行程序信号,Arduino开发板运行程序后启动继电器开关闭合;The microprocessor sends a running program signal to the Arduino development board. After the Arduino development board runs the program, the start relay switch is closed;
直流电机启动,通过丝杆联动套筒,套筒带动钢条,钢条带动剪刀的一个刀片运动,剪断水果的结果母枝;The DC motor starts and links the sleeve through the screw rod. The sleeve drives the steel bar, and the steel bar drives a blade of the scissors to move and cut off the fruit-bearing mother branches;
剪刀上的夹持装置夹紧水果的结果母枝,并且给飞行控制器发送信号;The clamping device on the scissors clamps the fruit-bearing mother branch and sends a signal to the flight controller;
无人机飞行到水果收集筐上方,打开剪刀,水果落入水果收集筐中,完成水果采摘。The drone flies above the fruit collection basket, opens the scissors, and the fruit falls into the fruit collection basket to complete fruit picking.
作为优选的技术方案,所述微型处理器上运行模糊图像判断程序,只保留清晰图像,具体为:As a preferred technical solution, a fuzzy image judgment program is run on the microprocessor and only clear images are retained, specifically as follows:
将RGB-D相机实时采集的RGB图像经过LoG算子处理并计算得到RGB图像的灰度方差,如果灰度方差值低于预先设定的阈值k,就可以自动判定为模糊图像,需要重新采集图像再做判断,直到所采集RGB图像的灰度方差值高于阈值,才能将此清晰的RGB图像和深度图像作为输入图像。The RGB image collected in real time by the RGB-D camera is processed by the LoG operator and the gray variance of the RGB image is calculated. If the gray variance value is lower than the preset threshold k, it can be automatically determined to be a blurred image and needs to be collected again. The image is then judged until the grayscale variance value of the collected RGB image is higher than the threshold, and then this clear RGB image and depth image can be used as the input image.
作为优选的技术方案,所述微型处理器上运行水果定位程序,将定位信息处理后转换为无人机飞行的路径信息,具体为:As a preferred technical solution, a fruit positioning program is run on the microprocessor to process the positioning information and convert it into UAV flight path information, specifically as follows:
微型处理器上运行水果定位程序后,输出多个备选水果果实的最大外接矩形框并且给出相应的置信度值,将所有置信度大于0.75的矩形框标记的区域保存,获得水果果实在RGB图像中的像素坐标;将此坐标映射到优化的深度图像上,以提取果实的空间信息,通过融合果实在RGB图像中的像素坐标和在深度图像上提取的深度信息,获得果实在RGB-D相机坐标系中的空间坐标;通过RGB-D相机和无人机中心的位置关系,将RGB-D相机坐标系中的坐标转换为无人机坐标系中的坐标,即可计算得出无人机的飞行目的地坐标。After running the fruit positioning program on the microprocessor, the maximum circumscribed rectangular boxes of multiple candidate fruits are output and the corresponding confidence values are given, and the areas marked by all rectangular boxes with confidence levels greater than 0.75 are saved to obtain the RGB location of the fruit. Pixel coordinates in the image; map this coordinate to the optimized depth image to extract the spatial information of the fruit. By fusing the pixel coordinates of the fruit in the RGB image and the depth information extracted on the depth image, the RGB-D shape of the fruit is obtained. The spatial coordinates in the camera coordinate system; through the positional relationship between the RGB-D camera and the center of the drone, the coordinates in the RGB-D camera coordinate system are converted into coordinates in the drone coordinate system, and the drone can be calculated The flight destination coordinates of the aircraft.
作为优选的技术方案,所述直流电机启动,通过丝杆联动套筒,套筒带动钢条,钢条带动剪刀的一侧刀片运动,剪断水果的结果母枝,具体为:As a preferred technical solution, the DC motor is started, and the sleeve is linked through the screw rod. The sleeve drives the steel bar, and the steel bar drives the blade on one side of the scissors to move to cut off the fruit-bearing mother branches, specifically:
直流电机启动后开始转动,带动丝杆做圆周运动,丝杆与套筒之间通过螺纹进行连接,套筒在丝杆上做水平运动,套筒与钢条固定在一起,钢条与剪刀的一侧刀片固定,此时剪刀的一侧刀片做类似圆弧运动,与剪刀的另一侧固定刀片闭合,剪断水果的结果母枝。The DC motor starts to rotate after starting, driving the screw rod to move in a circle. The screw rod and the sleeve are connected through threads. The sleeve moves horizontally on the screw rod. The sleeve and the steel bar are fixed together. The steel bar and the scissors are connected. The blade on one side is fixed. At this time, the blade on one side of the scissors makes an arc-like motion and closes with the fixed blade on the other side of the scissors to cut off the fruit-bearing mother branch.
有益效果beneficial effects
本发明与现有技术相比,具有如下优点和有益效果:Compared with the existing technology, the present invention has the following advantages and beneficial effects:
(1)本发明搭载在无人机平台上,能够适应丘陵果园的复杂地形条件,有效解决了采摘机器人面对高大果树上的果实无法工作的问题。(1) The present invention is mounted on a drone platform and can adapt to the complex terrain conditions of hilly orchards, effectively solving the problem that picking robots cannot work when facing fruits on tall fruit trees.
(2)本发明将RGB-D相机、微型处理器、Arduino开发板和采摘机构结合起来,能够实现水果的自动定位和自动控制采摘,为自动采摘无人机的发展提供了基础。(2) The present invention combines an RGB-D camera, a microprocessor, an Arduino development board and a picking mechanism to realize automatic positioning and automatic control of picking of fruits, providing a foundation for the development of automatic picking drones.
(3)本发明提出的采摘机构带有夹持装置,剪断结果果枝的同时防止果实掉落,能够有效避免水果采摘后掉落导致的损伤。(3) The picking mechanism proposed by the present invention is equipped with a clamping device, which can cut off the fruit branches while preventing the fruits from falling, and can effectively avoid damage caused by the fruits falling after being picked.
附图说明Description of the drawings
为了更清楚地说明本申请实施例中的技术方案,下面将对实施例描述中所需要使用的附图作简单地介绍,显而易见地,下面描述中的附图仅仅是本申请的一些实施例,对于本领域普通技术人员来讲,在不付出创造性劳动的前提下,还可以根据这些附图获得其他的附图。In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the description of the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can also be obtained based on these drawings without exerting creative efforts.
图1是本发明实施例1的水果自动采收装置的整体示意图;Figure 1 is an overall schematic diagram of the automatic fruit harvesting device according to Embodiment 1 of the present invention;
图2是本发明实施例1的水果自动采收装置的结构立体示意图;Figure 2 is a schematic structural perspective view of the automatic fruit harvesting device according to Embodiment 1 of the present invention;
图3是本发明实施例1的水果自动采收装置搭载到无人机上的整体示意图;Figure 3 is an overall schematic diagram of the automatic fruit harvesting device mounted on a drone according to Embodiment 1 of the present invention;
图4是本发明实施例2的水果自动采收装置的控制方法的流程图。Figure 4 is a flow chart of the control method of the automatic fruit harvesting device in Embodiment 2 of the present invention.
附图标号说明:Explanation of reference numbers:
01、水果定位组件;02、控制及信息传输设备;03、采摘机构;0101、RGB-D相机;0102、微型处理器;0103、光电传感器;0201、Arduino开发板;0202、继电器;0203、电机控制板;0204、USB-TypeB数据线;0301、电池;0302、电机保护壳;0303、电机;0304、、丝杆;0305、套筒;0306、支撑杆;0307、钢条;0308、带夹持机构;0309、剪刀。01. Fruit positioning component; 02. Control and information transmission equipment; 03. Picking mechanism; 0101. RGB-D camera; 0102. Microprocessor; 0103. Photoelectric sensor; 0201. Arduino development board; 0202. Relay; 0203. Motor Control board; 0204, USB-TypeB data cable; 0301, battery; 0302, motor protective case; 0303, motor; 0304, screw rod; 0305, sleeve; 0306, support rod; 0307, steel bar; 0308, belt clamp Holding mechanism; 0309, scissors.
本发明的实施方式Embodiments of the invention
为了使本技术领域的人员更好地理解本申请方案,下面将结合本申请实施例中的附图,对本申请实施例中的技术方案进行清楚、完整地描述。显然,所描述的实施例仅仅是本申请一部分实施例,而不是全部的实施例。基于本申请中的实施例,本领域技术人员在没有做出创造性劳动前提下所获得的所有其他实施例,都属于本申请保护的范围。In order to enable those in the technical field to better understand the solution of the present application, the technical solution in the embodiment of the present application will be clearly and completely described below in conjunction with the drawings in the embodiment of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, but not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative efforts shall fall within the scope of protection of this application.
在本申请中,除非另有明确的规定或限定,术语“安装”、“相连”、“连接”、“固定”等术语应做广义理解。例如,可以是固定连接,也可以是可拆卸连接,或一体连接;可以是机械连接,也可以是电连接;可以是直接连接,也可以通过中间媒介间接相连,也可以是两个元件内部的连通,也可以是仅为表面接触。对于本领域的普通技术人员而言,可以根据具体情况理解上述术语在本申请中的具体含义。In this application, unless otherwise expressly stated or limited, the terms "mounted", "connected", "connected", "fixed" and other terms should be interpreted broadly. For example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection, or it can be an electrical connection; it can be a direct connection, or it can be an indirect connection through an intermediate medium, or it can be an internal connection between two components. Connectivity can also be surface contact only. For those of ordinary skill in the art, the specific meanings of the above terms in this application can be understood according to specific circumstances.
实施例Example 11 :
如图1所示,本实施例提供了一种搭载在无人机上的水果自动采收装置,包括水果定位组件01、控制及信息传输设备02和采摘机构03。As shown in Figure 1, this embodiment provides an automatic fruit harvesting device mounted on a drone, including a fruit positioning component 01, a control and information transmission device 02, and a picking mechanism 03.
进一步的,如图2所示,所述的水果定位组件01包括RGB-D相机0101、微型处理器(上位机)0102和光电传感器0103;所述的RGB-D相机0101安装在采摘机构的支撑杆0306中间位置,由彩色相机和红外相机组成;所述的彩色相机提供了红、绿、蓝三个通道的信息,用于采集RGB图像;所述的红外相机提供了一个深度信息通道,用于采集深度图像;所述的RGB-D相机0101可以同时获取目标的颜色、轮廓和位置特征;所述的微型处理器0102安装在无人机的最上方,内置存储器、NVIDIA Jetson TX2 GPU和8GB RAM;所述的存储器为计算机可读存储设备,其存储有Ubuntu18.04操作系统、Python编程语言编写的水果定位算法的相关程序;所述的NVIDIA Jetson TX2 GPU和8GB RAM用于执行Python编程语言的相关程序;所述的光电传感器0103包含漫反射红外发射器和红外接收器,红外发射器发射的红外线,有结果母枝进入到剪刀0309中间时,红外接收器会接收到漫反射回来的红外线,将此信息以光电信号的形式反馈给微型处理器0102。Further, as shown in Figure 2, the fruit positioning component 01 includes an RGB-D camera 0101, a microprocessor (host computer) 0102 and a photoelectric sensor 0103; the RGB-D camera 0101 is installed on the support of the picking mechanism. The middle position of pole 0306 is composed of a color camera and an infrared camera; the color camera provides information on three channels of red, green and blue, used to collect RGB images; the infrared camera provides a depth information channel, which is used to collect RGB images. Used to collect depth images; the RGB-D camera 0101 can simultaneously obtain the color, contour and position characteristics of the target; the microprocessor 0102 is installed at the top of the drone, with built-in memory, NVIDIA Jetson TX2 GPU and 8GB RAM; the memory is a computer-readable storage device, which stores the relevant programs of the fruit positioning algorithm written in the Ubuntu18.04 operating system and Python programming language; the NVIDIA Jetson TX2 GPU and 8GB RAM are used to execute related programs in Python programming language; the photoelectric sensor 0103 contains a diffuse reflection infrared transmitter and an infrared receiver. The infrared rays emitted by the infrared transmitter have results when the mother branch enters the middle of the scissors 0309 , the infrared receiver will receive the diffusely reflected infrared rays and feedback this information to the microprocessor 0102 in the form of photoelectric signals.
进一步的,如图2所示,所述的控制及信息传输设备02包括Arduino开发板0201、继电器0202、电机控制板0203、USB-TypeB数据线0204和相关电路线;所述的Arduino开发板0201由一块AVR单片机、一个晶振或振荡器和直流电源组成,含有数字输入/输出引脚(其中6个可用作PWM输出),模拟输入,晶振时钟,电源插孔,ICSP接头和复位按钮;所述的AVR单片机的处理核心是ATMEGA328P;所述的继电器0202是一种电控制器件,是用小电流控制大电流运作的自动开关,它通过三条杜邦线与Arduino开发板0201相连接;所述的三条杜邦线分别连接到Arduino开发板0201上的正负极和信号输出端口上,用于为继电器0202提供电源和输入信号;所述的电机控制板0203是分别是电源控制板和驱动控制板;所述的电源控制板由一个电磁开关和电路板组成,用于控制电机0303的启停;所述的驱动控制板用于控制电机0303的输入电压和电流;所述的USB-TypeB数据线0204用于连接Arduino开发板0201与微型处理器0102,能够实现程序下载和数据通讯;所述的相关电路线为整个控制设备提供电源和数据传输。Further, as shown in Figure 2, the control and information transmission device 02 includes an Arduino development board 0201, a relay 0202, a motor control board 0203, a USB-TypeB data line 0204 and related circuit lines; the Arduino development board 0201 It consists of an AVR microcontroller, a crystal oscillator or oscillator and a DC power supply. It contains digital input/output pins (6 of which can be used as PWM output), analog input, crystal oscillator clock, power jack, ICSP connector and reset button; The processing core of the AVR microcontroller is ATMEGA328P; the relay 0202 is an electrical control device, an automatic switch that uses small current to control large current operation. It is connected to the Arduino development board 0201 through three DuPont lines; Three DuPont lines are connected to the positive and negative poles and signal output ports on the Arduino development board 0201 respectively, and are used to provide power and input signals to the relay 0202; the motor control board 0203 is the power control board and the drive control board respectively; The power control board is composed of an electromagnetic switch and a circuit board and is used to control the start and stop of the motor 0303; the drive control board is used to control the input voltage and current of the motor 0303; the USB-TypeB data line 0204 Used to connect the Arduino development board 0201 and the microprocessor 0102 to enable program downloading and data communication; the related circuit lines provide power and data transmission for the entire control device.
进一步的,如图2所示,所述的采摘机构03包括电池0301、电机保护壳0302、电机0303、丝杆0304、套筒0305、支撑杆0306、钢条0307、带夹持机构0308的剪刀0309组成;所述的电池0301为水果自动采收装置提供电能;所述的电机保护壳0302与支撑杆0306通过螺钉固定连接;所述的电机0303与丝杆0304的一端连接;所述的丝杆0304的另一端与套筒0305连接;所述的套筒0305的另一端与钢条0307用螺钉连接;所述的钢条0307的另一端与剪刀0309的一个刀片通过螺钉连接;所述的剪刀0309的另一个刀片与支撑杆0306通过螺钉固定连接;所述的剪刀0309上的夹持机构0308的两片夹持片分别与剪刀0309的两个刀片用螺丝固定;所述的电机保护壳0302由3D打印的塑料件;所述的电机0303为直流电机,用于控制剪刀0309上一刀片的运动;所述的支撑杆0306采用碳纤维材料。Further, as shown in Figure 2, the picking mechanism 03 includes a battery 0301, a motor protective shell 0302, a motor 0303, a screw rod 0304, a sleeve 0305, a support rod 0306, a steel bar 0307, and scissors with a clamping mechanism 0308. 0309 consists of; the battery 0301 provides electric energy for the automatic fruit harvesting device; the motor protective shell 0302 and the support rod 0306 are fixedly connected by screws; the motor 0303 is connected to one end of the screw 0304; the wire The other end of the rod 0304 is connected to the sleeve 0305; the other end of the sleeve 0305 is connected to the steel bar 0307 with screws; the other end of the steel bar 0307 is connected to a blade of the scissors 0309 through screws; The other blade of the scissors 0309 is fixedly connected to the support rod 0306 through screws; the two clamping pieces of the clamping mechanism 0308 on the scissors 0309 are respectively fixed with the two blades of the scissors 0309 with screws; the motor protective shell 0302 is made of 3D printed plastic parts; the motor 0303 is a DC motor, used to control the movement of the blade on the scissors 0309; the support rod 0306 is made of carbon fiber material.
水果自动采收装置搭载到无人机上的整体示意图如图3所示。The overall schematic diagram of the automatic fruit harvesting device mounted on a drone is shown in Figure 3.
实施例Example 2:2:
本实施例提供了一种搭载在无人机上的水果自动采收装置的控制方法,如图4所示,包括以下步骤:This embodiment provides a control method for an automatic fruit harvesting device mounted on a drone, as shown in Figure 4, including the following steps:
(1)无人机带动水果自动采收装置飞行到第一棵果树前,采用RGB-D相机0101采集果实的RGB图像和深度图像,输入给微型处理器0102;(1) The drone drives the automatic fruit harvesting device to fly to the first fruit tree, uses the RGB-D camera 0101 to collect the RGB image and depth image of the fruit, and inputs them to the microprocessor 0102;
(2)微型处理器0102上运行模糊图像判断程序,只保留清晰图像,具体方法为:(2) Run the blurred image judgment program on the microprocessor 0102 to retain only clear images. The specific method is:
将RGB-D相机0101实时采集的RGB图像经过LoG算子处理并计算得到RGB图像的灰度方差,如果灰度方差值低于预先设定的阈值k,就可以自动判定为模糊图像,需要重新采集图像再做判断,直到所采集RGB图像的灰度方差值高于阈值,才能将此清晰的RGB图像和深度图像作为输入图像。The RGB image collected in real time by the RGB-D camera 0101 is processed by the LoG operator and the grayscale variance of the RGB image is calculated. If the grayscale variance value is lower than the preset threshold k, it can be automatically determined as a blurred image and needs to be re- Collect the image and then make a judgment. Until the grayscale variance value of the collected RGB image is higher than the threshold, the clear RGB image and depth image can be used as the input image.
(3)微型处理器0102上运行水果定位程序,将定位信息处理后转换为无人机飞行的路径信息,传输给飞行控制器,控制无人机飞行到目的地位置,具体方法为:(3) The fruit positioning program runs on the microprocessor 0102, which processes the positioning information and converts it into the flight path information of the UAV, and transmits it to the flight controller to control the UAV to fly to the destination location. The specific method is:
微型处理器0102上运行水果定位程序后,输出多个备选水果果实的最大外接矩形框并且给出相应的置信度值,将所有置信度大于0.75的矩形框标记的区域保存,获得水果果实在RGB图像中的像素坐标;将此坐标映射到优化的深度图像上,以提取果实的空间信息,通过融合果实在RGB图像中的像素坐标和在深度图像上提取的深度信息,获得果实在RGB-D相机0101坐标系中的空间坐标;通过RGB-D相机0101和无人机中心的位置关系,将RGB-D相机0101坐标系中的坐标转换为无人机坐标系中的坐标,即可计算得出无人机的飞行目的地坐标。After running the fruit positioning program on the microprocessor 0102, the maximum circumscribed rectangular boxes of multiple candidate fruits are output and the corresponding confidence values are given, and the areas marked by all rectangular boxes with confidence levels greater than 0.75 are saved to obtain the location of the fruit. Pixel coordinates in the RGB image; map this coordinate to the optimized depth image to extract the spatial information of the fruit. By fusing the pixel coordinates of the fruit in the RGB image and the depth information extracted on the depth image, the RGB- The spatial coordinates in the coordinate system of the D camera 0101; through the positional relationship between the RGB-D camera 0101 and the center of the drone, the coordinates in the coordinate system of the RGB-D camera 0101 are converted into coordinates in the coordinate system of the drone, which can be calculated Get the flight destination coordinates of the drone.
(4)光电传感器0103检测到水果的结果母枝进入剪刀0309中间时,向微型处理器0102发射信号;(4) When the photoelectric sensor 0103 detects that the fruit-bearing mother branch enters the middle of the scissors 0309, it emits a signal to the microprocessor 0102;
(5)微型处理器0102收到光电传感器0103的信号后运行水果定位程序,再次定位水果并判断无人机是否到达目的地位置;(5) After receiving the signal from the photoelectric sensor 0103, the microprocessor 0102 runs the fruit positioning program, locates the fruit again and determines whether the drone has reached the destination location;
(6)微型处理器0102给Arduino开发板0201发送运行程序信号,Arduino开发板0201运行程序后启动继电器0202开关闭合;(6) The microprocessor 0102 sends a running program signal to the Arduino development board 0201. After the Arduino development board 0201 runs the program, the start relay 0202 switch is closed;
(7)直流电机0303启动,通过丝杆0304联动套筒0305,套筒0305带动钢条0307,钢条0307带动剪刀0309的一个刀片运动,剪断水果的结果母枝,具体方法为:(7) The DC motor 0303 starts and links the sleeve 0305 through the screw rod 0304. The sleeve 0305 drives the steel bar 0307, and the steel bar 0307 drives a blade of the scissors 0309 to move to cut off the fruit-bearing mother branches. The specific method is:
直流电机0303启动后开始转动,带动丝杆0304做圆周运动,丝杆0304与套筒0305之间通过螺纹进行连接,套筒0305在丝杆0304上做水平运动,套筒0305与钢条0307固定在一起,钢条0307与剪刀0309的一侧刀片固定,此时剪刀0309的一侧刀片做类似圆弧运动,与剪刀0309的另一侧固定刀片闭合,剪断水果的结果母枝。DC motor 0303 starts to rotate and drives the screw rod 0304 to move in a circle. The screw rod 0304 and the sleeve 0305 are connected through threads. The sleeve 0305 moves horizontally on the screw rod 0304. The sleeve 0305 is fixed to the steel bar 0307. Together, the steel bar 0307 and the blade on one side of the scissors 0309 are fixed. At this time, the blade on one side of the scissors 0309 makes an arc-like motion and closes with the fixed blade on the other side of the scissors 0309 to cut off the fruit-bearing mother branch.
(8)剪刀0309上的夹持机构0308夹紧水果的结果母枝,并且给飞行控制器发送信号;(8) The clamping mechanism 0308 on the scissors 0309 clamps the fruit-bearing mother branch and sends a signal to the flight controller;
(9)无人机飞行到水果收集筐上方,打开剪刀0309,水果落入水果收集筐中,完成水果采摘。(9) The drone flies above the fruit collection basket, opens the scissors 0309, and the fruit falls into the fruit collection basket, completing fruit picking.
上述实施例为本发明较佳的实施方式,但本发明的实施方式并不受上述实施例的限制,其他的任何未背离本发明的精神实质与原理下所作的改变、修饰、替代、组合、简化,均应为等效的置换方式,都包含在本发明的保护范围之内。The above embodiments are preferred embodiments of the present invention, but the embodiments of the present invention are not limited to the above embodiments. Any other changes, modifications, substitutions, combinations, etc. may be made without departing from the spirit and principles of the present invention. All simplifications should be equivalent substitutions, and are all included in the protection scope of the present invention.

Claims (10)

  1. 一种搭载在无人机上的水果自动采收装置,其特征在于,包括水果定位组件、控制及信息传输设备和采摘机构;An automatic fruit harvesting device mounted on a drone, which is characterized by including a fruit positioning component, control and information transmission equipment and a picking mechanism;
    所述的水果定位组件包括RGB-D相机、微型处理器和光电传感器;所述的RGB-D相机安装在采摘机构的支撑杆中间位置,所述的RGB-D相机同时获取目标的颜色、轮廓和位置特征;所述的微型处理器安装在无人机的最上方,所述的光电传感器包含漫反射红外发射器和红外接收器,红外发射器发射的红外线,有结果母枝进入到剪刀中间时,红外接收器会接收到漫反射回来的红外线,将此信息以光电信号的形式反馈给微型处理器;The fruit positioning component includes an RGB-D camera, a microprocessor and a photoelectric sensor; the RGB-D camera is installed in the middle of the support rod of the picking mechanism, and the RGB-D camera simultaneously acquires the color and outline of the target. and location characteristics; the microprocessor is installed at the top of the drone, and the photoelectric sensor includes a diffuse reflection infrared transmitter and an infrared receiver. The infrared rays emitted by the infrared transmitter have the result that the mother branch enters the middle of the scissors. When the infrared receiver receives the diffusely reflected infrared ray, this information is fed back to the microprocessor in the form of a photoelectric signal;
    所述的控制及信息传输设备包括Arduino开发板、继电器、电机控制板和USB-TypeB数据线;所述继电器与Arduino开发板相连,所述的电机控制板是分别是电源控制板和驱动控制板;所述的电源控制板由一个电磁开关和电路板组成,用于控制电机的启停;所述的驱动控制板用于控制电机的输入电压和电流;所述的USB-TypeB数据线用于连接Arduino开发板与微型处理器,能够实现程序下载和数据通讯;The control and information transmission equipment includes an Arduino development board, a relay, a motor control board and a USB-TypeB data line; the relay is connected to the Arduino development board, and the motor control boards are a power control board and a drive control board respectively. ; The power control board is composed of an electromagnetic switch and a circuit board and is used to control the start and stop of the motor; the drive control board is used to control the input voltage and current of the motor; the USB-TypeB data line is used to Connecting the Arduino development board and the microprocessor enables program downloading and data communication;
    所述采摘机构包括电池、电机保护壳、电机、丝杆、套筒、支撑杆、钢条、带夹持机构的剪刀;所述的电池为水果自动采收装置提供电能;所述的电机保护壳与支撑杆通过螺钉固定连接;所述的电机与丝杆的一端连接;所述的丝杆的另一端与套筒连接;所述的套筒的另一端与钢条用螺钉连接;所述的钢条的另一端与剪刀的一个刀片通过螺钉连接;所述的剪刀的另一个刀片与支撑杆通过螺钉固定连接;所述的剪刀上的夹持机构的两片夹持片分别与剪刀的两个刀片用螺丝固定。The picking mechanism includes a battery, a motor protective shell, a motor, a screw rod, a sleeve, a support rod, a steel bar, and scissors with a clamping mechanism; the battery provides electrical energy for the automatic fruit harvesting device; the motor protection The shell and the support rod are fixedly connected with screws; the motor is connected to one end of the screw rod; the other end of the screw rod is connected to the sleeve; the other end of the sleeve is connected to the steel bar with screws; The other end of the steel bar is connected to one blade of the scissors through screws; the other blade of the scissors is fixedly connected to the support rod through screws; the two clamping pieces of the clamping mechanism on the scissors are respectively connected with the scissors. The two blades are secured with screws.
  2. 根据权利要求1所述一种搭载在无人机上的水果自动采收装置,其特征在于,所述的RGB-D相机包括彩色相机和红外相机;所述的彩色相机提供了红、绿、蓝三个通道的信息,用于采集RGB图像;所述的红外相机提供了一个深度信息通道,用于采集深度图像。An automatic fruit harvesting device mounted on a drone according to claim 1, characterized in that the RGB-D camera includes a color camera and an infrared camera; the color camera provides red, green, and blue Three channels of information are used to collect RGB images; the infrared camera provides a depth information channel for collecting depth images.
  3. 根据权利要求1所述一种搭载在无人机上的水果自动采收装置,其特征在于,微型处理器内置存储器、NVIDIA Jetson TX2 GPU和8GB RAM;所述的存储器为计算机可读存储设备,其存储有Ubuntu18.04操作系统、Python编程语言编写的水果定位算法的相关程序;所述的NVIDIA Jetson TX2 GPU和8GB RAM用于执行Python编程语言的相关程序。An automatic fruit harvesting device mounted on a drone according to claim 1, characterized in that the microprocessor has built-in memory, NVIDIA Jetson TX2 GPU and 8GB RAM; the memory is a computer-readable storage device, and The related programs of the fruit positioning algorithm written in the Ubuntu18.04 operating system and Python programming language are stored; the NVIDIA Jetson TX2 GPU and 8GB RAM are used to execute the related programs of the Python programming language.
  4. 根据权利要求1所述一种搭载在无人机上的水果自动采收装置,其特征在于,所述的Arduino开发板包括一块AVR单片机、一个晶振或振荡器和一个直流电源,并含有数字输入/输出引脚、模拟输入、晶振时钟、电源插孔、ICSP接头和复位按钮。An automatic fruit harvesting device mounted on a drone according to claim 1, characterized in that the Arduino development board includes an AVR microcontroller, a crystal oscillator or oscillator and a DC power supply, and contains digital input/ Output pins, analog inputs, crystal clock, power jack, ICSP header and reset button.
  5. 根据权利要求1所述一种搭载在无人机上的水果自动采收装置,其特征在于,所述的AVR单片机的处理核心是ATMEGA328P。An automatic fruit harvesting device mounted on a drone according to claim 1, characterized in that the processing core of the AVR microcontroller is ATMEGA328P.
    所述继电器通过三条杜邦线与Arduino开发板相连接;所述的三条杜邦线分别连接到Arduino开发板上的正负极和信号输出端口上,用于为继电器提供电源和输入信号。The relay is connected to the Arduino development board through three DuPont lines; the three DuPont lines are respectively connected to the positive and negative poles and signal output ports on the Arduino development board to provide power and input signals to the relay.
  6. 根据权利要求1所述一种搭载在无人机上的水果自动采收装置,其特征在于,所述的电机保护壳由3D打印的塑料件;所述的电机为直流电机,用于控制剪刀上一刀片的运动;所述的支撑杆采用碳纤维材料。An automatic fruit harvesting device mounted on a drone according to claim 1, characterized in that the motor protective shell is made of 3D printed plastic parts; the motor is a DC motor used to control the scissors. The movement of a blade; the support rod is made of carbon fiber material.
  7. 根据权利要求1-6中任一项所述一种搭载在无人机上的水果自动采收装置的控制方法,其特征在于,包括下述步骤:A control method for an automatic fruit harvesting device mounted on a drone according to any one of claims 1-6, characterized in that it includes the following steps:
    无人机带动水果自动采收装置飞行到第一棵果树前;The drone drives the automatic fruit harvesting device to fly to the first fruit tree;
    采用RGB-D相机采集果实的RGB图像和深度图像,输入给微型处理器;Use an RGB-D camera to collect RGB images and depth images of the fruit and input them to the microprocessor;
    微型处理器上运行模糊图像判断程序,只保留清晰图像;The blurred image judgment program runs on the microprocessor and only clear images are retained;
    微型处理器上运行水果定位程序,将定位信息处理后转换为无人机飞行的路径信息,传输给飞行控制器,控制无人机飞行到目的地位置;The fruit positioning program runs on the microprocessor, and the positioning information is processed and converted into the flight path information of the UAV, which is transmitted to the flight controller to control the UAV to fly to the destination location;
    光电传感器检测到水果的结果母枝进入剪刀中间时,向微型处理器发射信号;When the photoelectric sensor detects that the fruit-bearing mother branch enters the middle of the scissors, it emits a signal to the microprocessor;
    微型处理器收到光电传感器的信号后运行水果定位程序,再次定位水果并判断无人机是否到达目的地位置;After receiving the signal from the photoelectric sensor, the microprocessor runs the fruit positioning program, locates the fruit again and determines whether the drone has reached the destination location;
    微型处理器给Arduino开发板发送运行程序信号,Arduino开发板运行程序后启动继电器开关闭合;The microprocessor sends a running program signal to the Arduino development board. After the Arduino development board runs the program, the start relay switch is closed;
    直流电机启动,通过丝杆联动套筒,套筒带动钢条,钢条带动剪刀的一个刀片运动,剪断水果的结果母枝;The DC motor starts and links the sleeve through the screw rod. The sleeve drives the steel bar, and the steel bar drives a blade of the scissors to move and cut off the fruit-bearing mother branches;
    剪刀上的夹持装置夹紧水果的结果母枝,并且给飞行控制器发送信号;The clamping device on the scissors clamps the fruit-bearing mother branch and sends a signal to the flight controller;
    无人机飞行到水果收集筐上方,打开剪刀,水果落入水果收集筐中,完成水果采摘。The drone flies above the fruit collection basket, opens the scissors, and the fruit falls into the fruit collection basket to complete fruit picking.
  8. 根据权利要求7所述的控制方法,其特征在于,所述微型处理器上运行模糊图像判断程序,只保留清晰图像,具体为:The control method according to claim 7, characterized in that a fuzzy image judgment program is run on the microprocessor to retain only clear images, specifically:
    将RGB-D相机实时采集的RGB图像经过LoG算子处理并计算得到RGB图像的灰度方差,如果灰度方差值低于预先设定的阈值k,就可以自动判定为模糊图像,需要重新采集图像再做判断,直到所采集RGB图像的灰度方差值高于阈值,才能将此清晰的RGB图像和深度图像作为输入图像。The RGB image collected in real time by the RGB-D camera is processed by the LoG operator and the gray variance of the RGB image is calculated. If the gray variance value is lower than the preset threshold k, it can be automatically determined to be a blurred image and needs to be collected again. The image is then judged until the grayscale variance value of the collected RGB image is higher than the threshold, and then this clear RGB image and depth image can be used as the input image.
  9. 根据权利要求7所述的控制方法,其特征在于,所述微型处理器上运行水果定位程序,将定位信息处理后转换为无人机飞行的路径信息,具体为:The control method according to claim 7, characterized in that a fruit positioning program is run on the microprocessor to process the positioning information and convert it into UAV flight path information, specifically:
    微型处理器上运行水果定位程序后,输出多个备选水果果实的最大外接矩形框并且给出相应的置信度值,将所有置信度大于0.75的矩形框标记的区域保存,获得水果果实在RGB图像中的像素坐标;将此坐标映射到优化的深度图像上,以提取果实的空间信息,通过融合果实在RGB图像中的像素坐标和在深度图像上提取的深度信息,获得果实在RGB-D相机坐标系中的空间坐标;通过RGB-D相机和无人机中心的位置关系,将RGB-D相机坐标系中的坐标转换为无人机坐标系中的坐标,即可计算得出无人机的飞行目的地坐标。After running the fruit positioning program on the microprocessor, the maximum circumscribed rectangular boxes of multiple candidate fruits are output and the corresponding confidence values are given, and the areas marked by all rectangular boxes with confidence levels greater than 0.75 are saved to obtain the RGB location of the fruit. Pixel coordinates in the image; map this coordinate to the optimized depth image to extract the spatial information of the fruit. By fusing the pixel coordinates of the fruit in the RGB image and the depth information extracted on the depth image, the RGB-D shape of the fruit is obtained. The spatial coordinates in the camera coordinate system; through the positional relationship between the RGB-D camera and the center of the drone, the coordinates in the RGB-D camera coordinate system are converted into coordinates in the drone coordinate system, and the drone can be calculated The coordinates of the aircraft’s flight destination.
  10. 根据权利要求7所述的控制方法,其特征在于,所述直流电机启动,通过丝杆联动套筒,套筒带动钢条,钢条带动剪刀的一侧刀片运动,剪断水果的结果母枝,具体为:The control method according to claim 7, characterized in that the DC motor is started to link the sleeve through the screw rod, the sleeve drives the steel bar, and the steel bar drives one side blade of the scissors to move to cut off the fruit-bearing mother branch. Specifically:
    直流电机启动后开始转动,带动丝杆做圆周运动,丝杆与套筒之间通过螺纹进行连接,套筒在丝杆上做水平运动,套筒与钢条固定在一起,钢条与剪刀的一侧刀片固定,此时剪刀的一侧刀片做类似圆弧运动,与剪刀的另一侧固定刀片闭合,剪断水果的结果母枝。The DC motor starts to rotate after starting, driving the screw rod to move in a circle. The screw rod and the sleeve are connected through threads. The sleeve moves horizontally on the screw rod. The sleeve and the steel bar are fixed together. The steel bar and the scissors are connected. The blade on one side is fixed. At this time, the blade on one side of the scissors makes an arc-like motion and closes with the fixed blade on the other side of the scissors to cut off the fruit-bearing mother branch.
PCT/CN2022/143449 2022-05-30 2022-12-29 Automatic fruit harvesting apparatus mounted on unmanned aerial vehicle, and control method therefor WO2023231408A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202210596569.3 2022-05-30
CN202210596569.3A CN114916318B (en) 2022-05-30 2022-05-30 Automatic fruit harvesting device carried on unmanned aerial vehicle and control method thereof

Publications (1)

Publication Number Publication Date
WO2023231408A1 true WO2023231408A1 (en) 2023-12-07

Family

ID=82812141

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/143449 WO2023231408A1 (en) 2022-05-30 2022-12-29 Automatic fruit harvesting apparatus mounted on unmanned aerial vehicle, and control method therefor

Country Status (2)

Country Link
CN (1) CN114916318B (en)
WO (1) WO2023231408A1 (en)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114916318B (en) * 2022-05-30 2023-05-02 华南农业大学 Automatic fruit harvesting device carried on unmanned aerial vehicle and control method thereof
CN116138039B (en) * 2023-01-10 2024-04-12 华南农业大学 Unmanned fruit picking machine and operation method

Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104899869A (en) * 2015-05-14 2015-09-09 浙江大学 Plane and barrier detection method based on RGB-D camera and attitude sensor
CN105783819A (en) * 2016-03-10 2016-07-20 江苏大学 RGB-D-based automatic transplanting seedling growing condition-operation effect composite detection method
CN108082477A (en) * 2017-12-15 2018-05-29 佛山市神风航空科技有限公司 A kind of unmanned plane for acrophyta sampling
WO2020134283A1 (en) * 2018-12-28 2020-07-02 江苏农林职业技术学院 Vegetable and fruit picker device and picking method
CN111418349A (en) * 2020-03-19 2020-07-17 南京赫曼机器人自动化有限公司 Intelligent fruit picking robot and method for realizing fruit picking
CN113424706A (en) * 2021-06-24 2021-09-24 华南农业大学 Unmanned aerial vehicle self-balancing picking device based on lever principle and control method thereof
DE102020121554A1 (en) * 2020-08-17 2022-02-17 Thorsten Seckert Robotic system and mobile robot for picking stalked fruits of a plant
CN216058333U (en) * 2021-11-08 2022-03-18 南京林业大学 Intelligent movement fruit picking robot
CN114916318A (en) * 2022-05-30 2022-08-19 华南农业大学 Automatic fruit harvesting device carried on unmanned aerial vehicle and control method thereof

Family Cites Families (19)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR101458534B1 (en) * 2014-07-16 2014-11-05 (주)테크맥스텔레콤 Ducted type unmanned aircraft for pine nut picking
CN204741851U (en) * 2015-07-13 2015-11-11 绍兴文理学院 Ware is picked to unmanned aerial vehicle chinese torreya fruit
CN107182442A (en) * 2017-06-24 2017-09-22 匡致宇 Pluck unmanned plane
KR20170118641A (en) * 2017-09-29 2017-10-25 임용철 drones mounting electric and scissors
JP7049814B2 (en) * 2017-11-30 2022-04-07 株式会社デンソー Harvest robot system
KR101999478B1 (en) * 2018-01-29 2019-07-11 강원대학교산학협력단 A drone for fruit harvesting and a system including the same
GB201802335D0 (en) * 2018-02-13 2018-03-28 Norwegian Univ Of Life Sciences Device for picking fruit
CN208453240U (en) * 2018-05-23 2019-02-01 福鼎市巨鼎茶叶科技有限公司 A kind of unmanned plane tea picker
CN209057568U (en) * 2018-09-27 2019-07-05 东北林业大学 A kind of camellia oleifera fruit flower identification positioning system based on machine vision
CN109592059B (en) * 2018-10-24 2020-09-11 武汉利能机电科技有限公司 Modularized multifunctional unmanned plane acquisition device
JP2020130070A (en) * 2019-02-21 2020-08-31 国立自然エネルギー株式会社 Fruit harvesting apparatus
CN110580725A (en) * 2019-09-12 2019-12-17 浙江大学滨海产业技术研究院 Box sorting method and system based on RGB-D camera
CN210868817U (en) * 2019-09-26 2020-06-30 农业农村部规划设计研究院 Intelligent picking machine
CN110663357A (en) * 2019-09-26 2020-01-10 农业农村部规划设计研究院 Intelligent picking machine
CN110754219A (en) * 2019-11-08 2020-02-07 仲恺农业工程学院 Vision-oriented bionic snake mouth picking mechanism, unmanned aerial vehicle harvesting device and harvesting method thereof
CN111587665B (en) * 2020-05-18 2024-05-03 华南农业大学 Four-degree-of-freedom multi-eye visual rotary-flying picking robot and picking method thereof
CN112136505B (en) * 2020-09-07 2021-11-26 华南农业大学 Fruit picking sequence planning method based on visual attention selection mechanism
CN113508686B (en) * 2021-05-27 2022-04-12 中国农业大学 String tomato picking end effector, robot and picking method thereof
CN113716057B (en) * 2021-09-03 2023-06-09 吉林大学重庆研究院 Unmanned aerial vehicle independently picks litchi device

Patent Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104899869A (en) * 2015-05-14 2015-09-09 浙江大学 Plane and barrier detection method based on RGB-D camera and attitude sensor
CN105783819A (en) * 2016-03-10 2016-07-20 江苏大学 RGB-D-based automatic transplanting seedling growing condition-operation effect composite detection method
CN108082477A (en) * 2017-12-15 2018-05-29 佛山市神风航空科技有限公司 A kind of unmanned plane for acrophyta sampling
WO2020134283A1 (en) * 2018-12-28 2020-07-02 江苏农林职业技术学院 Vegetable and fruit picker device and picking method
CN111418349A (en) * 2020-03-19 2020-07-17 南京赫曼机器人自动化有限公司 Intelligent fruit picking robot and method for realizing fruit picking
DE102020121554A1 (en) * 2020-08-17 2022-02-17 Thorsten Seckert Robotic system and mobile robot for picking stalked fruits of a plant
CN113424706A (en) * 2021-06-24 2021-09-24 华南农业大学 Unmanned aerial vehicle self-balancing picking device based on lever principle and control method thereof
CN216058333U (en) * 2021-11-08 2022-03-18 南京林业大学 Intelligent movement fruit picking robot
CN114916318A (en) * 2022-05-30 2022-08-19 华南农业大学 Automatic fruit harvesting device carried on unmanned aerial vehicle and control method thereof

Also Published As

Publication number Publication date
CN114916318A (en) 2022-08-19
CN114916318B (en) 2023-05-02

Similar Documents

Publication Publication Date Title
WO2023231408A1 (en) Automatic fruit harvesting apparatus mounted on unmanned aerial vehicle, and control method therefor
Hayashi et al. Robotic harvesting system for eggplants
CN101356877B (en) Cucumber picking robot system and picking method in greenhouse
CN109328973B (en) Intelligent rubber tapping system for rubber trees and control method of intelligent rubber tapping system
CN111418349B (en) Intelligent fruit picking robot and method for realizing fruit picking
Scarfe et al. Development of an autonomous kiwifruit picking robot
CN108633482A (en) A kind of fruit picking aircraft
CN103503637A (en) Intelligent identification picking robot and picking method
CN111758397B (en) Intelligent pepper picking device based on visual identification
CN216058333U (en) Intelligent movement fruit picking robot
CN109247153A (en) A kind of fertile mandarin orange branch pruning intelligent robot based on Internet of Things
CN113141871A (en) Daylily picking robot system based on Beidou navigation
CN115316129B (en) Self-adaptive bionic picking device based on binocular vision recognition and fruit stringing picking method
EP4201191A1 (en) Fruit picking robotic installation on platforms
CN110402683B (en) Intelligent cotton picker and control method thereof
Chatzimichali et al. Design of an advanced prototype robot for white asparagus harvesting
CN208016381U (en) A kind of fruit and vegetable picking machinery
Feng et al. Design and test of harvesting robot for table-top cultivated strawberry
CN211020054U (en) Fruit and vegetable picking device
CN208079864U (en) A kind of unmanned plane binocular visual positioning tomato picker
CN114995494A (en) Unmanned aerial vehicle automatic tip removing device based on visual positioning and control method thereof
CN116267197A (en) Greenhouse cucumber high-efficient intelligent harvesting device suitable for shelter from under environment
CN113228939B (en) Intelligent vertical corn header and corn combine
Zhang et al. Automatic fruit picking technology: a comprehensive review of research advances
CN108718707B (en) Intelligent wolfberry picking system

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22944709

Country of ref document: EP

Kind code of ref document: A1