CN114152254A

CN114152254A - Positioning method, device and system, operation method, system and electronic device

Info

Publication number: CN114152254A
Application number: CN202111531764.XA
Authority: CN
Inventors: 韩世豪; 萧延强; 邓飞杨
Original assignee: Guangzhou Xaircraft Technology Co Ltd
Current assignee: Guangzhou Xaircraft Technology Co Ltd
Priority date: 2021-12-14
Filing date: 2021-12-14
Publication date: 2022-03-08

Abstract

The embodiment of the application provides a positioning method, equipment and system, an operation method, system and electronic equipment. Each operation device sends the IMU data and the local image to the positioning device, and the positioning device can calculate the coordinate data of each operation device in the reference coordinate system according to the IMU data and the local image sent by each operation device. Therefore, a scheme of relatively high RTK for correlated positioning does not need to be installed in each operating device, IMU data and local images with relatively low cost are acquired, and the positions of the IMU data and the local images relative to each operating device are calculated by the positioning device and then are mapped into a reference coordinate system, so that the scheme of correlated positioning of the multi-operating device with low cost and high stability can be realized.

Description

Positioning method, device and system, operation method, system and electronic device

Technical Field

The present disclosure relates to the field of automatic control technologies, and in particular, to a positioning method, a positioning device, a positioning system, a positioning method, a positioning system, an operating system, and an electronic device.

Background

At present, a joint positioning scheme of multiple aircrafts or ground operation equipment generally needs to install a Real Time Kinematic (RTK) sensor in each aircraft or ground operation equipment for positioning, and meanwhile, multiple aircrafts need to constantly keep communication through a wireless communication module to mutually transmit information so as to determine mutual position information. Because the existing multi-aircraft combined positioning mode needs each aircraft to be provided with an RTK sensor, the price is high, and the situations of insufficient GPS star number and RTK positioning failure frequently occur under trees beside urban high buildings.

Therefore, the existing multi-device combined positioning scheme has the technical problems of high cost and poor positioning stability.

Disclosure of Invention

In order to solve the above technical problems, embodiments of the present application provide a positioning method, a positioning device, a positioning system, an operating method, an operating system, and an electronic device.

In a first aspect, an embodiment of the present application provides a positioning method, where the positioning device is in communication connection with at least one operating device; the method comprises the following steps:

receiving IMU data and local images acquired by each operating device;

determining the relative position of each operation device relative to the positioning device according to the IMU data and the local image acquired by each operation device;

and determining coordinate data of each operating device in a reference coordinate system where the positioning device is located according to the corresponding relative position of each operating device.

According to a specific embodiment of the application, a reference mark for assisting the positioning of the operation equipment is arranged on the surface of the positioning equipment;

the step of determining the relative position of each operating device with respect to the positioning device according to the IMU data and the local image acquired by each operating device includes:

calculating the relative position of each operation device relative to the reference mark according to the IMU data and the local image acquired by each operation device;

the determining the coordinate data of each operation device in the reference coordinate system where the positioning device is located according to the corresponding relative position of each operation device includes:

and calculating the coordinate data of each operating device in the reference coordinate system according to the relative position of each operating device relative to the reference identifier and the reference coordinate data of the reference identifier in the reference coordinate system.

According to an embodiment of the present application, the step of calculating the relative position of each operating device with respect to the reference identifier according to the IMU data and the local image collected by each operating device includes:

performing pre-integration processing on the IMU data;

extracting point features and line features in a local image of a current frame acquired by operation equipment, and performing optimization screening processing on the extracted point features and line features, wherein the point features and the line features in the local image comprise point features and line features of a reference mark;

triangularization processing is carried out on the point characteristics and the line characteristics after screening processing to obtain point-line depth data;

and calculating the relative position of the working equipment relative to the reference mark according to the dotted line depth data and the IMU data after pre-integration processing.

According to a specific embodiment of the present application, the step of extracting point features and line features in a local image of a current frame acquired by an operation device, and performing optimization screening processing on the extracted point features and line features includes:

tracking the local image of the current frame according to the feature detection result of the local image of the previous frame to acquire point features and line features in the local image of the current frame;

performing feature matching on the point features and the line features detected in the local image of the current frame, and deleting abnormal information, wherein the abnormal information comprises abnormal point features and/or abnormal line features;

and carrying out distortion removal processing on the residual point features and line features, and projecting the residual point features and line features to a normalized camera coordinate system established by taking the reference identifier as an origin.

According to a specific embodiment of the present application, before the step of extracting the point feature and the line feature in the local image of the current frame acquired by the operating device, the method further includes:

judging whether the local image of the current frame is a key frame or not according to the parallax between the local image of the previous frame and the local image of the current frame;

if the local image of the current frame is a key frame, extracting point features and line features in the local image of the current frame acquired by the operation equipment;

the step of calculating the relative position of the working equipment with respect to the reference mark from the point-line depth data and the pre-integrated IMU data includes:

constructing a joint optimization function according to a residual error term constructed by the IMU data after pre-integration processing and a reprojection error term constructed by the point-line depth data;

and carrying out nonlinear optimization processing on the combined optimization function, and calculating the position data of the operating equipment in the normalized camera coordinate system as the relative position of the aircraft relative to the reference mark.

According to a specific embodiment of the application, the positioning device is in communication connection and/or power supply connection with each operating device through a cable, and the end part of the cable is detachably connected with each operating device through an electric control connecting piece;

after the step of determining the coordinate data of each operation device in the reference coordinate system where the positioning device is located according to the corresponding relative position of each operation device, the method further includes:

controlling the winding and unwinding of the cable according to the operation area of each operation device;

and when the corresponding cable of the operating equipment is detected to be in a winding state, the corresponding cable is controlled to be separated from the operating equipment through the electric control connecting piece.

In a second aspect, an embodiment of the present application provides an operation method, which is applied to a positioning device, where the positioning device is in communication connection with at least one operation device; the method comprises the following steps:

acquiring a predetermined integral operation area;

dividing the whole operation area into sub-areas corresponding to the operation equipment according to the coordinate data of the operation equipment in the reference coordinate system where the positioning equipment is located, wherein the sub-areas of the operation equipment are not overlapped with each other; wherein the coordinate data is obtained based on the positioning method of any one of the first aspect;

sending the coordinate data of the sub-area of each operating device to the corresponding operating device; the sub-area is used for indicating the working range of the working equipment.

According to a specific embodiment of the present application, after the step of sending the coordinate data of the sub-area of each working device to the corresponding working device, the method further includes:

acquiring real-time operation progress of each operation device in a corresponding sub-area, wherein the real-time operation progress comprises any one of complete operation and partial operation;

when the completely finished real-time operation progress is obtained, generating a to-be-operated sub-region for an unoperated region in a sub-region corresponding to partially finished first-class operation equipment based on the real-time operation progress, or generating the to-be-operated sub-region based on the unoperated region which is not allocated to the operation equipment in the whole operation region;

and distributing the sub-area to be operated for the fully completed second type operation equipment for the real-time operation progress.

According to a specific embodiment of the present application, the step of generating a to-be-operated sub-region for a region which is not operated in a sub-region corresponding to a partially completed operation device based on a real-time operation progress includes:

determining a worked sub-area and a non-worked sub-area in sub-areas corresponding to partially finished working equipment, and determining a sub-area to be worked based on position information of the non-worked sub-area and second type of working equipment and/or the number of working objects of the non-worked sub-area;

alternatively, the first and second electrodes may be,

and determining an unfinished area which is not allocated to the working equipment in the whole working area, and determining a to-be-worked subarea based on the position information of the unfinished area and the second type of working equipment and/or the number of working objects in the unfinished area.

According to a specific embodiment of the present application, the step of dividing the entire working area into sub-areas corresponding to the working devices according to the coordinate data of each working device in the reference coordinate system where the positioning device is located further includes:

the entire work area is divided according to the number of work devices in the entire work area.

In a third aspect, an embodiment of the present application provides a positioning device, where the positioning device includes a device body, a communication unit, and a processor, where the communication unit is in communication connection with at least one operating device;

the communication unit is used for receiving IMU data and local images acquired by each operating device;

the processor is used for determining the relative position of each operation device relative to the positioning device according to the IMU data and the local image acquired by each operation device, and determining the coordinate data of each operation device in the reference coordinate system where the positioning device is located according to the corresponding relative position of each operation device.

According to a specific embodiment of the present application, the positioning device is configured to be electrically connected to each working device through a cable, and the cable includes an integrated charging cable and a communication cable.

According to a specific embodiment of this application, the positioning device pass through automatically controlled connecting piece with the connection can be dismantled to the tip of cable, the control end of automatically controlled connecting piece with processor electricity is connected.

According to a specific embodiment of the present application, the apparatus body includes a moving body and a driving wheel, the moving body being disposed on the driving wheel;

the mobile vehicle body is marked with a plurality of reference marks, each reference mark is provided with a cable interface, each cable interface is used for being connected with one end of a cable, and the other end of the cable is connected with an operation device.

In a fourth aspect, an embodiment of the present application provides a positioning system, which includes a positioning device and at least one operating device, where the positioning device is in communication connection with the at least one operating device; wherein the content of the first and second substances,

the operation equipment is used for sending the acquired IMU data and the local image to the positioning equipment;

the positioning equipment is used for determining the relative position of each operation equipment relative to the positioning equipment according to the IMU data and the local image collected by each operation equipment, and determining the coordinate data of each operation equipment in the reference coordinate system where the positioning equipment is located according to the corresponding relative position of each operation equipment.

In a fifth aspect, an embodiment of the present application provides a multi-machine cooperation system, which includes a positioning device and at least one operating device, where the positioning device is in communication connection with the at least one operating device;

the positioning equipment is used for acquiring a predetermined overall operation area, and dividing the overall operation area into sub-areas corresponding to the operation equipment according to the coordinate data of the operation equipment in a reference coordinate system where the positioning equipment is located, wherein the sub-areas of the operation equipment are not overlapped; wherein the coordinate data is obtained based on the positioning method of any one of the first aspect;

the operation equipment is used for receiving the coordinate data of the sub-area of each operation equipment sent by the positioning equipment; the sub-area is used for indicating the working range of the working equipment.

In a sixth aspect, an embodiment of the present application provides an electronic device, including a memory and a processor, where the memory is used to store a computer program, and the computer program executes, when the processor runs, the positioning method in any one of the first aspect or the working method in any one of the second aspect.

In a seventh aspect, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program, and the computer program, when executed on a processor, executes the positioning method according to any one of the first aspect or the working method according to any one of the second aspect.

According to the positioning method, the positioning device, the positioning system, the operation method, the operation system and the electronic device, the positioning device is connected with at least one operation device. Each operation device sends the IMU data and the local image to the positioning device, and the positioning device can calculate the coordinate data of each operation device in the reference coordinate system according to the IMU data and the local image sent by each operation device. Therefore, a scheme of relatively high RTK for correlated positioning does not need to be installed in each operating device, IMU data and local images with relatively low cost are acquired, and the positions of the IMU data and the local images relative to each operating device are calculated by the positioning device and then are mapped into a reference coordinate system, so that the scheme of correlated positioning of the multi-operating device with low cost and high stability can be realized.

Drawings

In order to more clearly explain the technical solutions of the present application, the drawings needed to be used in the embodiments are briefly introduced below, and it should be understood that the following drawings only illustrate some embodiments of the present application and therefore should not be considered as limiting the scope of protection of the present application. Like components are numbered similarly in the various figures.

Fig. 1 is a schematic flowchart illustrating a positioning method applied to a positioning device according to an embodiment of the present application;

fig. 2 is a schematic connection diagram of a positioning device and an aircraft according to a positioning method provided by an embodiment of the present application;

fig. 3 is a process diagram illustrating front-end calculation steps involved in a positioning method provided by an embodiment of the present application;

fig. 4 is a process diagram illustrating a back-end computing step involved in the positioning method provided by the embodiment of the present application;

fig. 5 is a flowchart illustrating an operation method according to an embodiment of the present application.

Detailed Description

The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments.

The components of the embodiments of the present application, generally described and illustrated in the figures herein, can be arranged and designed in a wide variety of different configurations. Thus, the following detailed description of the embodiments of the present application, presented in the accompanying drawings, is not intended to limit the scope of the claimed application, but is merely representative of selected embodiments of the application. All other embodiments, which can be derived by a person skilled in the art from the embodiments of the present application without making any creative effort, shall fall within the protection scope of the present application.

Hereinafter, the terms "including", "having", and their derivatives, which may be used in various embodiments of the present application, are intended to indicate only specific features, numbers, steps, operations, elements, components, or combinations of the foregoing, and should not be construed as first excluding the existence of, or adding to, one or more other features, numbers, steps, operations, elements, components, or combinations of the foregoing.

Furthermore, the terms "first," "second," "third," and the like are used solely to distinguish one from another and are not to be construed as indicating or implying relative importance.

Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the various embodiments of the present application belong. The terms (such as those defined in commonly used dictionaries) should be interpreted as having a meaning that is consistent with their contextual meaning in the relevant art and will not be interpreted in an idealized or overly formal sense unless expressly so defined herein in various embodiments.

Example 1

Referring to fig. 1, a schematic flowchart of a positioning method provided in an embodiment of the present application is shown, where the positioning method is applied to a positioning device 201 shown in fig. 2, and the positioning device 201 is connected to at least one operating device 202. As shown in fig. 1 and 2, the method mainly includes the following steps:

s101, receiving IMU data and local images acquired by each operating device;

as shown in fig. 2, the positioning device 201 is connected to each operating device 202, the positioning device 201 interacts with each operating device 202, and the operating devices 202 do not need to be equipped with a communication hardware base, and rely on the positioning device 201 to realize interaction.

In this embodiment, the positioning apparatus may include an apparatus body, a positioning unit, and a communication unit, where the apparatus body is a main carrying device of the positioning apparatus, for example, a central control cart shown in fig. 2. The Positioning unit is used for collecting Positioning data of the Positioning device, and the Positioning unit can collect satellite Positioning data, such as Global Positioning data (GPS for short) or Beidou Positioning data, for the satellite Positioning device. The communication unit may include a wireless communication unit and a wired communication unit, the wireless communication unit may include bluetooth, WiFi, GPRS, etc., and the wired communication unit may include a cable communication, such as the cable 203 shown in fig. 2 is the wired communication unit of the positioning device 201. Of course, the positioning apparatus may further include other devices such as a driving motor, and is not limited.

The working device 202 may be a device for implementing joint positioning by the positioning device 201, including but not limited to an aircraft or a ground working device, and the positioning method provided by the present embodiment is particularly suitable for an aerial joint positioning scheme of an aircraft. Wherein, the aircraft can include flight control unit, power supply, image acquisition unit, communication unit etc. and image acquisition unit can be binocular camera. In this embodiment, the aircraft is equipped with a binocular camera and an Inertial Measurement Unit (IMU for short) for respectively acquiring binocular images and IMU data of the aircraft, and the binocular camera may be a binocular fisheye camera or a camera combination with a relatively fixed viewing angle. The communication unit of the aircraft is connected with the communication unit of the positioning device, and communication between the aircraft and the positioning device is realized, for example, wired communication interaction between the aircraft and the positioning device is realized through a cable shown in fig. 2.

In the positioning method provided by this embodiment, the inertial measurement unit of the working equipment may be used to acquire IMU data of the working equipment, and the binocular camera of the working equipment may acquire a local image, where the local image may be an image of a local area in an environment facing the binocular camera of the working equipment, and the local area may be a local area in the working area, a local area in the sky, or a local area on the ground, which is not limited in this embodiment. The positioning equipment receives the IMU data and the local images acquired by the operation equipment, the actual coordinate position of the operation equipment can be calculated through the IMU data and the local images of the operation equipment, joint positioning among the operation equipment is not needed, and the hardware structure is simplified.

It should be noted that the IMU data may be data acquired by using the IMU, or may also be data acquired by using a gyroscope and an accelerometer which are independently arranged from each other, and it is understood that data acquired by an integrated unit, such as the IMU, may not be the same data as the IMU data in the present embodiment, and data of the same nature may be referred to as IMU data.

The application scenarios of the positioning method provided by the embodiment can be various, such as formation and performance of operation equipment, pesticide spraying in farmland, fruit picking in fruit trees and the like. Wherein, to the fruit tree scene of plucking, the positioning device can remove at the road between two adjacent fruit trees, and the operation equipment that the positioning device is connected can fly in positioning device's both sides, picks the fruit on the local both sides fruit tree of road respectively, and many operation equipment are in coordination with the operation, have improved unmanned aerial vehicle and have plucked fruit efficiency.

And S102, determining the relative position of each operation device relative to the positioning device according to the IMU data and the local image collected by each operation device.

And S103, determining coordinate data of each operating device in a reference coordinate system where the positioning device is located according to the corresponding relative position of each operating device.

A three-dimensional coordinate system where the positioning equipment is located is established in advance, and the three-dimensional coordinate system can use the positioning equipment as an origin point and can also use other fixed operation sites as the origin point without limitation.

The positioning equipment can calculate the coordinates of the operation equipment in a reference coordinate system after receiving IMU data and local images collected by the operation equipment, and in the process, the relative position of the operation equipment relative to the positioning equipment is calculated, and then the coordinate data of the operation equipment in the reference coordinate system where the positioning equipment is located is determined according to the corresponding relative position of the operation equipment.

In specific implementation, the positioning device surface is provided with a reference identifier for assisting in positioning the operation device, and the step of determining the relative position of each operation device with respect to the positioning device according to the IMU data and the local image acquired by each operation device may include:

correspondingly, the step of determining the coordinate data of each operation device in the reference coordinate system where the positioning device is located according to the corresponding relative position of each operation device includes:

The reference mark may be understood as a position point where the work device rests on the positioning device before the work, the position point may be marked on the surface of the positioning device, and the mark may be a digital mark or a pattern mark, but is not limited thereto. It should be noted that if there are a plurality of working devices communicating with the positioning device, the number of identifiers may be consistent with the number of working devices, and it can be understood that: each mark corresponds to each operation equipment one to one, so that accurate positioning of each operation equipment is guaranteed, and positioning confusion among the operation equipment is avoided. For convenience of understanding, the following description will be given by taking fig. 2 as an example, and assuming that the multi-machine cooperative system includes one positioning device and four working devices, 4 positions 1, 2, 3 and 4 may be marked on the positioning device as the reference identifier. Before starting the operation, or in an initial state such as suspension of replenishment or charging, the operation device starts to take off from the reference mark. For example, work device a takes off from reference mark 1, work device B takes off from reference mark 2, work device C takes off from reference mark 3, and work device D takes off from reference mark 4. Of course, this is merely an exemplary illustration, and the specific number of the working devices and the setting rule of the reference identifier may be specifically set according to the actual job, and are not limited.

The work equipment takes off from the corresponding reference marker, the flight trajectory may be as shown by the dashed line in fig. 2, and on this continuous flight trajectory, the work equipment continuously acquires the local images. In the initial takeoff stage, the local image acquired by the operation equipment may include a reference identifier, the real-time position of the initial takeoff stage moment can be directly calculated according to the IMU data and the local image, and the relative position of the operation equipment and the reference identifier at each flight moment can be iteratively deduced according to the position continuity between adjacent moments.

The position of the benchmark marker on the positioning equipment is fixed relative to the positioning equipment, and the position of each working equipment can be mapped into a reference coordinate system according to the relative position of the working equipment relative to the benchmark marker, so that the positions of the working equipment can be displayed in a centralized mode in the same coordinate system.

The positioning method provided by the embodiment of the application does not need to install a scheme of directly performing associated positioning between multiple operation devices by using a high-cost RTK in each operation device, and can realize the associated control scheme of the multiple operation devices with low cost and high stability by acquiring IMU data and local images with low cost, calculating the positions of the multiple operation devices by using the positioning device and then mapping the positions of the multiple operation devices into a reference coordinate system.

On the basis of the foregoing embodiment, the step of calculating, according to the IMU data and the local image collected by each of the operating devices, a relative position reference identifier of each of the operating devices with respect to the reference identifier on the positioning device in S102 may include:

performing pre-integration processing on the IMU data;

extracting point features and line features from a local image of a current frame acquired by operation equipment, and performing optimization screening processing on the extracted point features and line features, wherein the point features and the line features in the local image both comprise point features and line features of a reference mark;

The embodiment adds a specific scheme of determining the relative position by utilizing IMU data and a local image. In the embodiment, the scheme of using the visual local image and the inertial measurement pose is divided into the front end and the rear end, wherein the front end is used for processing IMU (inertial measurement unit) and binocular image data, and the rear end is used for carrying out close coupling optimization estimation on the pose of the operation equipment on the basis of point-line characteristics.

performing feature matching on point feature points and line features detected in a local image of a current frame, and deleting abnormal information, wherein the abnormal information comprises abnormal point features and/or abnormal line features;

In one embodiment, as shown in fig. 3, the front end performs the following working procedure, the front end independently detects point features and line features in two threads, tracks the feature points that have been observed in the previous frame by using the KLT optical flow algorithm, eliminates the feature points that have failed in tracking the previous frame and the points that exceed the boundary of the current frame image, simultaneously eliminates abnormal point features or line features by using the base matrix F by using the method of RANSAC, and then performs distortion correction on the feature points detected in the current frame and projects the feature points to the normalized camera coordinate system of the current frame.

And simultaneously, detecting line segments in the current frame in another thread by using an LSD detector, extracting an LBD descriptor of each line segment, matching the LBD descriptor with the line segments in the previous frame, screening two end points of the line segments by using a basic matrix F by using a RANSAC method, removing abnormal line segments, and then performing distortion-removing projection on the two end points of the line segments to a normalized camera coordinate system. And meanwhile, pre-integrating angular velocity data and acceleration data acquired by the IMU sensor to obtain the relative motion state between frames.

if the local image of the current frame is a key frame, extracting point features and line features in the local image of the current frame acquired by the operation equipment; and the number of the first and second groups,

and carrying out nonlinear optimization processing on the joint optimization function, and calculating the position data of the operating equipment in the normalized camera coordinate system as the relative position of the operating equipment relative to the reference mark.

As shown in fig. 4, the backend workflow is as follows:

after the system is initialized, whether a current frame is a key frame or not is judged according to the parallax of the previous frame and the next frame, if the parallax exceeds a certain threshold value, the current frame is judged to be not the key frame, if not, the current frame is discarded, and if the parallax is within a certain value range, the current frame is judged to be the key frame. If the current frame is the key frame, the feature points of the current frame are triangulated to obtain the initial depths of the feature points, and meanwhile, a Prock matrix is constructed for the line features of the current frame, so that Prock coordinates are obtained. And finally, further screening the optimized point line characteristics, and rejecting abnormal points to obtain all the point line characteristics covered by the operating equipment. And calculating the positions of all the point-line characteristics of the current working equipment in a normalized coordinate system established by taking the reference mark as an origin, wherein the positions are the relative positions of the working equipment relative to the reference mark.

On the basis of the embodiment, according to another specific implementation manner of the application, the positioning device is in communication connection and/or power supply connection with each operating device through a cable, and the end part of the cable is detachably connected with each operating device through an electric control connecting piece;

The present embodiment adds a description of the cable connection scheme. In order to ensure uninterrupted operation in consideration of the high power demand of the operation equipment in the continuous operation scene, a charging cable is arranged, and the positioning equipment is continuously connected with the operation equipment through the charging cable. After the operation equipment is detected to be in a state to be charged or a charging request of the operation equipment is received, the charging cable can be controlled to charge the operation equipment.

In addition, the cable can comprise a communication cable, and data can be directly transmitted and received through the communication cable, so that the cost of the communication module can be saved, the communication stability can be improved, and the environmental influence can be avoided.

In addition, in consideration of the winding problem possibly existing in the operation scene or in order to facilitate the dismounting and the storage, the scheme of self-separation of the winding is added in the embodiment. When the positioning equipment controls the cable to be wound and unwound, whether the positioning equipment is in a winding state can be detected through the winding and unwinding state of the cable or the image analysis and other modes. For example, when the resistance is large in the process of winding up the automatic reel, the operating equipment can be considered to be in the winding state, or the continuity of the characteristic points corresponding to the cable is tracked through image processing, and if the characteristic points are in the discontinuous state, the operating equipment is considered to be in the winding state. When the operating equipment is monitored to be in a winding state, the end part of the cable can be controlled to be directly separated from the operating equipment, so that the cable can be firstly retracted to the positioning equipment, and then the operating equipment is controlled to be independently retracted, and the damage of the cable winding to the dragging of the operating equipment is avoided. The electronic control connecting piece used in the embodiment can be an automatic clamping component or an occlusion component, the processor controls the operation equipment to automatically clamp or occlude the end part of the cable, and when the winding state is monitored, the operation equipment cuts off the cable or loosens the clamping component or the occlusion component to realize automatic separation.

Example 2

Referring to fig. 5, a flowchart of an operation method provided by an embodiment of the present application is shown, where the provided operation method is applied to the positioning device 201 shown in fig. 2, and the positioning device 201 is communicatively connected to at least one operation device 202. As shown in fig. 5, the operation method mainly includes the following steps:

step S501, obtaining a predetermined integral operation area;

after coordinate data of each operation device in a reference coordinate system is obtained, a global map can be obtained in advance through a positioning device, the global map can be combined to accurately map the surrounding environment, moving areas such as roads in the surrounding environment and the whole area to be operated at the top of a fruit tree are determined, and sub-areas for respective operation are divided for each flight area.

Step S502, dividing the whole operation area into sub-areas corresponding to the operation equipment according to the coordinate data of the operation equipment in the reference coordinate system, wherein the sub-areas of the operation equipment are not overlapped;

step S503, sending the coordinate data of the sub-area of each operating device to the corresponding operating device; the sub-area is used for indicating the working range of the working equipment.

On the basis of the above embodiment, after the coordinate data of each operating device in the reference coordinate system is acquired, all the operating areas to be operated can be divided according to the coordinate data of each operating device, so as to determine the operating area corresponding to each operating device.

Specifically, a plurality of sub-areas are divided from the entire work area in the vicinity as work areas corresponding to the respective work apparatuses, based on coordinate data in a reference coordinate system in which the respective work apparatuses are located. In order to avoid possible collision, the sub-regions are limited to be not overlapped when being divided.

In specific implementation, the step of dividing the entire operation area into sub-areas corresponding to the operation devices according to the coordinate data of each operation device in the reference coordinate system where the positioning device is located further includes:

Of course, in order to avoid the uneven sub-area division from affecting the overall work progress, when the sub-area division is performed, the overall work area may be divided according to the number of the work devices in the overall work area. For example, if the number of the working devices in the entire working area is N, the entire working area may be equally divided or approximately equally divided into N sub-areas, so that a plurality of working devices may simultaneously turn on and complete the working task. Of course, the division may be performed unequally according to the work efficiency of different pieces of work equipment, that is, a larger sub-area may be divided for a work equipment with higher work efficiency, and a smaller sub-area may be divided for a work equipment with relatively lower work efficiency, and the like, without limitation.

And then, sending the coordinate data of the sub-region of each operating device to the corresponding operating device so as to enable each operating device to execute corresponding operation in the operation range divided by the coordinate data of the sub-region.

As shown in fig. 2, the positioning device is connected with 4 working devices ABCD, and at this time, the map can be divided into 4 non-overlapping sub-areas Z1-Z4, and the sub-areas are not overlapped with each other, so that the collision of the working devices can be effectively avoided. After the 4 sub-areas are determined, the closest sub-area is matched as a work area for each work apparatus on the principle of proximity. All the operation devices work in respective operation areas, so that collision of the operation devices is avoided, and work division cooperation improves operation efficiency.

On the basis of the above embodiment, after the step of sending the coordinate data of the sub-region of each work apparatus to the corresponding work apparatus, the method may further include:

In this embodiment, after the sub-regions are divided, the sub-regions are secondarily allocated according to the situation that the job completion schedules of different job devices are different, so that the overall job completion efficiency is improved. Specifically, the real-time job progress of each job device is monitored, and particularly, whether the job implementation progress of a certain job device is completely completed or not is monitored, that is, the job device preferentially completes job tasks in sub-regions allocated to the job device in advance. At this time, the real-time job schedules of the other job devices are partially completed, and the partial job devices are defined as the first type job devices.

Then, to fully utilize the efficiency of each job device, the sub-area that has not been operated can be allocated again to the job device that is completed with priority at this time. The partial sub-area which is not operated can be an area which is not operated in the first type of operation equipment or an area which is not allocated in the whole operation area, and the partial area which is not operated is defined as an area to be operated and is allocated to the operation equipment which is preferentially completed, so that the completion progress of the whole operation area is accelerated.

In the embodiment, in consideration of imbalance of work area division or difference of work efficiency of the work equipment, fruit picking operation of a part of the work equipment in the work area may be completed, fruit picking operation of other work equipment is more remained, for example, no fruit tree can be operated in the work area Z1 corresponding to the work equipment, more fruit trees are remained in the work area Z2 corresponding to the work equipment B, in order to improve the whole work area, the work area Z2 may be divided into Z21 and Z22, Z21 may be divided into the work area Z1 of the work equipment a, and the work area Z2 of the work equipment B may be updated to Z22. Promote whole operating efficiency like this to the realization is according to the work area of each unmanned aerial vehicle of actual task dynamic adjustment. Certainly, when performing redistribution, the redistribution may be performed only for the case that the operation progress is completely completed and partially completed, and certainly, the redistribution that is not completely completed may also be divided into half and less than half, or the case of completing a large amount and completing a small amount, and only the incomplete area of the operation equipment with a relatively small completion amount is redistributed to the operation equipment that has been completely completed or partially completed to perform the operation, so as to further improve the operation efficiency.

Further, the step of generating a to-be-operated sub-region for a region which is not operated in a sub-region corresponding to the partially completed operation device based on the real-time operation progress may include:

alternatively, the first and second electrodes may be,

The embodiment further limits the partition scheme of the subareas to be operated which need to be redistributed. In one case, the sub-area to be operated is determined based on the position information, and an area in the sub-area to be operated which is closer to the second type of operation device and is not operated is used as the sub-area to be operated. As shown in the above example, the work area Z2 corresponding to the work equipment B has many fruit trees left, and in order to increase the overall work area, the work area Z2 may be divided into the work areas Z21 and Z22, and the work area Z21 may be divided into the work area Z1 of the work equipment a having a relatively short distance.

Another division situation may be that the sub-regions to be operated are determined based on the number of the operation objects in the sub-region to be operated, that is, the sub-regions to be operated are reasonably distributed according to the number of the operation objects, so that the operation efficiency and the operation progress of each operation device can be guaranteed to be basically consistent. For example, the number of the second operation devices which are completely completed and the range of all the sub-areas which are not operated are counted, so that the current operation condition is redistributed evenly and reasonably, and the overall operation progress of all the operation devices is improved as much as possible.

The two conditions can be combined, namely the overall efficiency can be improved, unnecessary flying around can be avoided, and the operation progress can be improved according to the principle of comprehensive equal division and nearby distribution.

Example 3

The embodiment of the present application provides a schematic structural diagram of a positioning apparatus, and the positioning apparatus may be applied to the positioning apparatus 201 shown in fig. 2. The positioning device can comprise a device body, a communication unit and a processor, wherein the communication unit is in communication connection with at least one working device;

Example 4

In addition, with continued reference to fig. 2, an embodiment of the present application further provides a positioning system, which includes a positioning device 201 and at least one working device 202, where the positioning device 201 is communicatively connected to the at least one working device 202; wherein the content of the first and second substances,

the operation device 202 is configured to send the acquired IMU data and the local image to the positioning device 201;

the positioning device 201 is configured to determine a relative position of each operating device with respect to the positioning device 201 according to the IMU data and the local image acquired by each operating device, and determine coordinate data of each operating device 202 in a reference coordinate system where the positioning device 201 is located according to the corresponding relative position of each operating device 202.

In addition, with continued reference to fig. 2, the present embodiment further provides a multi-level collaboration system, which includes a central control device 201 and at least one operating device 202, where the positioning device 201 is connected to the at least one operating device 202 in a communication manner;

the positioning device 201 is configured to obtain a predetermined overall operation area, and divide the overall operation area into sub-areas corresponding to the operation devices 202 according to coordinate data of the operation devices 202 in a reference coordinate system where the positioning device 201 is located, where the sub-areas of the operation devices 202 do not overlap with each other; the coordinate data is obtained based on the positioning method provided by the embodiment;

the working equipment 202 is configured to receive coordinate data of a sub-area of each working equipment 202 sent by the positioning equipment 201; the sub-area is used to indicate the working range of the working device 202.

In addition, an electronic device is further provided, and includes a memory and a processor, where the memory is used to store a computer program, and the computer program executes the positioning method or the operation method provided in the foregoing embodiments when the processor runs.

In addition, the present application also provides a computer-readable storage medium, which stores a computer program, and when the computer program runs on a processor, the computer program executes the positioning method or the operation method described in the above embodiments.

According to the positioning method, the positioning device, the positioning system, the operation method, the operation system and the electronic device, the positioning device is connected with at least one operation device. Each operation device sends the IMU data and the local image to the positioning device, and the positioning device can calculate the coordinate data of each operation device in the reference coordinate system according to the IMU data and the local image sent by each operation device. Therefore, a scheme of relatively high RTK for correlated positioning does not need to be installed in each operating device, IMU data and local images with relatively low cost are acquired, and the positions of the IMU data and the local images relative to each operating device are calculated by the positioning device and then are mapped into a reference coordinate system, so that the scheme of correlated positioning of the multi-operating device with low cost and high stability can be realized. The specific implementation process of the positioning device, the electronic device, and the computer-readable storage medium can be referred to as the specific implementation process of the positioning method shown in fig. 1, and details are not repeated here.

The above description is only for the specific embodiments of the present application, but the scope of the present application is not limited thereto, and any person skilled in the art can easily conceive of the changes or substitutions within the technical scope of the present application, and shall be covered by the scope of the present application.

Claims

1. The positioning method is applied to positioning equipment, and the positioning equipment is in communication connection with at least one piece of operation equipment; the method comprises the following steps:

receiving IMU data and local images acquired by each operating device;

2. The method according to claim 1, wherein the positioning device surface is provided with a reference mark for assisting the positioning of the working device;

the step of determining the coordinate data of each operation device in the reference coordinate system where the positioning device is located according to the corresponding relative position of each operation device includes:

3. The method of claim 2, wherein the step of calculating the relative position of each work implement with respect to the fiducial markers from the IMU data and the local images captured by each work implement comprises:

performing pre-integration processing on the IMU data;

4. The method according to claim 3, wherein the step of extracting point features and line features in the local image of the current frame acquired by the operating device and performing the optimized screening process on the extracted point features and line features comprises:

5. The method according to claim 4, wherein the step of extracting the point feature and the line feature in the partial image of the current frame acquired by the working device is preceded by the method further comprising:

6. The method of claim 1, wherein the positioning device is communicatively and/or electrically connected to each work device via a cable, an end of the cable being removably connected to each work device via an electrically controlled connector;

7. The operation method is applied to positioning equipment, and the positioning equipment is in communication connection with at least one operation equipment; the method comprises the following steps:

acquiring a predetermined integral operation area;

dividing the whole operation area into sub-areas corresponding to the operation equipment according to the coordinate data of the operation equipment in the reference coordinate system where the positioning equipment is located, wherein the sub-areas of the operation equipment are not overlapped with each other; wherein the coordinate data are obtained based on the positioning method according to any one of claims 1 to 6;

8. The method of claim 7, wherein after the step of sending the coordinate data of the sub-region of each work device to the corresponding work device, the method further comprises:

9. The method according to claim 8, wherein the step of generating a to-be-operated subarea for an unoperated area in a subarea corresponding to the partially completed operation equipment based on the real-time operation progress comprises:

alternatively, the first and second electrodes may be,

10. The method according to claim 7, wherein the step of dividing the entire working area into sub-areas corresponding to the working devices according to the coordinate data of the working devices in the reference coordinate system of the positioning device further comprises:

11. The positioning device is characterized by comprising a device body, a communication unit and a processor, wherein the communication unit is in communication connection with at least one operating device;

12. The positioning apparatus according to claim 11, wherein the positioning apparatus is configured to be electrically connected to each of the working apparatuses by a cable, and the cable includes an integrated charging cable and a communication cable.

13. The positioning apparatus of claim 12, wherein the positioning apparatus is removably connected to the end of the cable by an electrically controlled connector, a control end of the electrically controlled connector being electrically connected to the processor.

14. The positioning apparatus according to any one of claims 11 to 13, wherein the apparatus body includes a moving vehicle body and a drive wheel, the moving vehicle body being provided on the drive wheel; the mobile vehicle body is marked with a plurality of reference marks, each reference mark is provided with a cable interface, each cable interface is used for being connected with one end of a cable, and the other end of the cable is connected with an operation device.

15. The positioning system is characterized by comprising a positioning device and at least one working device, wherein the positioning device is in communication connection with the at least one working device; wherein the content of the first and second substances,

16. The multi-machine cooperation system is characterized by comprising a positioning device and at least one operating device, wherein the positioning device is in communication connection with the at least one operating device;

the positioning equipment is used for acquiring a predetermined overall operation area, and dividing the overall operation area into sub-areas corresponding to the operation equipment according to the coordinate data of the operation equipment in a reference coordinate system where the positioning equipment is located, wherein the sub-areas of the operation equipment are not overlapped; wherein the coordinate data are obtained based on the positioning method according to any one of claims 1 to 6;

17. An electronic device, comprising a memory and a processor, wherein the memory is used for storing a computer program, and the computer program is used for executing the positioning method of any one of claims 1 to 6 or the working method of any one of claims 7 to 10 when the processor runs.

18. A computer-readable storage medium, characterized in that it stores a computer program which, when run on a processor, performs the positioning method of any one of claims 1 to 6, or the working method of any one of claims 7 to 10.