WO2023050545A1 - 一种基于机器视觉的室外自动工作控制系统、方法及设备 - Google Patents
一种基于机器视觉的室外自动工作控制系统、方法及设备 Download PDFInfo
- Publication number
- WO2023050545A1 WO2023050545A1 PCT/CN2021/132653 CN2021132653W WO2023050545A1 WO 2023050545 A1 WO2023050545 A1 WO 2023050545A1 CN 2021132653 W CN2021132653 W CN 2021132653W WO 2023050545 A1 WO2023050545 A1 WO 2023050545A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- outdoor automatic
- boundary
- automatic tool
- outdoor
- machine vision
- Prior art date
Links
- 238000000034 method Methods 0.000 title claims abstract description 49
- 238000013528 artificial neural network Methods 0.000 claims abstract description 13
- 239000013598 vector Substances 0.000 claims description 39
- 230000008569 process Effects 0.000 claims description 20
- 238000006073 displacement reaction Methods 0.000 claims description 14
- 230000009466 transformation Effects 0.000 claims description 11
- 238000005457 optimization Methods 0.000 claims description 7
- 230000010354 integration Effects 0.000 claims description 4
- 230000004927 fusion Effects 0.000 claims description 3
- 238000005516 engineering process Methods 0.000 abstract description 8
- 238000010586 diagram Methods 0.000 description 10
- 238000013507 mapping Methods 0.000 description 8
- 238000013135 deep learning Methods 0.000 description 6
- 239000000203 mixture Substances 0.000 description 6
- 238000012545 processing Methods 0.000 description 6
- 230000000007 visual effect Effects 0.000 description 6
- 238000011065 in-situ storage Methods 0.000 description 5
- 238000013473 artificial intelligence Methods 0.000 description 4
- 238000004364 calculation method Methods 0.000 description 4
- 230000001186 cumulative effect Effects 0.000 description 4
- 230000003993 interaction Effects 0.000 description 4
- 230000001133 acceleration Effects 0.000 description 3
- 230000008859 change Effects 0.000 description 3
- 238000009333 weeding Methods 0.000 description 3
- 238000012790 confirmation Methods 0.000 description 2
- 238000012423 maintenance Methods 0.000 description 2
- 230000003068 static effect Effects 0.000 description 2
- 102100034112 Alkyldihydroxyacetonephosphate synthase, peroxisomal Human genes 0.000 description 1
- 244000025254 Cannabis sativa Species 0.000 description 1
- 101000799143 Homo sapiens Alkyldihydroxyacetonephosphate synthase, peroxisomal Proteins 0.000 description 1
- 238000004140 cleaning Methods 0.000 description 1
- 239000004020 conductor Substances 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 239000000284 extract Substances 0.000 description 1
- 230000004720 fertilization Effects 0.000 description 1
- 230000002452 interceptive effect Effects 0.000 description 1
- 238000004540 process dynamic Methods 0.000 description 1
- 238000011897 real-time detection Methods 0.000 description 1
- 238000009331 sowing Methods 0.000 description 1
- 239000013589 supplement Substances 0.000 description 1
- 230000001502 supplementing effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
- G05D1/02—Control of position or course in two dimensions
- G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
- G05D1/0231—Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means
- G05D1/0246—Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means using a video camera in combination with image processing means
- G05D1/0253—Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means using a video camera in combination with image processing means extracting relative motion information from a plurality of images taken successively, e.g. visual odometry, optical flow
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
- G05D1/02—Control of position or course in two dimensions
- G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
- G05D1/0276—Control of position or course in two dimensions specially adapted to land vehicles using signals provided by a source external to the vehicle
Definitions
- the invention relates to equipment automation technology, in particular to the automatic work control technology of outdoor working equipment.
- Outdoor automatic tools are more and more used in people's daily life based on the convenience of use.
- some outdoor automatic tools that can automatically complete auxiliary tasks in a specific work area are very popular, such as garden weeding robots.
- the first problem to be solved is how to accurately identify the corresponding working area boundary and determine the working area; on this basis, how to plan the working path in the working area to maximize the The range covers the entire work area while also being able to cross work area boundaries.
- the current common outdoor automatic tools such as garden weeding robots, often use a fixed-frequency alternating current or a fixed-frequency electric pulse to input a fixed-frequency alternating current or a fixed-frequency electric pulse on the wire, and use the changing current to generate a magnetic field on the conductor.
- the electromagnetic boundary on this basis, the inductance or Hall element is used to sense the strength of the magnetic field signal on the ground and the plane, and the line patrol movement and tracking are carried out.
- it can only do disordered ping-pong motion within the range of the electromagnetic boundary frame.
- the sensor has a single structure and is easily interfered by the alternating current of the low mains power. Once a certain boundary is powered off and damaged, the machine will easily leave the working area and cross the boundary, causing potential safety hazards.
- the object of the present invention is to provide a kind of outdoor automatic work control system based on machine vision, realize the control accuracy and reliability of the automatic work of improving outdoor automatic tools based on machine vision;
- the present invention further provides an outdoor automatic work control method based on machine vision, and automatic work equipment capable of running the outdoor automatic work control method.
- the outdoor automatic work control system based on machine vision includes a machine vision image sensor, an intelligent control unit, a positioning module, and a motion sensing module,
- the machine vision image sensor acquires image information around the outdoor automatic tool in real time
- the positioning module obtains the spatial position information of the outdoor automatic tool
- the motion sensing module acquires motion state information of the outdoor automatic tool
- the intelligent control unit is based on the image information around the outdoor automatic tool acquired by the machine vision image sensor, the positioning module acquires the spatial position information of the outdoor automatic tool, and the motion sensor module acquires the motion state information of the outdoor automatic tool. After the boundary runs for a week, a characteristic coordinate map of the boundary of the working area is established;
- the intelligent control unit calculates the positional relationship of the outdoor automatic tool relative to the working area boundary in real time based on the image information around the outdoor automatic tool, the spatial position information, the motion state information, and the established characteristic coordinate map of the working area boundary, and automatically When the tool reaches the boundary of the working area, it forms an automatic work control mode for the outdoor automatic tool to return to the working area.
- the neural network is used to identify and segment the image, and judge whether the outdoor automatic tool is close to the specified working area boundary; if the boundary cannot be detected, the outdoor automatic tool is controlled to continue driving according to the current state , until the boundary is identified; at that time, the absolute coordinate points obtained by the positioning module in the outdoor automatic tool are synchronously collected to construct the corresponding boundary feature vector;
- the intelligent control unit controls the outdoor automatic tool to continuously walk autonomously along the boundary line, and collects a series of boundary feature vectors autonomously until the feature vectors repeat, and establishes the working area boundary based on the acquired boundary coordinate points and corresponding feature vectors The feature coordinate map of .
- the intelligent control unit can control the outdoor automatic tool to continuously walk autonomously along the boundary line based on the external remote control signal, and collect a series of boundary feature vectors autonomously until the feature vector repeats, based on the obtained boundary coordinate points and corresponding eigenvectors to build a eigencoordinate map of the working area boundary.
- the intelligent control unit estimates the motion process information of the outdoor automatic tool based on the surrounding image information of the outdoor automatic tool acquired by the machine vision image sensor, and the motion process information includes the displacement information and the spatial attitude information of the outdoor automatic tool autonomously walking.
- the intelligent control unit acquires the transformation relationship between the RT coordinate system between two adjacent frames, multiplies the obtained multiple RTs to obtain the transformation relationship between the current frame and the original position, and then performs iterative optimization;
- the fusion motion sensing module obtains the motion state information of the outdoor automatic tool, and calculates the displacement of the outdoor automatic tool through time integration, thereby completing the estimation of the outdoor automatic tool pose information and motion displacement information.
- the outdoor automatic work control method based on machine vision includes:
- a characteristic coordinate map of the boundary of the working area is established based on the image information around the outdoor automatic tool acquired by machine vision;
- control method establishes the characteristic coordinate map of the boundary of the working area, it includes:
- Control the movement of the outdoor automatic tool obtain the image information around the outdoor automatic tool to judge whether the outdoor automatic tool is in the specified type of work area;
- the position coordinates of the boundary points of the work area are collected, and based on the coordinate points of each position and their corresponding feature vectors, a feature map of the work area boundary is generated from the feature vectors and position coordinates.
- the outdoor automatic work control method judges whether the outdoor automatic tool is in the designated work area by identifying the presence, absence, or proportion of the corresponding work area in the surrounding image of the outdoor automatic tool.
- the outdoor automatic work control method also fuses the depth information of the boundary line of the working area and the angle information of the outdoor automatic tool with respect to the boundary line of the working area to determine whether the outdoor automatic tool is in the designated working area.
- the method further estimates the motion process information of the outdoor automatic tool based on the acquired image information around the outdoor automatic tool, and the motion process information includes the displacement information and the space attitude information of the outdoor automatic tool autonomously walking.
- the method first acquires the transformation relationship between the RT coordinate system between two adjacent frames, and multiplies the acquired multiple RTs to obtain the transformation relationship between the current frame and the original position;
- the automatic operation control mode for controlling the outdoor automatic tool to return to the working area includes one or more of edge-to-edge mode, random mode, path planning mode, automatic recharging mode, and obstacle avoidance mode.
- the present invention provides a terminal device, which includes a processor, a memory, and a program stored on the memory and operable on the processor.
- the program code is loaded by the processor and executes the above-mentioned outdoor automatic Steps of the job control method.
- the solution provided by the present invention greatly simplifies the deployment conditions of outdoor automatic working equipment based on machine vision and neural network technology, increases the reliability and control accuracy of automatic work of outdoor automatic working equipment in the working area, and improves the work of outdoor automatic working equipment. efficiency.
- the solution provided by the present invention can increase the working efficiency and control precision of the outdoor automatic working equipment while controlling the cost, and fundamentally realize the full coverage of the working area.
- the solution provided by the invention improves the coverage of the working area and reduces the possibility that the equipment is affected by other alternating magnetic fields, electromagnetic boundary line disconnection and power failure during operation.
- Fig. 1 is the composition example figure of outdoor automatic work control system among the present invention
- Fig. 2 is the setting schematic diagram of outdoor automatic tool among the present invention
- Fig. 3 is the flowchart example diagram of the feature coordinate map of establishing the working area boundary among the present invention.
- Fig. 4 is a logical example diagram for controlling the automatic operation of outdoor automatic tools in the present invention.
- Fig. 5 is an example diagram of the process of automatically establishing a work map by a garden robot in an example of the present invention
- Fig. 6 is an example diagram of the flow of the garden robot in the random working mode in the example of the present invention.
- Fig. 7 is an example diagram of the working path of the garden robot in the random working mode in the example of the present invention.
- Fig. 8 is an example diagram of the working path with absolute position information when the garden robot is in the path planning mode in the example of the present invention
- Fig. 9 is an example diagram of the working path under pure machine vision when the garden robot is in the path planning mode in the example of the present invention.
- Fig. 10 is an example diagram of the workflow of the garden robot in the path planning mode in the example of the present invention.
- Fig. 11 is an example diagram of the workflow of the garden robot in the obstacle avoidance mode in the example of the present invention.
- this scheme provides a path planning scheme for outdoor automatic tools based on machine vision, which realizes the camera-led, greatly simplifies the deployment conditions of outdoor automatic tools through machine vision and neural networks, and increases the outdoor automatic tools.
- the reliability in the working area improves the working efficiency of outdoor automatic tools.
- this program builds a set of outdoor automatic work control system based on machine vision, realizes machine vision technology based on deep learning, and independently identifies the working area, so that the machine can keep working automatically in the working area.
- FIG. 1 it shows an example of the composition of the outdoor automatic work control system based on machine vision given by this solution.
- the outdoor automatic work control system 100 based on machine vision mainly includes several functional modules such as a machine vision image sensor 110 , an AI intelligent unit 120 , an embedded processor 130 , a positioning module 140 , and a motion sensing module 150 .
- the machine vision image sensor 110 acquires the surrounding image information of the outdoor automatic tool in real time.
- the machine vision image sensor interacts with the data of the AI intelligent unit 120, and can input the acquired image information around the outdoor automatic tool to the AI intelligent unit 120 for AI algorithm calculation and processing to identify the corresponding working area boundary.
- the specific composition of the machine vision image sensor 110 may be determined according to actual requirements. For example, it may be one or more of a single-eye camera, a binocular camera, a panoramic camera, RGBD, and the like.
- the specific deployment of the machine vision image sensor can be determined according to actual needs, for example, it can be deployed around and/or on the top of the outdoor automatic tool body.
- the positioning module 140 in this system is used to obtain the spatial position information of the outdoor automatic tool.
- the positioning module 140 can perform data interaction with the embedded processor 130, complete spatial positioning calculation reference, and correct the cumulative error of the system.
- the specific composition of the positioning module 140 can be determined according to actual needs.
- a radio positioning module is preferably used, such as satellite positioning system, GNSS, RTK, RTD, base station positioning, AGPS, UWB ultra-wideband positioning module, One or more of Bluetooth positioning modules, etc.
- the specific deployment of the positioning module 140 may be determined according to actual needs, and as an example, it may be deployed inside the body of the outdoor automatic tool.
- the motion sensing module 150 in this system is used to obtain the motion state information of the outdoor automatic tool.
- the motion sensing module 150 performs data interaction with the embedded processor 130 to complete real-time detection of the motion state of the outdoor automatic tool, calculates the walking distance of the outdoor automatic tool through a sensor fusion algorithm, and determines a rough coordinate for the boundary point, so as to Used to map the work area.
- the specific composition of the motion sensing module 150 may be determined according to actual requirements.
- an inertial navigation sensor, a motor digital encoder, etc. are preferably used.
- the motion sensing module can be deployed inside the outdoor automatic tool body during specific deployment.
- the AI intelligent unit 120 in this system cooperates with the embedded processor 130 to form the intelligent control unit of the entire automatic work control system, and completes the automatic work control of the outdoor automatic tools.
- the resulting intelligent control unit performs data interaction with the machine vision image sensor, and the machine vision based on deep learning completes the autonomous identification of the working area, and controls the outdoor automatic tools to automatically work in the working area.
- the specific composition of the AI intelligent unit 120 can be determined according to actual needs.
- one or more of GPU, FPGA, DSP, NCU, artificial intelligence ASIC, etc. are preferably used.
- the specific deployment of the AI intelligent unit 120 and the embedded processor 130 may be determined according to actual needs. For example, they may be deployed inside the body of an outdoor automatic tool.
- the system also includes a power supply module 161, a charging module 162 and a motor drive module 163. These modules cooperate with the embedded processor and are deployed inside the outdoor automatic tool body to complete the basic driving control of the outdoor automatic tool.
- the configuration of the power supply module 161 , the charging module 162 and the motor driving module 163 is not limited, and may be determined according to actual needs.
- the system is further provided with an obstacle recognition sensor 180 and an electromagnetic boundary sensor 170 to further improve the accuracy and reliability of the system for controlling outdoor automatic tools.
- the obstacle recognition sensor 180 here performs data interaction with the embedded processor 130, and is used to record the current obstacles on the ground surface, further supplementing the machine vision recognition process, and assisting in establishing a work task area according to the surface obstacles.
- the obstacle recognition sensor 180 may be determined according to actual needs, and as an example, it may preferably be deployed around the body of the outdoor automatic tool.
- the electromagnetic boundary sensor 170 here is specifically an electromagnetic boundary sensor used to assist judgment in the case of limited light. As a further supplement to machine vision recognition, it assists in establishing a work task area according to the preset electromagnetic boundary on the ground.
- the electromagnetic boundary sensor 170 may be determined according to actual requirements, and as an example, it may preferably be deployed around the body of the outdoor automatic tool.
- the outdoor automatic work control system when controlling the outdoor automatic tool to work, firstly, the AI intelligent unit acquires the image information around the outdoor automatic tool based on the machine vision image sensor, and the positioning module obtains the spatial position information of the outdoor automatic tool And the motion sensing module obtains the motion state information of the outdoor automatic tool, and establishes a characteristic coordinate map of the working area boundary after the outdoor automatic tool runs along the working area boundary for one week;
- the AI intelligent unit calculates the positional relationship of the outdoor automatic tool relative to the working area boundary in real time, and performs an outdoor When the automatic tool reaches the boundary of the working area, an automatic work control mode for the outdoor automatic tool to return to the working area is formed.
- This solution controls the outdoor automatic tool to complete the work area identification and automatically completes the full coverage work realization process in the identified work area.
- the outdoor automatic work control system given in this scheme mainly includes two stages of establishing a working area map and automatic work when controlling outdoor automatic tools for automatic work.
- This stage is used for the initial work of the outdoor automatic tool to establish a characteristic coordinate map of the working area boundary for the area to be worked.
- the outdoor automatic work control system controls the outdoor automatic tool to automatically find the boundary of the work area to run for a week in the work area, and establishes the characteristic coordinate map of the boundary of the work area through the machine vision image sensor, radio positioning module and motion sensor.
- control system collects images around the outdoor automatic tool in real time through the machine vision sensor, and then the artificial intelligence unit uses the neural network to identify and segment the collected image, and judges whether the outdoor automatic tool is close to the specified working area boundary:
- the absolute coordinate points in the positioning information obtained by the positioning module in the outdoor automatic tool, the boundary direction vector based on the pose of the outdoor automatic tool, and the feature information in the boundary image are collected synchronously, and the outdoor automatic tool is established accordingly.
- the feature vector of the currently reached boundary point is established accordingly.
- control system will control the outdoor automatic tool to adjust the driving direction, so that the outdoor automatic tool can walk autonomously and continuously along the boundary line of the working area, and collect a series of boundary feature vectors synchronously until the feature vector appears repeat.
- control system filters and optimizes the stored coordinate points and their feature vectors through the embedded processor, and establishes a feature coordinate map of the boundary of the working area based on the optimized data.
- the automatic outdoor tool when controlling the outdoor automatic tool to run around the boundary of the working area, in addition to the aforementioned method of automatically finding the boundary of the working area in the working area, the automatic outdoor tool can also be manually controlled to run along the boundary of the working area. Run for a week.
- the control system synchronously collects the coordinate points and eigenvectors of the corresponding boundary points, filters and optimizes the collected coordinate points and their eigenvectors, and based on the optimization Based on the data, a characteristic coordinate map of the working area boundary is established.
- the remote control mode can be used to assist the outdoor automatic tool to complete the work area
- the establishment of boundary feature coordinate map The operator (such as the user) can remotely control the outdoor automatic tool to walk along the boundary of the work area through the remote control device, and collect a series of boundary feature vectors synchronously during the walking process until the feature vector repeats, and the collected coordinate points and their features
- the vectors are filtered and optimized, and a characteristic coordinate map of the working area boundary is established based on the optimized data.
- the artificial intelligence unit in the system performs calculations based on the neural network algorithm of deep learning on the images around the outdoor automatic tools collected and input by the machine vision image sensor , optimization, acceleration; on this basis, identify and segment the working area in the image to output machine-recognizable working area boundary information, where the working area boundary information includes boundary types, direction vectors and normal vectors, and The included angle with the direction of the device, and based on this information, the corresponding boundary feature vector is input into the processing unit of the system.
- the outdoor automatic work control system also uses the image information collected and input by one or more machine vision image sensors to estimate the movement process information of the outdoor automatic tool during the process of building the map of the working area.
- the motion process information here mainly refers to the displacement information of the autonomous walking of the outdoor automatic tool and the spatial attitude information.
- this solution acquires the RT coordinate system transformation relationship between two adjacent frames of pictures, and multiplies the obtained multiple RTs to obtain the transformation relationship between the current frame and the original position, that is, calculates the outdoor automatic tool frame and then perform iterative optimization on this basis; at the same time, the artificial intelligence processing unit fuses the speed information and acceleration information of outdoor automatic tools collected by motion sensing modules such as encoders and gyroscopes, and calculates the automatic The displacement change of the outdoor automatic tool completes the estimation of the pose information and motion information of the outdoor automatic tool.
- the features of the boundary line image can be extracted more conveniently after machine vision recognition based on the neural network.
- the motion changes between frames are calculated to filter out unnecessary feature points, so as to achieve more efficient use of image information and calculate the pose of outdoor automatic tools
- Information and motion information are estimated to obtain the walking path and direction angle of the outdoor automatic tool, so that the boundary coordinates of the boundary line map to be established are more complete and accurate.
- the estimation of motion pose only depends on the data between two adjacent frames of images (including information such as image, acceleration, angular velocity, etc.), errors in pose estimation will accumulate.
- the positioning information of the outdoor automatic tool is obtained through the radio positioning module, and the cumulative error is eliminated based on this, and the positioning accuracy of the automatic tool is improved.
- the high-precision positioning point information of the outdoor automatic tool can be obtained with the assistance of the radio module , to correct the drift error amount of the cumulative error in real time or intermittently, so as to eliminate the offset of the boundary point coordinates caused by the cumulative error, so that a complete and closed boundary line map can be formed based on the corrected data.
- the outdoor automatic work control system When the outdoor automatic work control system completes the construction of the work area map, it stores the coordinate points of each position and their corresponding feature vectors based on the position coordinates of the boundary points of the work area and the way the machine traverses through this position, and generates A feature map of the work area boundary consisting of feature vectors and position coordinates.
- the outdoor automatic work control system controls the outdoor automatic tool to construct the work area map for the corresponding work area, it controls the outdoor automatic tool to find the boundary of the work area in the work area by means of automatic or manual remote control according to the control mode.
- the outdoor automatic tool is controlled to initialize, that is, the outdoor automatic tool is controlled to rotate in situ, and the image information around the outdoor automatic tool is acquired.
- the outdoor automatic tool is controlled to rotate in situ, which can scan and obtain the working boundary image around the outdoor automatic tool, so that the outdoor automatic tool can find the boundary line in the nearest way, and start to build a map of the working area along the line;
- the outdoor automatic tool is controlled to rotate in situ, and the image of the working area around the outdoor automatic tool can also be scanned and obtained.
- the image recognition results around the outdoor automatic tool are all within the working area, it can be judged that the outdoor automatic tool is in the working area and can start safe work.
- the outdoor automatic tool is controlled to rotate in situ, and the magnetometer can be calibrated at the same time to obtain a more accurate orientation angle.
- the outdoor automatic tool is controlled to find the boundary movement of the working area in the working area, and based on machine vision, it is calculated in real time whether the outdoor automatic tool is close to the boundary of the working area.
- the outdoor automatic tool reaches the boundary of the working area, the feature vector and position coordinates of the boundary point are collected, and the position coordinates of the boundary point and the corresponding feature vector are stored.
- the outdoor automatic tool is controlled to traverse the boundary of the working area for a week, and the coordinate points of each boundary position and their corresponding feature vectors are stored, and a feature map of the working area boundary is generated from the feature vectors and position coordinates.
- human-computer interactive confirmation can be carried out on the characteristic map of the established working area boundary as required to complete the final confirmation and protection.
- the outdoor automatic working control system controls the outdoor automatic working tool by comparing the characteristic coordinates of the outdoor automatic tool itself with the characteristic coordinate map of the working area boundary; on the other hand, by The deep learning method of machine vision can identify and segment the working area in the image, output the boundary information of the working area that can be recognized by the machine, and help the control system of the outdoor automatic tool decide whether it is in the designated working area, that is, whether it is in the working area or not.
- the deep learning method of machine vision can identify and segment the working area in the image, output the boundary information of the working area that can be recognized by the machine, and help the control system of the outdoor automatic tool decide whether it is in the designated working area, that is, whether it is in the working area or not.
- the area boundary feature map Within the working area defined by the area boundary feature map.
- the depth information of the boundary line calculated by the neural network and the angle information of the outdoor automatic tool relative to the boundary line can be further integrated to judge whether the outdoor automatic tool is working in the designated working area, so as to improve the accuracy of judgment.
- the outdoor automatic working control system After determining that the outdoor automatic working tool is in the working area defined by the characteristic map of the working area boundary, the outdoor automatic working control system will control and control the work of the outdoor automatic working tool in the working area defined by the characteristic map of the working area boundary based on the established characteristic map of the working area boundary Automatic driving in the area according to the predetermined working mode, that is, automatic work is realized.
- the outdoor automatic work control system also utilizes the input surrounding image information of the image sensor, fuses the absolute positioning information of the motion sensing module and the positioning module, and generates the characteristic coordinates and position of the outdoor automatic working tool itself in real time. Estimation of posture information and motion information. Further ensure the automatic work of outdoor automatic working tools in the designated working area, realize autonomous walking without stepping out of the designated working area and automatic work according to a certain operating mode and logic.
- the outdoor automatic work control system controls the automatic work of the outdoor automatic tool in the working area defined by the characteristic coordinate map, it calculates the positional relationship of the outdoor automatic tool relative to the boundary of the working area based on machine vision in real time, and when the outdoor automatic tool reaches the boundary of the working area , according to the selected automatic working mode, control the outdoor automatic tool to return to the working area according to the mode requirements.
- the automatic working mode here may be one or more of edge-edge mode, random mode, path planning mode, automatic recharging mode, and obstacle avoidance mode.
- This example takes the garden robot as an example.
- a corresponding software program of the outdoor automatic work control system based on machine vision is set in the garden robot.
- the software program When the software program is run by the corresponding processor, it can execute the aforementioned outdoor automatic work control method. A step of.
- the courtyard robot deployed in this way can automatically plan the working area through machine vision according to the different tasks assigned by the user.
- the working area is identified by the deep learning algorithm and semantically segmented, and the natural boundary is generated into a machine-recognizable Boundary information of the working area, including boundary types, direction vectors and normal vectors, and the angle between them and the direction of the equipment; then, the identification and planning of the working area are specifically completed.
- the garden robot When the garden robot is started and running, it can automatically perform garden maintenance work, such as weeding, leaf cleaning, snow removal, fertilization, grass seed sowing, patrol and other garden maintenance work.
- garden maintenance work such as weeding, leaf cleaning, snow removal, fertilization, grass seed sowing, patrol and other garden maintenance work.
- the garden robot When the garden robot is working, it mainly includes a mapping mode and an automatic working mode.
- the garden robot given in this example can adopt two mapping modes, namely manual remote mapping mode and automatic mapping mode (see Figure 5).
- the machine vision algorithm of deep learning will generate multi-dimensional electronic information based on the environment.
- Information map that is, a map of characteristic coordinates of the boundary of the working area.
- the courtyard robot walks a closed path along the boundary of the working area, which is the approximate working area.
- the system automatically recognizes the boundary of the working area and establishes a working restricted area.
- the work boundary and work restricted area can also be manually set again on the upper computer manually.
- the garden robot is controlled to automatically work along the boundary of the designated work area in the work area and to establish a map of the work area along the boundary line.
- the courtyard robot when the courtyard robot automatically establishes the map of the working area, the courtyard robot is initialized first, and the courtyard robot is controlled to rotate in situ in the working area. When there is no boundary, it goes straight in a random direction to find the boundary of the working area. At the same time, obtain the image of the surrounding environment of the garden robot, and calculate and judge whether the garden robot is in the specified type of work area based on the acquired image, such as: lawn, street, snow, etc.
- FIG. 5 it gives an example of the logical process of realizing the automatic mapping of the garden robot in this example.
- the garden robot (hereinafter referred to as the device) around the outer boundary of the working area, or start from the charging pile set at the boundary of the map, and set the machine to enter the map building mode.
- the device begins to initially map the outer boundaries of the work area.
- the device walks straight from the starting point or rotates on the spot.
- the machine vision sensor on the device obtains the surrounding environment image of the garden robot.
- the intelligent processing unit in the device will calculate and judge whether the garden robot is in the specified type of work area based on the acquired image. , such as: lawn, street, snow, etc. If not, stop the alarm.
- the intelligent processing unit in the equipment recognizes that the equipment is close to or is about to drive out of the continuous boundary of the designated working area, and then sends an edge command to the control execution unit, and the equipment turns to the direction close to the boundary of the working area, and the running direction of the equipment is parallel to the working area.
- Area boundary direction vector while ensuring that the device is guaranteed to work within the specified type of work area. Take the starting point 1 of the map boundary as the starting point of the boundary until returning to this point to complete the closure of the working area.
- the starting point is the characteristic coordinate point of the boundary found for the first time or the characteristic coordinate point of the charging pile set on the working boundary.
- this example scheme conveniently extracts the features of the boundary line image through neural network-based machine vision recognition; at the same time, it calculates the motion change between frames through the difference between two adjacent frames of images, and can filter Remove unnecessary feature points to achieve more efficient use of image information, calculate the estimation of the pose information and motion information of the device, obtain the path and direction angle that the device has traveled, and make the boundary of the boundary line map to be established
- the coordinates are more complete and accurate.
- the garden robot After completing the mapping of the working area, the garden robot enters the automatic working mode, and will complete the automatic work with full coverage in the working area planned by the established map.
- the device can perform automatic working modes such as edge mode, random mode, path planning mode, automatic recharging mode, and obstacle avoidance mode.
- the device automatically works randomly in the working area and completes the full coverage of the working area.
- the device when the device enters the automatic random working mode, acquires images of the surrounding environment of the garden robot, and calculates and judges whether the device is in a specified type of working area based on the acquired images;
- random path coverage is carried out in the working area.
- the intelligent control unit in the device judges whether the device is in the designated working area; if the intelligent control in the device The unit recognizes that the equipment is close to or is about to leave the boundary of the designated work area, and then sends a steering command to the control execution unit of the equipment, which controls the equipment to turn and move away from the boundary of the work area according to the corresponding steering strategy, so that the equipment can ensure that the specified type of work work in the area.
- This example takes the random mode as an example.
- the device automatically works randomly in the working area and completes the full coverage of the working area.
- the intelligent control unit in the device completes the visual boundary in the path planning working mode according to the position information provided by the radio positioning module, the surrounding environment image of the device collected by the machine vision sensor, and the identified boundary position information. Identify and work automatically.
- the device in this working mode, can provide position information through the radio positioning module, and integrate the feature information of the boundary line image extracted by the machine vision recognition of the neural network, through the difference between two adjacent frames of images , to calculate the motion change between frames, calculate the estimation of the device's pose information and motion information, and obtain the coordinates and direction angles that the device has traveled.
- the path planning algorithm is used to cover the boundary map of the work area to achieve path planning with full coverage of the work area.
- the device can only perform positioning navigation and pose estimation through vision. Clockwise or counterclockwise, combined with the direction vector information of the boundary line, out of the trajectory parallel and perpendicular to the map boundary, at most traverse the boundary line, so as to realize path planning and complete coverage of the working area.
- the device After the device receives the absolute position information sent by the radio positioning module, it can still continue path planning according to the working mode of combining visual signals and position signals.
- FIG. 10 it is a flow chart showing the realization of the machine vision-based path planning working mode of the device given in this example.
- the device when the device is working in the path planning mode, the device obtains the surrounding environment image of the garden robot, and calculates and judges whether the device is in the specified type of working area according to the obtained image; if not, it will issue a shutdown alarm.
- control device moves straight according to the current working posture, and at the same time acquires images of the surrounding environment of the garden robot.
- the intelligent control unit in the equipment will recognize whether the equipment is close to the boundary of the working area.
- the intelligent control unit in the equipment recognizes that the equipment is close to the boundary of the working area, and simultaneously calculates the depth information, direction vector and normal vector of the boundary line.
- the motion control unit in the device controls the device to turn at a certain angle, so that the forward direction of the device is parallel to the boundary line.
- control device continues to go straight for a preset distance.
- the motion control unit in the equipment controls the equipment to turn to a certain angle, so that the forward direction of the equipment is perpendicular to the boundary line.
- the device completes the full coverage of the working area boundary map according to this mode.
- the device after the device completes the full coverage of the working area boundary map, it can automatically return to the initial working position, charging pile, etc. according to the set route.
- the device can recognize and process dynamic and static obstacles during operation.
- the machine vision sensor in the device inputs the image of the surrounding environment of the device, and the intelligent control unit in the device recognizes obstacles according to the image of the surrounding environment input by the machine vision sensor, and will enter the obstacle avoidance mode.
- the intelligent control unit in the device detects an obstacle for the first time, and the recognition time is less than the preset trigger time t, the intelligent control unit in the device judges that a dynamic obstacle has been encountered, and sends a command to the motion control unit of the device to control the device Stop moving until the obstacle clears and move on.
- the intelligent control unit in the device When the intelligent control unit in the device detects an obstacle and continues to recognize the obstacle, and the recognition time is greater than the preset trigger time t, the intelligent control unit in the device encounters a static obstacle by default, and the intelligent control unit in the device sends The command is sent to the motion control unit of the device to control the device to try to bypass obstacles; at the same time, the intelligent control unit in the device will regenerate a planned route to ensure the coverage of the working area map.
- this example is given to the outdoor automatic working equipment dominated by machine vision sensors, which greatly simplifies the deployment conditions of outdoor automatic working equipment through machine vision and neural networks, and increases the reliability of outdoor automatic equipment in the working area. It improves the working efficiency of outdoor automatic working equipment.
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Aviation & Aerospace Engineering (AREA)
- Radar, Positioning & Navigation (AREA)
- Remote Sensing (AREA)
- General Physics & Mathematics (AREA)
- Automation & Control Theory (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Multimedia (AREA)
- Electromagnetism (AREA)
- Manipulator (AREA)
- Control Of Position, Course, Altitude, Or Attitude Of Moving Bodies (AREA)
Abstract
一种基于机器视觉的室外自动工作控制系统（100）、方法及设备，控制室外自动工具沿工作区域边界运行一周后，基于机器视觉所获取到的室外自动工具周围图像信息建立工作区域边界的特征坐标地图；控制室外自动工具在由特征坐标地图限定的工作区域内工作，并基于机器视觉实时计算室外自动工具相对于工作区域边界的位置关系，并在室外自动工具到达工作区域边界时，对室外自动工具形成返回工作区域的自动工作控制模式。基于机器视觉和神经网络技术大大简化了室外自动工作设备的部署的条件，增加的室外自动工作设备在工作区域内自动工作的可靠性和控制精度，提高了室外自动设备的工作效率。
Description
本发明涉及设备自动化技术，具体涉及室外工作设备自动工作控制技术。
室外自动工具基于使用时的便捷性，在人们的日常生活中得到越来越多的应用。特别是一些能够在特定工作区域内自动完成辅助任务的室外自动工具，极大的受到欢迎，如庭院除草机器人等。对于这类的室外自动工具在实现时，其首要解决的问题是如何精确识别对应的工作区域边界，确定工作区域；在此基础上，如何在工作区域内进行工作路径规划，以尽可能的最大范围覆盖整个工作区域，同时还能够越过工作区域边界。
对此，目前常见的室外自动工具，如庭院除草机器人，在确定工作区域边界方面，往往是使用导线上输入固定频率的交变电流或者固定频率的电脉冲，利用变化的电流在导体上产生磁场的作为电磁边界；在此基础上，再使用电感或者霍尔元件感应地面及平面上的磁场信号强弱，进行巡线运动和跟踪。同时在运动路径方面，只能在电磁边界框定范围内做无序的乒乓运动。
这样的技术在实际应用时存在诸多问题，其中集中在如下几点：
(1)只能简单的就一维的电磁边界和远近进行非常粗糙的判断，无法利用电感值进行精确地定位或姿态解算；
(2)无法建立工作地图，也无法定位，在电磁边界框定范围内做无序的乒乓运动工作和覆盖地图的效率低下；
(3)传感器结构单一，容易受低下市电的交变电的干扰，一旦某边界断电和损坏，机器容易驶离工作区域，发生越界，造成安全隐患。
由此可见，现有的室外自动工具存在自动控制精度不高，以及可靠性差的问题，如何有效提高室外自动工具自动工作的控制精度以及可靠性为本领域亟需解决的问题
发明内容
针对现有室外自动工具的自动工作控制技术在控制精度和可靠性方面所存在的问题，需要一种新的室外自动工具的自动工作控制技术。
为此，本发明的目的在于提供一种基于机器视觉的室外自动工作控制系统，实现基于机器视觉来实现提高室外自动工具自动工作的控制精度以及可靠性；
进一步的，本发明还进一步提供基于机器视觉的室外自动工作控制方法，以及可运行该室外自动工作控制方法的自动工作设备。
为了达到上述目的，本发明提供的基于机器视觉的室外自动工作控制系统，包括机器视觉图像传感器、智能控制单元、定位模块、运动传感模块，
所述机器视觉图像传感器实时获取室外自动工具周围图像信息；
所述定位模块获取室外自动工具的空间位置信息；
所述运动传感模块获取室外自动工具的运动状态信息；
所述智能控制单元基于机器视觉图像传感器获取的室外自动工具周围图像信息、定位模块获取室外自动工具的空间位置信息以及运动传感模块获取室外自动工具的运动状态信息，在室外自动工具沿工作区域边界运行一周后，建立工作区域边界的特征坐标地图；
所述智能控制单元基于室外自动工具周围图像信息、空间位置信息运动状态信息，以及所建立的工作区域边界的特征坐标地图，实时计算室外自动工具相对于工作区域边界的位置关系，并在室外自动工具到达工作区域边界时，对室外自动工具形成返回工作区域的自动工作控制模式。
进一步的，所述智能控制单元
根据机器视觉图像传感器采集的周围图像信息，利用神经网络对图像进行识别和分割，并判断室外自动工具是否接近指定的工作区域边界；如未能检测到边界，控制室外自动工具依据当前状态继续行驶，直到识别到边界；届时，同步的采集室外自动工具中定位模块获取到的绝对坐标点，以构建对应的边界特征向量；
所述智能控制单元控制室外自动工具连续的沿边界线自主行走，并自主采集一系列边界的特征向量，直到出现特征向量重复，基于所获取到的边界坐标点以及对应的特征向量来建立工作区域边界的特征坐标地图。
进一步的，所述智能控制单元可基于外部遥控信号控制室外自动工具连续的沿边界线自主行走，并自主采集一系列边界的特征向量，直到出现特征向量重复，基于所获取到的边界坐标点以及对应的特征向量来建立工作区域边界的特征坐标地图。
进一步的，所述智能控制单元基于机器视觉图像传感器获取的室外自动工具周围图像信息估计室外自动工具的运动过程信息，所述运动过程信息包括室外自动工具自主行走的位移信息及空间姿态信息。
进一步的，所述智能控制单元通过获取相邻两帧之间的RT坐标系变换关系，将获取得到的多个RT相乘获取得到当前帧与原始位置之间的变换关系，然后进行迭代优化；同时，融合运动传感模块获取室外自动工具的运动状态信息，通过时间积分计算室外自动工具的位移，由此完成对室外自动工具位姿信息及运动位移信息的估算。
为了达到上述目的，本发明提供的基于机器视觉的室外自动工作控制方法，包括：
控制室外自动工具沿工作区域边界运行一周后，基于机器视觉所获取到的室外自动工具周围图像信息建立工作区域边界的特征坐标地图；
控制室外自动工具在由特征坐标地图限定的工作区域内工作，并基于机器视觉实时计算室外自动工具相对于工作区域边界的位置关系，并在室外自动工具到达工作区域边界时，对室外自动工具形成返回工作区域的自动工作控制模式。
进一步的，所述控制方法在建立工作区域边界的特征坐标地图时，包括：
控制室外自动工具动作，获取室外自动工具周围图像信息，以判断取室外自动工具是否处于指定类型的工作区域内；
根据获取到的室外自动工具周围图像信息计算判断室外自动工具是否接近工作区域的边界；
采集工作区域边界点的位置坐标，基于各个位置的坐标点及其对应的特征向量，生成由特征向量和位置坐标的工作区域边界的特征地图。
进一步的，所述室外自动工作控制方法通过识别室外自动工具周围图像中 相应工作区域的有、无、或比例，判断室外自动工具是否在指定工作区域内。
进一步的，所述室外自动工作控制方法还融合工作区域边界线深度信息，以及室外自动工具相对于工作区域边界线的夹角信息来判断室外自动工具是否在指定工作区域内。
进一步的，所述方法还基于获取的室外自动工具周围图像信息估计室外自动工具的运动过程信息，所述运动过程信息包括室外自动工具自主行走的位移信息及空间姿态信息。
进一步的，所述方法首先通过获取相邻两帧之间的RT坐标系变换关系，将获取得到的多个RT相乘获取得到当前帧与原始位置之间的变换关系；
接着，进行迭代优化，并融合运动传感模块获取室外自动工具的运动状态信息，通过时间积分计算室外自动工具的位移，由此完成对室外自动工具位姿信息及运动位移信息的估算。
进一步的，所述控制室外自动工具返回工作区域的自动工作控制模式包括沿边模式、随机模式、路径规划模式、自动回充模式、避障模式中的一种或多种。
为了达到上述目的，本发明提供的一种终端设备，设备包括处理器、存储器及存储在存储器上并可在处理器上运行的程序，所述程序代码由所述处理器加载并执行上述室外自动工作控制方法的步骤。
本发明提供的方案基于机器视觉和神经网络技术大大简化了室外自动工作设备的部署的条件，增加的室外自动工作设备在工作区域内自动工作的可靠性和控制精度，提高了室外自动设备的工作效率。
本发明提供的方案在具体应用时，能够在控制成本的同时增加室外自动工作设备的工作效率和控制精度，从根本上实现对工作区域的全覆盖。
本发明提供的方案在具体应用时，提高工作区域的覆盖率，以及降低了设备在运行过程中受其他交变磁场，和电磁边界线断开和断电影响的可能性。
以下结合附图和具体实施方式来进一步说明本发明。
图1为本发明中室外自动工作控制系统的构成示例图；
图2为本发明中室外自动工具的设置原理图；
图3为本发明中建立工作区域边界的特征坐标地图的流程示例图；
图4为本发明中控制室外自动工具自动工作的逻辑示例图；
图5为本发明实例中庭院机器人自动建立工作地图的流程示例图；
图6为本发明实例中庭院机器人进行随机工作模式的流程示例图；
图7为本发明实例中庭院机器人进行随机工作模式的工作路径示例图；
图8为本发明实例中庭院机器人进行路径规划模式时，有绝对位置信息下的工作路径示例图；
图9为本发明实例中庭院机器人进行路径规划模式时，纯机器视觉下的工作路径示例图；
图10为本发明实例中庭院机器人在路径规划模式下的工作流程示例图；
图11为本发明实例中庭院机器人在避障模式下的工作流程示例图。
为了使本发明实现的技术手段、创作特征、达成目的与功效易于明白了解，下面结合具体图示，进一步阐述本发明。
针对室外工作环境的特殊性，本方案给出基于机器视觉的室外自动工具路径规划方案，实现以摄像头为主导，通过机器视觉和神经网络大大简化室外自动工具的部署的条件，增加的室外自动工具的在工作区域内的可靠性，提高室外自动工具的工作效率。
在此基础上，本方案构建一套基于机器视觉的室外自动工作控制系统，实现基于深度学习的机器视觉技术，自主识别工作区域，使得机器保持在工作区域内自动工作。
参见图1，其所示为本方案给出的基于机器视觉的室外自动工作控制系统的一种构成示例。
本基于机器视觉的室外自动工作控制系统100主要包括机器视觉图像传感器110，AI智能单元120，嵌入式处理器130，定位模块140，运动传感模块150等这几个功能模块。
其中，机器视觉图像传感器110实时获取室外自动工具周围图像信息。该 机器视觉图像传感器与AI智能单元120数据交互，可将获取到的室外自动工具周围图像信息输入至AI智能单元120进行AI算法运算处理，识别相应的工作区域边界。
在具体实现时，该机器视觉图像传感器110的具体构成可根据实际需求而定，作为举例，其可以为目摄像头，双目摄像头，全景摄像头，RGBD等中的一种或多种。
该机器视觉图像传感器在具体部署时，可根据实际需求而定，作为举例，可部署在室外自动工具本体的四周和/或顶部。
本系统中的定位模块140，用于获取室外自动工具的空间位置信息。
作为优选，该定位模块140可与嵌入式处理器130进行数据交互，完成空间定位计算参考，可对系统的累积误差进行修正。
在具体实现时，该定位模块140的具体构成可根据实际需求而定，作为举例，优选采用无线电定位模块，如卫星定位系统、GNSS、RTK、RTD、基站定位、AGPS、UWB超宽带定位模块、蓝牙定位模块等中的一种或多种
如图2所示，该定位模块140在具体部署时，可根据实际需求而定，作为举例，可部署在室外自动工具本体的内部。
本系统中的运动传感模块150，用于获取室外自动工具的运动状态信息。
具体的，本运动传感模块150与嵌入式处理器130进行数据交互，完成实时检测室外自动工具的运动状态，通过传感器融合算法计算室外自动工具行走距离，为边界点确定一个大致的坐标，以用于绘制工作区域地图。
在具体实现时，该运动传感模块150的具体构成可根据实际需求而定，作为举例，优选采用惯性导航传感器、电机数字编码器等。
该运动传感模块在具体部署时，可部署在室外自动工具本体的内部。
本系统中的AI智能单元120与嵌入式处理器130配合构成整个自动工作控制系统的智能控制单元，完成对室外自动工具的自动工作控制。
由此形成的智能控制单元，与机器视觉图像传感器进行数据交互，基于深度学习的机器视觉完成工作区域的自主识别，并控制室外自动工具自动工作在工作区域内。
在具体实现时，该AI智能单元120的具体构成可根据实际需求而定，作 为举例，优选采用GPU、FPGA、DSP、NCU、人工智能ASIC等中的一种或多种。
如图2所示，该AI智能单元120与嵌入式处理器130在具体部署时，可根据实际需求而定，作为举例，可部署在室外自动工具本体的内部。
同时，本系统中还包括电源模块161、充电模块162以及电机驱动模块163，这些模块与嵌入式处理器配合，部署在室外自动工具本体的内部，以完成对室外自动工具的基础驱动控制。
这里对于电源模块161、充电模块162以及电机驱动模块163的构成，不加以限定，可根据实际需求而定。
在此基础上，本系统中进一步设置障碍识别传感器180和电磁边界传感器170，以进一步提高本系统实现对室外自动工具控制的精度和可靠性。
这里的障碍识别传感器180与嵌入式处理器130进行数据交互，用于记录当前地表的障碍物，对机器视觉识别处理的进一步补充，根据地表障碍物，辅助建立工作任务区。
如图2所示，该障碍识别传感器180，可根据实际需求而定，作为举例，优选可部署在室外自动工具本体的四周。
这里的电磁边界传感器170，具体为用于在光线受限的情况下辅助判断的电磁边界传感器，作为对机器视觉识别的进一步补充，根据地表预设的电磁边界，辅助建立工作任务区。
如图2所示，该电磁边界传感器170，可根据实际需求而定，作为举例，优选可部署在室外自动工具本体的四周。
基于上述方案构成的室外自动工作控制系统，在控制室外自动工具进行工作时，首先，由AI智能单元基于机器视觉图像传感器获取的室外自动工具周围图像信息、定位模块获取室外自动工具的空间位置信息以及运动传感模块获取室外自动工具的运动状态信息，在室外自动工具沿工作区域边界运行一周后，建立工作区域边界的特征坐标地图；
接着，由AI智能单元基于室外自动工具周围图像信息、空间位置信息运动状态信息，以及所建立的工作区域边界的特征坐标地图，实时计算室外自动工具相对于工作区域边界的位置关系，并在室外自动工具到达工作区域边界时， 对室外自动工具形成返回工作区域的自动工作控制模式。
以下具体说明一下，本方案控制室外自动工具完成工作区域识别以及自动在识别出的工作区域内自动完成全覆盖工作实现过程。
本方案给出的室外自动工作控制系统在控制室外自动工具进行自动工作时，主要包括建立工作区域地图和自动工作两个阶段。
(一)工作区域地图建立阶段
该阶段用于室外自动工具初始工作时，针对待工作区域建立工作区域边界的特征坐标地图。
该阶段中，本室外自动工作控制系统控制室外自动工具在工作区域自动寻找工作区域边界运行一周，通过机器视觉图像传感器和无线电定位模块和运动传感器建立工作区域边界的特征坐标地图。
具体的，控制系统通过机器视觉传感器实时采集室外自动工具周围图像，再由人工智能单元利用神经网络对采集到的图像进行识别和分割，并判断室外自动工具是否贴近指定的工作区域边界：
如未能检测到边界，控制室外自动工具按照当前行驶状态继续直行，直至识别到边界；
在识别达到边界的同时，同步采集室外自动工具中定位模块获取的定位信息中的绝对坐标点、基于室外自动工具位姿的边界方向向量、边界图像内的特征物信息，据此建立室外自动工具当前所到达边界点的特征向量。
控制系统在室外自动工具达到工作区域边界时，将控制室外自动工具调整行驶方向，使得室外自动工具自主连续的沿工作区域边界线行走，并同步自主采集一系列边界的特征向量，直到出现特征向量重复。
此时，控制系统通过嵌入式处理器对存储的坐标点和其特征向量做滤波和优化，并基于优化的数据建立工作区域边界的特征坐标地图。
另外，作为替换方案，本方案在控制室外自动工具沿工作区域边界运行一周时，除了前述的在工作区域自动寻找工作区域边界运行一周的方式外，还可通过手动遥控室外自动工具沿工作区域边界运行一周。同样的，在手动遥控室外自动工具沿工作区域边界运行一周的过程中，控制系统同步采集对应边界点的坐标点和特征向量，对采集的坐标点和其特征向量做滤波和优化，并基于优 化的数据建立工作区域边界的特征坐标地图。
作为举例，针对室外自动工具难自主行走建立地图的场景，如工作边界图像定义不清晰，工作区域为一个整体大区域的局部区域时，此情况下，可通过遥控模式辅助室外自动工具完成工作区边界特征坐标地图的建立。可由操作人员(如用户)通过遥控装置对室外自动工具进行遥控沿工作区边界行走，并在行走过程中同步采集一系列边界的特征向量，直到出现特征向量重复，对采集的坐标点和其特征向量做滤波和优化，并基于优化的数据建立工作区域边界的特征坐标地图。
进一步的，本室外自动工作控制系统在行进工作区域地图构建的过程中，系统中的人工智能单元对机器视觉图像传感器采集并输入的室外自动工具周围图像，具体基于深度学习的神经网络算法进行运算，优化，加速；在此基础上对图像中的工作区域进行识别和分割处理，以输出机器可识别的工作区域边界信息，这里的工作区域边界信息包括边界种类，方向向量及法向量，及其与设备方向的夹角，并据此信息构成对应的边界特征向量输入系统的处理单元中。
进一步的，本室外自动工作控制系统在行进工作区域地图构建的过程中，还利用单个或多个机器视觉图像传感器采集并输入的图像信息估计室外自动工具的运动过程信息。
这里的运动过程信息主要是指室外自动工具自主行走的位移信息及空间的姿态信息。
具体的，本方案通过获取相邻两帧图片之间的RT坐标系变换关系，将获取得到的多个RT相乘获取得到当前帧与原始位置之间的变换关系，即计算出室外自动工具帧间的运动变化；然后在此基础上进行迭代优化；同时，人工智能处理单元融合运动传感模块例如编码器和陀螺仪采集到的室外自动工具的速度信息、加速度信息，通过时间积分来计算自室外自动工具的位移变化，完成对室外自动工具的位姿信息及运动信息的估计。
如此，本方案行进工作区域地图构建时，通过基于神经网络的机器视觉识别后，能够更方便的提取出边界线图像的特征。同时配合，通过相邻两帧图像之间的区别，来计算出帧间的运动变化来过滤掉不必要的特征点，由此实现更高效的利用图像信息，计算出对室外自动工具的位姿信息及运动信息的估计， 从而得到室外自动工具行走的路径和方向角，使得要建立的边界线地图的边界的坐标更加完整和准确。
进一步的，由于运动位姿的估计只依赖于相邻两帧图像之间的数据(包括图像，加速度，角速度等信息)，位姿估计中的误差会累加。对此，本方案中通过无线电定位模块获取室外自动工具的定位信息，并据此来消除累计误差，提高了自动工具的定位精度。
作为举例，对于通过神经网络和图像变换的方式处理边界地图时，若出现边界线难以闭合，难以构成一个封闭的图形时，此时，可通过无线电模块辅助获得室外自动工具的高精度定位点信息，来对累计误差的漂移误差量进行实时或者间断的修正，从而实现消除这个累计误差带来的边界点坐标的偏移，这样可基于修正后的数据形成一个完整且封闭的边界线地图。
本室外自动工作控制系统在完成工作区域地图构建时，以工作区域边界点的位置坐标和机器遍历走过这边位置的方式，将各个位置的坐标点及其对应的特征向量存贮下来，生成由特征向量和位置坐标的工作区域边界的特征地图。
参见图3，给了本室外自动工作控制系统控制室外自动工具进行工作区域地图构建的一种示例流程。
由图可知，本室外自动工作控制系统控制室外自动工具针对相应的工作区域，进行工作区域地图构建时，根据控制模式采用自动或手动遥控的方式控制室外自动工具在工作区域内寻找工作区域边界。
接着，控制室外自动工具进行初始化，即控制室外自动工具原地旋转，并获取室外自动工具周围的图像信息。
这里控制室外自动工具进行原地旋转，可实现扫描获取室外自动工具周围的工作边界图像，使得室外自动工具能够以就近的方式寻找边界线，并开始沿线构建工作区域地图；
这里控制室外自动工具进行原地旋转，还可实现扫描获取室外自动工具周围的工作区域图像，当室外自动工具周围图像识别结果都是工作区域内时，判断室外自动工具处在工作区域内可以开始安全的工作。
这里控制室外自动工具进行原地旋转，还可同时校准磁力计获得更准确地方向角。
接着，根据选定的控制模式控制室外自动工具在工作区域内寻找工作区域边界移动，并基于机器视觉实时计算室外自动工具是否贴近工作区域边界。
接着，在室外自动工具到达工作区域边界时，采集该边界点的特征向量和位置坐标，并将该边界点的位置坐标及其对应的特征向量进行存贮。
接着，据此控制室外自动工具遍历走过工作区域边界一周，将各个边界位置的坐标点及其对应的特征向量存贮，据此生成由特征向量和位置坐标的工作区域边界的特征地图。
最后，根据需要可对所建立的工作区域边界的特征地图进行人机交互确认，完成最后的确认和保护。
(二)自动工作阶段
基于第一步建立的工作区域边界的特征地图，本室外自动工作控制系统控制室外自动工作工具一方面通过比对室外自动工具自身的特征坐标与工作区域边界的特征坐标地图；另一方面，通过机器视觉的深度学习的方式，对图像中的工作区域进行识别和分割处理，输出机器可识别的工作区域边界信息，帮助室外自动工具的控制系统决策是否处在指定的工作区域，即是否处于工作区域边界特征地图所限定的工作区域内。
作为举例，可通过识别室外自动工具周围图像中相应工作区域的有、无、或比例，判断室外自动工具是否在指定工作区域内。
在此基础上，可进一步融合通过神经网络计算边界线深度信息的和室外自动工具相对于边界线夹角信息,来判断室外自动工具是否在指定工作区域内工作，以提高判断的精度。
在确定室外自动工作工具处于工作区域边界特征地图所限定的工作区域内，室外自动工作控制系统将基于建立的工作区域边界的特征地图控制控制室外自动工作工具在工作区域边界特征地图所限定的工作区域内按照预定的工作模式自动行驶，即实现自动工作。
进一步的，在该过程中，室外自动工作控制系统，同样利用图像传感器的输入周围图像信息，融合运动传感模块，定位模块的绝对定位信息，实时的生成室外自动工作工具自身的特征坐标和位姿信息及运动信息的估计。进一步确 保实现室外自动工作工具在指定工作区域内自动工作，实现不跨出该指定工作区域的自主行走和按照一定运行模式和逻辑的自动工作。
参见图4，给了本室外自动工作控制系统控制室外自动工具在由特征坐标地图限定的工作区域内自动工作的一种示例流程。
本室外自动工作控制系统控制室外自动工具在由特征坐标地图限定的工作区域自动工作时，基于机器视觉实时计算室外自动工具相对于工作区域边界的位置关系，并在室外自动工具到达工作区域边界时，根据选定的自动工作模式控制室外自动工具按模式要求返回工作区域。
作为举例，这里的自动工作模式可以为沿边模式、随机模式、路径规划模式、自动回充模式、避障模式中的一种或多种。
这里需要指出的，本方案中的自动工作模式并不限于上述这几种。
针对前述给出的基于机器视觉的室外自动工作控制系统方案，以下通过相应的实例进一步来说明其实现过程及其性能。
本实例以庭院机器人为例，本实例在庭院机器人内设置相应的基于机器视觉的室外自动工作控制系统的软件程序，该软件程序在被相应的处理器运行时，能够执行前述室外自动工作控制方法的步骤。
如此部署的庭院机器人，可根据用户布置的任务的不同，通过机器视觉自动规划工作区域，首先将工作区域由深度学习算法对工作区域的进行识别并进行语义分割，将自然边界生成机器可识别的工作区域边界信息，包括边界种类，方向向量及法向量，及其与设备方向的夹角；接着，具体完成工作区域的识别和规划。
该庭院机器人在启动运行时，能够自动进行庭院养护工作，例如除草，落叶清扫，除雪，施肥，播种草籽，巡逻等庭院养护工作。
本庭院机器人在进行工作时，主要包括建图模式和自动工作模式。
本实例给出的庭院机器人可采用两种建图模式，即手动遥控建图模式和自动建图模式(参见图5)。
对于手动遥控建图模式：
在该模式下，使用有线或无线遥控器，并选择建立地图模式，遥控庭院机 器人行走在工作区域，通过采集庭院机器人行走时的周围环境图像，通过深度学习的机器视觉算法产生基于环境的多维电子信息地图，即工作区域边界的特征坐标地图。
这样庭院机器人沿工作区域边界行走一个闭合路径，即为大致的工作区域，系统根据机器视觉的结果，自动识别工作区域边界，建立工作禁区。
当然作为补充方案，最后还可通过人工在上位机上手动再次设置工作边界及工作禁区。
对于自动建图模式：
该模式下，控制庭院机器人在工作区域内自动沿指定工作区域边界工作和沿着边界线建立工作区域地图的运行模式。
本实例中庭院机器人进行自动建立工作区域地图时，首先进行庭院机器人的初始化，控制庭院机器人在工作区域内原地旋转，无边界时沿随机方向直行，寻找工作区域边界。同时，获取庭院机器人周围环境图像，并根据获取到的图像计算判断庭院机器人是否处于指定类型的工作区域内，例如：草坪，街道，雪地等。
如图5所述，其给出了本实例中庭院机器人进行自动建图的一种实现逻辑过程示例。
首先放庭院机器人(以下简称设备)在工作区域外边界周围，或从设置在地图边界的充电桩出发，设定机器进入建立地图模式。
接着，设备会开始初次建立工作区域的外边界地图。设备从起点直线行走或原地旋转，该过程通过设备上的机器视觉传感器获取庭院机器人周围环境图像，设备内的智能处理单元将根据获取到的图像计算判断庭院机器人是否处于指定类型的工作区域内，例如：草坪，街道，雪地等。若不在，则停机报警。
接着，设备内的智能处理单元识别出设备接近或即将驶出所指定的工作区域的连续边界，则发送沿边指令给控制执行单元，设备转向贴近工作区域的边界的方向，设备运行方向平行于工作区域边界方向向量，同时确保设备保证在指定类型的工作区域内工作。将地图边界的起始点1作为边界起点，直到回到该点完成工作区域的闭合。起始点为第一次找到边界的特征坐标点或者设置在工作边界上的充电桩的特征坐标点。
接着，将获取到的特征坐标点存储在设备内部，优化并建立工作地图。
最后，用户确认，保存地图，建图结束。
本实例方案在实施时，其通过基于神经网络的机器视觉识别来方便的提取出边界线图像的特征；同时通过相邻两帧图像之间的区别，来计算出帧间的运动变化，能过滤掉不必要的特征点，实现更高效的利用图像信息，计算出针对该设备的位姿信息及运动信息的估计，以获得设备行驶过的路径和方向角，使得要建立的边界线地图的边界的坐标更加完整和准确。
本庭院机器人在完成工作区域建图后，进入自动工作模式，将在建立的地图所规划的工作区域内完成全覆盖的自动工作。
这里该模式下，设备可进行沿边模式、随机模式、路径规划模式、自动回充模式、避障模式的自动工作模式。
(一)对于随机模式
在该模式下，设备在工作区域内自动随机工作，并完成工作区域的全覆盖。
参见图6和图7，设备进入自动随机工作模式时，设备获取庭院机器人周围环境图像，并根据获取到的图像计算判断设备是否处于指定类型的工作区域内；
接着，在工作区域内进行随机路径覆盖，每当设备中的机器视觉图像传感器输入设备周围环境图像后，设备内的智能控制单元判断该设备是否在指定的工作区域内；若设备内的智能控制单元识别出设备接近或即将驶出所指定的工作区域边界，则发送转向指令给设备的控制执行单元，控制设备按照相应的转向策略转向并远离工作区域的边界，使设备保证在指定类型的工作区域内工作。
本实例以随机模式为例，在该模式下，设备在工作区域内自动随机工作，并完成工作区域的全覆盖。
(二)对于路径规划模式
该模式下，设备在运行过程中，由设备中智能控制单元根据无线电定位模块提供位置信息、机器视觉传感器采集的设备周围环境图像以及识别出的边界位置信息来完成径规划工作模式下的视觉边界识别及自动工作。
本实例在该模式下，根据采用的数据不同，具有两种工作方式：视觉信号与位置信号结合的工作方式和纯视觉信号的工作方式。
(1)视觉信号与位置信号结合的工作方式。
如图8所示，在该工作方式，本设备可以通过无线电定位模块提供位置信息,融合神经网络的机器视觉识别后提取出的边界线图像的特征信息，通过相邻两帧图像之间的区别，来计算出帧间的运动变化，计算出针对该设备的位姿信息及运动信息的估计，得到设备走过的坐标和方向角。在此基础上，再进一步结合边界线的方向向量信息，使用路径规划算法，覆盖工作区域边界地图，实现工作区域全覆盖的路径规划。
(2)纯视觉信号的工作方式
如图9所示，当设备的无线电定位信号突然受到遮挡，设备将可只通过视觉进行定位导航和位姿推算，为避免位姿的累计误差，该工作方式下，将控制设备只要沿着边界的顺时针或逆时针，结合边界线的方向向量信息，走出平行于和垂直于地图边界的轨迹，最多将边界线遍历，由此来实现路径规划完成对工作区域无遗漏的覆盖。
在此工作方式下，当设备收到无线电定位模块发来的绝对位置信息后，依然可以按照视觉信号与位置信号结合的工作方式继续路径规划。
参见图10，其所示为本实例给出的设备基于机器视觉路径规划工作模式的实现流程图。
该模式下，设备进行路径规划模式工作时，设备获取庭院机器人周围环境图像，并根据获取到的图像计算判断设备是否处于指定类型的工作区域内；如不在，则进行停机报警。
接着，控制设备按照当前工作姿态直行，并同时获取庭院机器人周围环境图像。每当设备中的机器视觉图像传感器输入设备周围环境图像后，设备内的智能控制单元将识别出设备是否接近工作区域边界。
接着，设备内的智能控制单元识别出设备接近工作区域边界时，并同时计算出边界线的深度信息、方向向量和法向向量。
接着，设备内的运动控制单元控制设备转一定得角度，使得设备前进方向平行于边界线。
接着，控制设备继续直行预设距离。
接着，设备内的运动控制单元控制设备转一定得角度，使得设备前进方向 垂直于边界线。
最后，设备据此模式完成工作区域边界地图的全覆盖。
另外，根据设定，设备在完成工作区域边界地图的全覆盖后，可根据设定路径自动返回初始工作位置、充电桩等等。
(三)对于避障模式
该模式下，设备在运行过程中能够实现对动态障碍物和静态障碍物的识别和处理。
如图11所示，设备中的机器视觉传感器输入设备周围环境图像，设备中的智能控制单元根据机器视觉传感器输入设备周围环境图像识别出了障碍物，将进入到避障模式。
在设备中的智能控制单元第一次检测到障碍物，识别时间小于预设的触发时间t时，设备中的智能控制单元判断遇到了动态障碍物，发送命令给设备的运动控制单元，控制设备停止移动，直到障碍物离开继续前行。
当设备中的智能控制单元检测到障碍物并持续识别到障碍物时，识别时间大于预设的触发时间t时，设备中的智能控制单元默认遇到了静态障碍物，设备中的智能控制单元发送命令给设备的运动控制单元，控制设备尝试绕过障碍物；同时设备内智能控制单元将重新生成一个规划的路线确保工作区域地图覆盖。
由上述实例可知，本实例给以机器视觉传感器为主导的室外自动工作设备，通过机器视觉和神经网络大大简化了室外自动工作设备的部署的条件，增加的室外自动设备的在工作区域内的可靠性，提高了室外自动工作设备的工作效率。
以上显示和描述了本发明的基本原理、主要特征和本发明的优点。本行业的技术人员应该了解，本发明不受上述实施例的限制，上述实施例和说明书中描述的只是说明本发明的原理，在不脱离本发明精神和范围的前提下，本发明还会有各种变化和改进，这些变化和改进都落入要求保护的本发明范围内。本发明要求保护范围由所附的权利要求书及其等效物界定。
Claims (13)
- 基于机器视觉的室外自动工作控制系统，其特征在于，包括机器视觉图像传感器、智能控制单元、定位模块、运动传感模块，所述机器视觉图像传感器实时获取室外自动工具周围图像信息；所述定位模块获取室外自动工具的空间位置信息；所述运动传感模块获取室外自动工具的运动状态信息；所述智能控制单元基于机器视觉图像传感器获取的室外自动工具周围图像信息、定位模块获取室外自动工具的空间位置信息以及运动传感模块获取室外自动工具的运动状态信息，在室外自动工具沿工作区域边界运行一周后，建立工作区域边界的特征坐标地图；所述智能控制单元基于室外自动工具周围图像信息、空间位置信息运动状态信息，以及所建立的工作区域边界的特征坐标地图，实时计算室外自动工具相对于工作区域边界的位置关系，并在室外自动工具到达工作区域边界时，对室外自动工具形成返回工作区域的自动工作控制模式。
- 根据权利要求1所述的基于机器视觉的室外自动工作控制系统，其特征在于，所述智能控制单元根据机器视觉图像传感器采集的周围图像信息，利用神经网络对图像进行识别和分割，并判断室外自动工具是否接近指定的工作区域边界；如未能检测到边界，控制室外自动工具依据当前状态继续行驶，直到识别到边界；届时，同步的采集室外自动工具中定位模块获取到的绝对坐标点，以构建对应的边界特征向量；所述智能控制单元控制室外自动工具连续的沿边界线自主行走，并自主采集一系列边界的特征向量，直到出现特征向量重复，基于所获取到的边界坐标点以及对应的特征向量来建立工作区域边界的特征坐标地图。
- 根据权利要求1所述的基于机器视觉的室外自动工作控制系统，其特征在于，所述智能控制单元基于外部遥控信号控制室外自动工具连续的沿边界线自主行走，并自主采集一系列边界的特征向量，直到出现特征向量重复，基于所获取到的边界坐标点以及对应的特征向量来建立工作区域边界的特征坐标地图。
- 根据权利要求1所述的基于机器视觉的室外自动工作控制系统，其特 征在于，所述智能控制单元基于机器视觉图像传感器获取的室外自动工具周围图像信息估计室外自动工具的运动过程信息，所述运动过程信息包括室外自动工具自主行走的位移信息及空间姿态信息。
- 根据权利要求4所述的基于机器视觉的室外自动工作控制系统，其特征在于，所述智能控制单元通过获取相邻两帧之间的RT坐标系变换关系，将获取得到的多个RT相乘获取得到当前帧与原始位置之间的变换关系，然后进行迭代优化；同时，融合运动传感模块获取室外自动工具的运动状态信息，通过时间积分计算室外自动工具的位移，由此完成对室外自动工具位姿信息及运动位移信息的估算。
- 基于机器视觉的室外自动工作控制方法，其特征在于，包括：控制室外自动工具沿工作区域边界运行一周后，基于机器视觉所获取到的室外自动工具周围图像信息建立工作区域边界的特征坐标地图；控制室外自动工具在由特征坐标地图限定的工作区域内工作，并基于机器视觉实时计算室外自动工具相对于工作区域边界的位置关系，并在室外自动工具到达工作区域边界时，对室外自动工具形成返回工作区域的自动工作控制模式。
- 根据权利要求6所述的基于机器视觉的室外自动工作控制方法，其特征在于，所述控制方法在建立工作区域边界的特征坐标地图时，包括：控制室外自动工具动作，获取室外自动工具周围图像信息，以判断取室外自动工具是否处于指定类型的工作区域内；根据获取到的室外自动工具周围图像信息计算判断室外自动工具是否接近工作区域的边界；采集工作区域边界点的位置坐标，基于各个位置的坐标点及其对应的特征向量，生成由特征向量和位置坐标的工作区域边界的特征地图。
- 根据权利要求6所述的基于机器视觉的室外自动工作控制方法，其特征在于，所述室外自动工作控制方法通过识别室外自动工具周围图像中相应工作区域的有、无、或比例，判断室外自动工具是否在指定工作区域内。
- 根据权利要求8所述的基于机器视觉的室外自动工作控制方法，其特征在于，所述室外自动工作控制方法还融合工作区域边界线深度信息，以及室 外自动工具相对于工作区域边界线的夹角信息来判断室外自动工具是否在指定工作区域内。
- 根据权利要求6所述的基于机器视觉的室外自动工作控制方法，其特征在于，所述方法还基于获取的室外自动工具周围图像信息估计室外自动工具的运动过程信息，所述运动过程信息包括室外自动工具自主行走的位移信息及空间姿态信息。
- 根据权利要求6所述的基于机器视觉的室外自动工作控制方法，其特征在于，所述方法首先通过获取相邻两帧之间的RT坐标系变换关系，将获取得到的多个RT相乘获取得到当前帧与原始位置之间的变换关系；接着，进行迭代优化，并融合运动传感模块获取室外自动工具的运动状态信息，通过时间积分计算室外自动工具的位移，由此完成对室外自动工具位姿信息及运动位移信息的估算。
- 根据权利要求6所述的基于机器视觉的室外自动工作控制方法，其特征在于，所述控制室外自动工具返回工作区域的自动工作控制模式包括沿边模式、随机模式、路径规划模式、自动回充模式、避障模式中的一种或多种。
- 一种终端设备，设备包括处理器、存储器及存储在存储器上并可在处理器上运行的程序，其特征在于，所述程序代码由所述处理器加载并执行权利要求6-12所述的室外自动工作控制方法的步骤。
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111151219.8 | 2021-09-29 | ||
| CN202111151219.8A CN113885495A (zh) | 2021-09-29 | 2021-09-29 | 一种基于机器视觉的室外自动工作控制系统、方法及设备 |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| WO2023050545A1 true WO2023050545A1 (zh) | 2023-04-06 |
Family
ID=79007978
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| PCT/CN2021/132653 WO2023050545A1 (zh) | 2021-09-29 | 2021-11-24 | 一种基于机器视觉的室外自动工作控制系统、方法及设备 |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN113885495A (zh) |
| WO (1) | WO2023050545A1 (zh) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116088533B (zh) * | 2022-03-24 | 2023-12-19 | 未岚大陆(北京)科技有限公司 | 信息确定方法、远程终端、设备、割草机及存储介质 |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109859158A (zh) * | 2018-11-27 | 2019-06-07 | 邦鼓思电子科技（上海）有限公司 | 一种基于视觉的工作区域边界的检测系统、方法及机器设备 |
| US20200050208A1 (en) * | 2018-08-08 | 2020-02-13 | The Toro Company | Autonomous machine navigation and training using vision system |
| CN111199677A (zh) * | 2019-12-25 | 2020-05-26 | 邦鼓思电子科技（上海）有限公司 | 一种室外区域的工作地图自动建立方法，装置，存储介质及工作设备 |
| US20200167993A1 (en) * | 2018-11-28 | 2020-05-28 | Delta Electronics, Inc. | Map constructing apparatus and map constructing method |
| CN113126613A (zh) * | 2019-12-30 | 2021-07-16 | 南京德朔实业有限公司 | 智能割草系统及其自主建图方法 |
| CN113296495A (zh) * | 2020-02-19 | 2021-08-24 | 苏州宝时得电动工具有限公司 | 自移动设备的路径形成方法、装置和自动工作系统 |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR101125233B1 (ko) * | 2010-11-25 | 2012-03-21 | 재단법인대구경북과학기술원 | 융합기술기반 보안방법 및 융합기술기반 보안시스템 |
| CN106647765B (zh) * | 2017-01-13 | 2021-08-06 | 深圳拓邦股份有限公司 | 一种基于割草机器人的规划平台 |
| WO2019241923A1 (en) * | 2018-06-20 | 2019-12-26 | Lingdong Technology (Beijing) Co., Ltd | Unmanned lawn mower with autonomous driving |
| CN110347153A (zh) * | 2019-06-26 | 2019-10-18 | 深圳拓邦股份有限公司 | 一种边界识别方法、系统及移动机器人 |
| US11480973B2 (en) * | 2019-07-15 | 2022-10-25 | Deere & Company | Robotic mower boundary detection system |
-
2021
- 2021-09-29 CN CN202111151219.8A patent/CN113885495A/zh active Pending
- 2021-11-24 WO PCT/CN2021/132653 patent/WO2023050545A1/zh unknown
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20200050208A1 (en) * | 2018-08-08 | 2020-02-13 | The Toro Company | Autonomous machine navigation and training using vision system |
| CN109859158A (zh) * | 2018-11-27 | 2019-06-07 | 邦鼓思电子科技（上海）有限公司 | 一种基于视觉的工作区域边界的检测系统、方法及机器设备 |
| US20200167993A1 (en) * | 2018-11-28 | 2020-05-28 | Delta Electronics, Inc. | Map constructing apparatus and map constructing method |
| CN111199677A (zh) * | 2019-12-25 | 2020-05-26 | 邦鼓思电子科技（上海）有限公司 | 一种室外区域的工作地图自动建立方法，装置，存储介质及工作设备 |
| CN113126613A (zh) * | 2019-12-30 | 2021-07-16 | 南京德朔实业有限公司 | 智能割草系统及其自主建图方法 |
| CN113296495A (zh) * | 2020-02-19 | 2021-08-24 | 苏州宝时得电动工具有限公司 | 自移动设备的路径形成方法、装置和自动工作系统 |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113885495A (zh) | 2022-01-04 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112584697B (zh) | 使用视觉系统的自主机器导航和训练 | |
| US11845189B2 (en) | Domestic robotic system and method | |
| EP1240562B1 (en) | Autonomous multi-platform robot system | |
| US8666554B2 (en) | System and method for area coverage using sector decomposition | |
| EP2884364B1 (en) | Autonomous gardening vehicle with camera | |
| US20110046784A1 (en) | Asymmetric stereo vision system | |
| CA2428360C (en) | Autonomous multi-platform robotic system | |
| EP2296072A2 (en) | Asymmetric stereo vision system | |
| EP2336801A2 (en) | System and method for deploying portable landmarks | |
| CN107357297A (zh) | 一种扫地机器人导航系统及其导航方法 | |
| CN207164586U (zh) | 一种扫地机器人导航系统 | |
| US20230236604A1 (en) | Autonomous machine navigation using reflections from subsurface objects | |
| WO2023050545A1 (zh) | 一种基于机器视觉的室外自动工作控制系统、方法及设备 | |
| JP2004133882A (ja) | 自律性マルチプラットフォーム・ロボットシステム | |
| Wu et al. | Study of Autonomous Robotic Lawn Mower Using Multi-Sensor Fusion Based Simultaneous Localization and Mapping | |
| CN115933681A (zh) | 基于激光和视觉方案的工作区域圈定方法及室外机器人 |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| 121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 21959106 Country of ref document: EP Kind code of ref document: A1 |
|
| NENP | Non-entry into the national phase |
Ref country code: DE |