CN112587036A - Unmanned sweeper based on machine vision and working method thereof - Google Patents
Unmanned sweeper based on machine vision and working method thereof Download PDFInfo
- Publication number
- CN112587036A CN112587036A CN202011457241.0A CN202011457241A CN112587036A CN 112587036 A CN112587036 A CN 112587036A CN 202011457241 A CN202011457241 A CN 202011457241A CN 112587036 A CN112587036 A CN 112587036A
- Authority
- CN
- China
- Prior art keywords
- eye camera
- unmanned sweeper
- image
- unmanned
- map
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- A—HUMAN NECESSITIES
- A47—FURNITURE; DOMESTIC ARTICLES OR APPLIANCES; COFFEE MILLS; SPICE MILLS; SUCTION CLEANERS IN GENERAL
- A47L—DOMESTIC WASHING OR CLEANING; SUCTION CLEANERS IN GENERAL
- A47L11/00—Machines for cleaning floors, carpets, furniture, walls, or wall coverings
- A47L11/24—Floor-sweeping machines, motor-driven
-
- A—HUMAN NECESSITIES
- A47—FURNITURE; DOMESTIC ARTICLES OR APPLIANCES; COFFEE MILLS; SPICE MILLS; SUCTION CLEANERS IN GENERAL
- A47L—DOMESTIC WASHING OR CLEANING; SUCTION CLEANERS IN GENERAL
- A47L11/00—Machines for cleaning floors, carpets, furniture, walls, or wall coverings
- A47L11/40—Parts or details of machines not provided for in groups A47L11/02 - A47L11/38, or not restricted to one of these groups, e.g. handles, arrangements of switches, skirts, buffers, levers
- A47L11/4002—Installations of electric equipment
- A47L11/4008—Arrangements of switches, indicators or the like
-
- A—HUMAN NECESSITIES
- A47—FURNITURE; DOMESTIC ARTICLES OR APPLIANCES; COFFEE MILLS; SPICE MILLS; SUCTION CLEANERS IN GENERAL
- A47L—DOMESTIC WASHING OR CLEANING; SUCTION CLEANERS IN GENERAL
- A47L11/00—Machines for cleaning floors, carpets, furniture, walls, or wall coverings
- A47L11/40—Parts or details of machines not provided for in groups A47L11/02 - A47L11/38, or not restricted to one of these groups, e.g. handles, arrangements of switches, skirts, buffers, levers
- A47L11/4061—Steering means; Means for avoiding obstacles; Details related to the place where the driver is accommodated
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/10—Terrestrial scenes
-
- A—HUMAN NECESSITIES
- A47—FURNITURE; DOMESTIC ARTICLES OR APPLIANCES; COFFEE MILLS; SPICE MILLS; SUCTION CLEANERS IN GENERAL
- A47L—DOMESTIC WASHING OR CLEANING; SUCTION CLEANERS IN GENERAL
- A47L2201/00—Robotic cleaning machines, i.e. with automatic control of the travelling movement or the cleaning operation
-
- A—HUMAN NECESSITIES
- A47—FURNITURE; DOMESTIC ARTICLES OR APPLIANCES; COFFEE MILLS; SPICE MILLS; SUCTION CLEANERS IN GENERAL
- A47L—DOMESTIC WASHING OR CLEANING; SUCTION CLEANERS IN GENERAL
- A47L2201/00—Robotic cleaning machines, i.e. with automatic control of the travelling movement or the cleaning operation
- A47L2201/04—Automatic control of the travelling movement; Automatic obstacle detection
Abstract
An STM32 embedded chip is installed on a chassis of the unmanned sweeper body and serves as a bottom-layer hardware driving controller, an embedded industrial personal computer carrying an Intel processor is installed inside the unmanned sweeper body and serves as an upper-layer algorithm application controller, and the STM32 embedded chip and the embedded industrial personal computer complete real-time data exchange through USB serial ports; 4 direct current motor are still installed to unmanned sweeping machine organism inside. The unmanned sweeper based on machine vision and the working method thereof have the advantages of reasonable structural design, simple working method, high automation and intelligent degree, the STM32 embedded chip is adopted as a bottom hardware driving controller to be matched with an embedded industrial personal computer carrying an Intel processor to be used as an upper algorithm application controller, the unmanned sweeper body can determine the self pose in real time and reasonably plan the path based on the machine vision, and the prospect is wide.
Description
Technical Field
The invention belongs to the technical field of unmanned floor sweeping machines, and particularly relates to an unmanned floor sweeping machine based on machine vision and a working method thereof.
Background
In recent decades, the fundamental research of science and technology has been rapidly advanced, and each breakthrough of science and technology causes economic revolution and social progress, and the living standard of people is continuously improved. With the arrival of the industrial 4.0 era based on the internet of things and artificial intelligence, more and more novel intelligent products are successively released, and an unmanned sweeper is one of the novel intelligent products.
Along with the change of science and technology of the unmanned sweeper day by day, the unmanned sweeper is safer and more intelligent due to the infinite innovation, and great convenience is brought to the life of people. The image processing technology is a key technology for improving the intelligence of the unmanned sweeper. Just like the eyes of the unmanned sweeper, the camera can receive the most abundant external information, and the development of the machine vision technology greatly promotes the intelligent progress of the unmanned sweeper. The real-time property is the most basic requirement of an image processing algorithm applied to the unmanned sweeper, and the acquired image is required to be processed at a higher speed so as to obtain a desired result in real time. The sensors used in the machine vision technology are mainly laser radar and camera, and thus can be divided into two categories, laser SLAM and visual SLAM. The laser radar has the advantages of accurate measurement, namely, the angle and the distance can be accurately measured, the external interference is small, but the price is expensive, the size is large, and the laser radar cannot be applied to small-sized equipment and low-cost equipment; laser SLAM has developed relatively early and therefore the algorithm architecture is well established. Compared with a laser radar, the camera is cheaper in price, small in size, convenient to carry and capable of providing rich environment information, so that the visual SLAM gradually becomes a current research hotspot. The visual SLAM is mainly divided into a monocular SLAM, a binocular SLAM and an RGB-DSLAM, the monocular SLAM system only needs one camera, but the monocular SLAM system cannot calculate the actual depth of a scene, and the calculated camera track has a scale drift phenomenon and cannot be practically applied. The RGB-DSLAM system uses an RGB-D camera which can directly obtain the depth value of an observation point without calculating in the system, thus greatly saving the calculation amount, but because the depth value is measured by using a mode of transmitting and receiving infrared light, the RGB-DSLAM system is easily influenced by external sunlight and infrared light emitted by other equipment when used in an outdoor scene, and therefore, the RGB-D camera can only be used indoors.
Therefore, it is necessary to develop an unmanned sweeper based on machine vision and a working method thereof, which adopt a binocular SLAM system to better conform to the characteristics of human eyes, calculate the true depth of a scene through the parallax of left and right views, avoid the phenomenon of scale drift, and enable the use scene to be unlimited, so as to solve the technical problems.
Chinese patent application No. cn201821383666.x discloses a low-noise intelligent unmanned sweeper, which can effectively weaken noise generated by vibration of an air pump through a sound insulation layer arranged on the inner surface of an air pump chamber, and does not improve an instant positioning and map construction system of the unmanned sweeper.
Disclosure of Invention
The purpose of the invention is as follows: in order to overcome the defects, the invention aims to provide the unmanned sweeper based on the machine vision and the working method thereof, the structural design is reasonable, the working method is simple, the automation and the intelligent degree are high, an STM32 embedded chip is adopted as a bottom hardware driving controller to be matched with an embedded industrial personal computer carrying an Intel processor to be used as an upper-layer algorithm application controller, the unmanned sweeper body can determine the self pose in real time and plan the path reasonably based on the machine vision, and the prospect is wide.
The purpose of the invention is realized by the following technical scheme:
an STM32 embedded chip is installed on a chassis of the unmanned sweeper body and serves as a bottom hardware driving controller, an embedded industrial personal computer carrying an Intel processor is installed inside the unmanned sweeper body and serves as an upper-layer algorithm application controller, and the STM32 embedded chip and the embedded industrial personal computer complete real-time data exchange through a USB serial port; the unmanned sweeper body is internally provided with 4 direct current motors, the 4 direct current motors drive 4 corresponding wheels at the bottom of the unmanned sweeper body in an independent mode, and each direct current motor is integrally provided with a speed reducer and an encoder; 6 side surfaces of the upper side surface, the lower side surface, the left side surface, the right side surface, the front side surface and the rear side surface of the unmanned sweeper body are respectively provided with an obstacle avoidance ultrasonic sensor; the front side of the top of the unmanned sweeper body is provided with a visual assembly, the middle of the top of the unmanned sweeper body is provided with a display screen, the embedded industrial personal computer is connected with a screen port of the display screen, and the display screen is used for displaying electric quantity and the current working state; the embedded industrial personal computer is in communication connection with the voice broadcasting module and controls the voice broadcasting module to broadcast; the left side surface of the unmanned sweeper body is provided with an external charging socket, and the right side surface of the unmanned sweeper body is provided with an RJ-45 network interface and a plurality of USB interfaces; the vision component comprises a left eye camera and a right eye camera which are horizontally and symmetrically arranged in a left-right mode.
The unmanned sweeper based on machine vision has compact structural design, reasonable arrangement and small volume, adopts an STM32 embedded chip as a bottom hardware driving controller and an embedded industrial personal computer carrying an Intel processor as an upper algorithm application controller, solves the problem of slow calculation when the STM32 embedded chip is used alone, the STM32 embedded chip is only used for integrating wheeled odometer and data acquisition thereof, the driving control of a direct current motor and the functions of calculation, packaging and data transmission to the embedded industrial personal computer, the embedded industrial personal computer is used for receiving environment information sensed by an obstacle avoidance ultrasonic sensor, a left eye camera and a right eye camera, and an environment map is processed, calculated and constructed by real-time and continuous motion of an unmanned sweeper body, and simultaneously carries out real-time communication and task distribution and transmits instructions to the STM32 embedded chip, and the direct current motor is controlled, so that the unmanned sweeper body can determine the self pose in real time and reasonably plan a path.
The direct current motor is integrally provided with the speed reducer and the encoder, the encoder consists of a Hall coded disc and a Hall element, the disc-shaped Hall coded disc with distributed magnetic poles detects the angular speed and the angular displacement of a coaxial electrode in the rotating process of the direct current motor, the angular speed and the angular displacement are converted into digital pulse signals through magnetoelectric induction and transmitted to an STM32 embedded chip, so that the action path of the unmanned sweeping machine body is calculated and the embedded industrial personal computer is transmitted for processing, the speed reducer performs speed reduction processing, and the problems that the unmanned sweeping machine body is large in turning radius and insufficient in motion flexibility are solved.
The unmanned sweeper in the prior art mostly adopts a single camera or a laser radar or an RGB-D camera. Although the laser radar has accurate measurement, can accurately measure the angle and the distance, is small in interference from the outside, the laser radar has high price and large volume, and is not suitable for being applied to small-sized equipment such as an unmanned sweeper and low-cost equipment; the single camera only has one camera, but the actual depth of a scene cannot be calculated, and the calculated camera track has a scale drift phenomenon; although the RGB-D camera can directly obtain the depth value of the observation point without calculating in the system, the calculation amount is greatly saved, but the depth value is measured by using a mode of transmitting and receiving infrared light, but the depth value is easily influenced by external sunlight and infrared light emitted by other equipment.
The binocular vision system formed by the left eye camera and the right eye camera is more in line with the characteristics of human eyes, the real depth of a scene is calculated through the parallax of left and right views, the scale drift phenomenon can be avoided, and the use scene is not limited.
Further, in the unmanned sweeper based on machine vision, the obstacle avoidance ultrasonic sensor, the left eye camera and the right eye camera transmit scanning data of the surrounding environment to the embedded industrial personal computer in real time through the net ports; and the STM32 embedded chip is used for driving and controlling the direct current motor.
The invention also relates to a working method of the unmanned sweeper based on machine vision, which comprises the following steps: according to the working method, the embedded industrial personal computer receives images collected by the left-eye camera and the right-eye camera in real time, calculates the real-time orientation and the position of the unmanned sweeper body through the images collected by the left-eye camera and the right-eye camera and the surrounding obstacle information of the obstacle avoidance ultrasonic sensor, constructs a dense map through a large number of images collected by the left-eye camera and the right-eye camera, transmits an instruction to the STM32 embedded chip, and controls the direct current motor, so that the unmanned sweeper body can determine the self pose in real time and reasonably plan a path; when the images collected by the left-eye camera and the right-eye camera are displayed back to a certain previous place, the place is identified through closed-loop detection, and accumulated map point matching errors are eliminated through a mandatory constraint condition of the loop.
Further, in the working method of the robot sweeper based on machine vision, the images acquired by the left-eye camera and the right-eye camera need to be optimized, and the working method specifically comprises the following steps:
(1) histogram equalization: the embedded industrial personal computer counts histograms of original images acquired by a left-eye camera and a right-eye camera, calculates new gray levels, then corrects the new gray levels into reasonable gray levels, calculates new histograms of the acquired images and generates new images;
(2) denoising an image: carrying out mean value filtering processing on the image, carrying out weighted average calculation on the field points of the pixel points to be processed of the image by using weight coefficients, and endowing the obtained calculation result to the points until each pixel point in the image is processed;
(3) threshold segmentation: performing threshold segmentation on the image by adopting a bimodal method;
(4) image edge extraction: and (3) carrying out image edge extraction detection on the image by adopting a canny operator, and then, adopting comprehensive operation of expansion and corrosion to make the edge information of the image clearer.
After the embedded industrial personal computer acquires the original images acquired by the left-eye camera and the right-eye camera, the deviation of the acquired images and the ideal images, such as noise, the need of correcting the image positions and the like, is reduced, and the problems can cause great interference on whether the constructed dense map is accurate or not.
According to the method, the histogram equalization is carried out on the obtained original image, the background part in the operated original image is restrained, the contrast of the image is improved, and the construction of a dense map at the back is facilitated; denoising the image through mean filtering; the threshold segmentation of the image to be detected is convenient for the next processing; the image edge extraction is carried out on the image after the threshold segmentation, so that the edge information of the image is conveniently extracted, and the integrity of the edge information is the basis of the subsequent image identification.
Further, the working method of the unmanned sweeper based on the machine vision comprises the steps of constructing a dense map, sequentially comprising tracking, depth map estimation, map construction and global map optimization, and positioning and map construction are carried out on the optimized image through the four steps of tracking, depth map estimation, map construction and global map optimization.
Further, in the working method of the unmanned sweeper based on machine vision, the tracking is to perform pixel sampling on the optimized image based on a pyramid region segmentation method and create a reference key frame; the depth estimation is based on the sampled pixel points to calculate the corresponding depth values.
Further, in the working method of the unmanned sweeper based on the machine vision, the map construction combines the constant-speed motion model and the reference key frame model, and constructs a map through the depth value; and performing nonlinear global optimization on the map by the global map optimization to finally obtain the constructed dense map.
Compared with the prior art, the invention has the following beneficial effects: the intelligent unmanned sweeping machine has the advantages that the structural design is reasonable, the working method is simple, the automation and the intelligent degree are high, the STM32 embedded chip is used as a bottom hardware driving controller and matched with an embedded industrial personal computer carrying an Intel processor to serve as an upper-layer algorithm application controller, a binocular vision system consisting of a left eye camera and a right eye camera is adopted, the characteristics of human eyes are better met, the real depth of a scene is calculated through the parallax of left and right views, the scale drift phenomenon can be avoided, the use scene is not limited, the unmanned sweeping machine body can determine the self pose in real time and reasonably plan the path based on the machine vision, and the application prospect is wide.
Drawings
FIG. 1 is a block diagram of an unmanned sweeper based on machine vision according to the present invention;
in the figure: STM32 embedded chip 1, embedded industrial computer 2, direct current motor 3, keep away barrier ultrasonic sensor 4, visual component 5, left eye camera 51, right eye camera 52, display screen 6, voice broadcast module 7.
Detailed Description
The technical solutions in the embodiments of the present invention will be described clearly and completely with reference to the specific embodiments and fig. 1, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
As shown in fig. 1, the following embodiments provide an unmanned sweeper based on machine vision, including an unmanned sweeper body, where a chassis of the unmanned sweeper body is provided with an STM32 embedded chip 1 as a bottom-layer hardware drive controller, an embedded industrial personal computer 2 carrying an Intel processor is installed inside the unmanned sweeper body as an upper-layer algorithm application controller, and the STM32 embedded chip 1 and the embedded industrial personal computer 2 complete real-time data exchange through a USB serial port; 4 direct current motors 3 are further mounted inside the unmanned sweeper body, the 4 direct current motors 3 drive 4 corresponding wheels at the bottom of the unmanned sweeper body in an independent mode, and a speed reducer and an encoder are integrally assembled on each direct current motor 3; 6 side surfaces of the upper side surface, the lower side surface, the left side surface, the right side surface, the front side surface and the rear side surface of the unmanned sweeper body are respectively provided with an obstacle avoidance ultrasonic sensor 4; the front side of the top of the unmanned sweeper body is provided with a vision assembly 5, the middle of the top of the unmanned sweeper body is provided with a display screen 6, the embedded industrial personal computer 2 is connected with a screen port of the display screen 6, and the display screen 6 is used for displaying electric quantity and the current working state; the voice broadcasting module 7 is arranged on the rear side of the top of the unmanned sweeper body, and the embedded industrial personal computer 2 is in communication connection with the voice broadcasting module 7 and controls the voice broadcasting module 7 to broadcast; the left side surface of the unmanned sweeper body is provided with an external charging socket, and the right side surface of the unmanned sweeper body is provided with an RJ-45 network interface and a plurality of USB interfaces; the visual component 5 comprises a left-eye camera 51 and a right-eye camera 52, and the left-eye camera 51 and the right-eye camera 52 are horizontally and symmetrically arranged in a left-right mode.
Further, the obstacle avoidance ultrasonic sensor 4, the left eye camera 51 and the right eye camera 52 transmit scanning data of the surrounding environment to the embedded industrial personal computer 2 in real time through the network port; the STM32 embedded chip 1 is used for driving and controlling the direct current motor 3.
Examples
The working method of the unmanned sweeper based on the machine vision comprises the following steps: the embedded industrial personal computer 2 receives images collected by the left-eye camera 51 and the right-eye camera 52 in real time, calculates the real-time orientation and the position of the unmanned sweeper body through the images collected by the left-eye camera 51 and the right-eye camera 52 and the surrounding obstacle information of the obstacle avoidance ultrasonic sensor 4, constructs a dense map through a large number of images collected by the left-eye camera 51 and the right-eye camera 52, transmits an instruction to the STM32 embedded chip 1, and controls the direct current motor 3 to enable the unmanned sweeper body to determine the self pose in real time and reasonably plan a path; when the images collected by the left-eye camera 51 and the right-eye camera 52 are displayed back to a previous place, the place is identified by closed-loop detection and the accumulated map point matching errors are eliminated by the mandatory constraint condition of the loop.
The image collected by the left-eye camera 51 and the right-eye camera 52 needs to be optimized, and the method specifically includes the following steps:
(1) histogram equalization: the embedded industrial personal computer 2 counts histograms of original images collected by the left-eye camera 51 and the right-eye camera 52, calculates new gray levels, then corrects the new gray levels into reasonable gray levels, calculates new histograms of the collected images, and generates new images;
(2) denoising an image: carrying out mean value filtering processing on the image, carrying out weighted average calculation on the field points of the pixel points to be processed of the image by using weight coefficients, and endowing the obtained calculation result to the points until each pixel point in the image is processed;
(3) threshold segmentation: performing threshold segmentation on the image by adopting a bimodal method;
(4) image edge extraction: and (3) carrying out image edge extraction detection on the image by adopting a canny operator, and then, adopting comprehensive operation of expansion and corrosion to make the edge information of the image clearer.
After the embedded industrial personal computer 2 acquires the original images acquired by the left-eye camera 51 and the right-eye camera 52, the deviation between the acquired images and the ideal images, such as noise, the need of correcting the image positions and the like, is reduced, and the problems can cause great interference on whether the dense map is accurately constructed.
According to the method, the histogram equalization is carried out on the obtained original image, the background part in the operated original image is restrained, the contrast of the image is improved, and the construction of a dense map at the back is facilitated; denoising the image through mean filtering; the threshold segmentation of the image to be detected is convenient for the next processing; the image edge extraction is carried out on the image after the threshold segmentation, so that the edge information of the image is conveniently extracted, and the integrity of the edge information is the basis of the subsequent image identification.
The method comprises the steps of constructing a dense map, sequentially tracking, estimating a depth map, constructing the map and optimizing the global map, and positioning and constructing the map of the optimized image through the four steps of tracking, estimating the depth map, constructing the map and optimizing the global map.
Further, the tracking is to perform pixel sampling on the optimized image based on a pyramid region segmentation method and create a reference key frame; the depth estimation is based on the sampled pixel points to calculate the corresponding depth values.
Further, the map construction combines a constant-speed motion model and a reference key frame model, and constructs a map through a depth value; and performing nonlinear global optimization on the map by the global map optimization to finally obtain the constructed dense map.
From the above, the unmanned sweeper based on machine vision of the invention has reasonable structural design, simple working method, compact structural design, reasonable arrangement and small volume, adopts the STM32 embedded chip 1 as a bottom hardware driving controller to be matched with the embedded industrial personal computer 2 carrying an Intel processor as an upper algorithm application controller, solves the problem of slow calculation when the STM32 embedded chip 1 is used alone, the STM32 embedded chip 1 is only used for integrating wheeled odometer and data acquisition thereof, the driving control of a direct current motor and the functions of calculation, packaging and data transmission to the embedded industrial personal computer 2, the embedded industrial personal computer 2 is used for receiving environmental information sensed by the obstacle avoidance ultrasonic sensor 4, the left eye camera 51 and the right eye camera 52, and the environmental map is processed, calculated and constructed in real-time and continuous motion of the unmanned sweeper body, and simultaneously carries out real-time communication and task distribution, and the instruction is transmitted to the STM32 embedded chip 1, the direct current motor 3 is controlled, and the unmanned sweeper body can determine the self pose in real time and reasonably plan the path.
The direct current motor 3 is integrally provided with the speed reducer and the encoder, the encoder consists of a Hall coded disc and a Hall element, the disc-shaped Hall coded disc with distributed magnetic poles detects the angular speed and the angular displacement of a coaxial electrode in the rotating process of the direct current motor 3, the angular speed and the angular displacement are converted into digital pulse signals through magnetoelectric induction and transmitted to the STM32 embedded chip 1, so that the action path of the unmanned sweeping machine body is calculated and the embedded industrial personal computer is transmitted for processing, the speed reducer performs speed reduction processing, and the problems that the unmanned sweeping machine body is large in turning radius and insufficient in motion flexibility are solved.
The unmanned sweeper in the prior art mostly adopts a single camera or a laser radar or an RGB-D camera. Although the laser radar has accurate measurement, can accurately measure the angle and the distance, is small in interference from the outside, the laser radar has high price and large volume, and is not suitable for being applied to small-sized equipment such as an unmanned sweeper and low-cost equipment; the single camera only has one camera, but the actual depth of a scene cannot be calculated, and the calculated camera track has a scale drift phenomenon; although the RGB-D camera can directly obtain the depth value of the observation point without calculating in the system, the calculation amount is greatly saved, but the depth value is measured by using a mode of transmitting and receiving infrared light, but the depth value is easily influenced by external sunlight and infrared light emitted by other equipment.
The binocular vision system formed by the left eye camera 51 and the right eye camera 52 is more in line with the characteristics of human eyes, the real depth of a scene is calculated through the parallax of left and right views, the phenomenon of scale drift can be avoided, and the use scene is not limited.
The specific working method of the invention is many, and the above description is only the preferred embodiment of the invention. It should be noted that the above examples are only for illustrating the present invention, and are not intended to limit the scope of the present invention. It will be apparent to those skilled in the art that various modifications can be made without departing from the principles of the invention and these modifications are to be considered within the scope of the invention.
Claims (7)
1. An unmanned sweeper based on machine vision is characterized by comprising an unmanned sweeper body, wherein an STM32 embedded chip (1) is installed on a chassis of the unmanned sweeper body to serve as a bottom hardware driving controller, an embedded industrial personal computer (2) carrying an Intel processor is installed inside the unmanned sweeper body to serve as an upper algorithm application controller, and the STM32 embedded chip (1) and the embedded industrial personal computer (2) complete real-time data exchange through USB serial ports; the unmanned sweeper body is also internally provided with 4 direct current motors (3), the 4 direct current motors (3) drive 4 corresponding wheels at the bottom of the unmanned sweeper body in an independent mode, and each direct current motor (3) is integrally provided with a speed reducer and an encoder; 6 side surfaces of the upper side, the lower side, the left side, the right side, the front side and the rear side of the unmanned sweeper body are provided with obstacle avoidance ultrasonic sensors (4); the front side of the top of the unmanned sweeper body is provided with a visual assembly (5), the middle of the top of the unmanned sweeper body is provided with a display screen (6), the embedded industrial personal computer (2) is connected with a network port of the display screen (6), and the display screen (6) is used for displaying electric quantity and the current working state; a voice broadcasting module (7) is arranged on the rear side of the top of the unmanned sweeper body, and the embedded industrial personal computer (2) is in communication connection with the voice broadcasting module (7) and controls the voice broadcasting module (7) to broadcast; the left side surface of the unmanned sweeper body is provided with an external charging socket, and the right side surface of the unmanned sweeper body is provided with an RJ-45 network interface and a plurality of USB interfaces; the visual component (5) comprises a left eye camera (51) and a right eye camera (52), wherein the left eye camera (51) and the right eye camera (52) are horizontally and symmetrically arranged in a left-right mode.
2. The machine vision-based unmanned sweeper is characterized in that the obstacle avoidance ultrasonic sensor (4), the left eye camera (51) and the right eye camera (52) transmit scanning data of the surrounding environment to the embedded industrial personal computer (2) in real time through a network port; the STM32 embedded chip (1) is used for driving and controlling the direct current motor (3).
3. The working method of the unmanned sweeper based on the machine vision according to any one of claims 1-2, characterized by comprising the following steps: the embedded industrial personal computer (2) receives images collected by a left-eye camera (51) and a right-eye camera (52) in real time, calculates the real-time orientation and the position of the unmanned sweeper body through the images collected by the left-eye camera (51) and the right-eye camera (52) and the surrounding obstacle information of the obstacle avoidance ultrasonic sensor (4), constructs a dense map through a large number of images collected by the left-eye camera (51) and the right-eye camera (52), transmits an instruction to the STM32 embedded chip (1), and controls the direct current motor (3), so that the unmanned sweeper body determines the self pose in real time and reasonably plans a path; when the images collected by the left-eye camera (51) and the right-eye camera (52) are displayed back to a certain previous place, the place is identified through closed-loop detection, and accumulated map point matching errors are eliminated through a mandatory constraint condition of the closed loop.
4. The working method of the robot sweeper based on the machine vision according to claim 3, wherein the images collected by the left-eye camera (51) and the right-eye camera (52) need to be optimized, and the working method specifically comprises the following steps:
(1) histogram equalization: the embedded industrial personal computer (2) counts the histograms of the original images collected by the left-eye camera (51) and the right-eye camera (52), calculates a new gray level, then corrects the new gray level into a reasonable gray level, calculates a new histogram of the collected images, and generates a new image;
(2) denoising an image: carrying out mean value filtering processing on the image, carrying out weighted average calculation on the field points of the pixel points to be processed of the image by using weight coefficients, and endowing the obtained calculation result to the points until each pixel point in the image is processed;
(3) threshold segmentation: performing threshold segmentation on the image by adopting a bimodal method;
(4) image edge extraction: and (3) carrying out image edge extraction detection on the image by adopting a canny operator, and then, adopting comprehensive operation of expansion and corrosion to make the edge information of the image clearer.
5. The working method of the robot sweeper based on the machine vision according to claim 4, wherein the dense map construction sequentially comprises tracking, depth map estimation, map construction and global map optimization, and the optimized image is positioned and mapped through the four steps of tracking, depth map estimation, map construction and global map optimization.
6. The working method of the machine vision-based unmanned sweeper is characterized in that the tracking is based on pyramid region segmentation to perform pixel point sampling on the optimized image and create a reference key frame; the depth estimation is based on the sampled pixel points to calculate the corresponding depth values.
7. The method of claim 6, wherein the mapping combines a constant-velocity motion model with a reference key frame model to map by depth values; and performing nonlinear global optimization on the map by the global map optimization to finally obtain the constructed dense map.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011457241.0A CN112587036A (en) | 2020-12-10 | 2020-12-10 | Unmanned sweeper based on machine vision and working method thereof |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011457241.0A CN112587036A (en) | 2020-12-10 | 2020-12-10 | Unmanned sweeper based on machine vision and working method thereof |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112587036A true CN112587036A (en) | 2021-04-02 |
Family
ID=75192645
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011457241.0A Pending CN112587036A (en) | 2020-12-10 | 2020-12-10 | Unmanned sweeper based on machine vision and working method thereof |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112587036A (en) |
Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20070053588A1 (en) * | 2005-09-08 | 2007-03-08 | Chao-Lieh Chen | Method for retrieving original intact characteristics of heavily polluted images and its image processing |
| CN103500341A (en) * | 2013-09-16 | 2014-01-08 | 安徽工程大学 | Recognition device used for road signboard |
| CN105411490A (en) * | 2015-10-26 | 2016-03-23 | 曾彦平 | Real-time positioning method of mobile robot and mobile robot |
| CN107506760A (en) * | 2017-08-04 | 2017-12-22 | 西南大学 | Traffic signals detection method and system based on GPS location and visual pattern processing |
| CN108520208A (en) * | 2018-03-23 | 2018-09-11 | 四川意高汇智科技有限公司 | Localize face recognition method |
| CN109230057A (en) * | 2018-07-26 | 2019-01-18 | 佛山市三水区希望火炬教育科技有限公司 | A kind of rubbish divides the small bodyguard robot of Chen Huanbao |
| CN110063694A (en) * | 2019-04-28 | 2019-07-30 | 彭春生 | A kind of binocular sweeping robot and working method |
| CN110264459A (en) * | 2019-06-24 | 2019-09-20 | 江苏开放大学（江苏城市职业学院） | A kind of interstices of soil characteristics information extraction method |
| CN110640741A (en) * | 2018-11-07 | 2020-01-03 | 宁波赛朗科技有限公司 | Grabbing industrial robot with regular-shaped workpiece matching function |
| CN111127402A (en) * | 2019-12-06 | 2020-05-08 | 东莞理工学院 | Visual detection method for welding quality of robot |
| CN111664842A (en) * | 2020-05-07 | 2020-09-15 | 苏州品坤智能科技有限公司 | Instant positioning and map building system of unmanned sweeper |
-
2020
- 2020-12-10 CN CN202011457241.0A patent/CN112587036A/en active Pending
Patent Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20070053588A1 (en) * | 2005-09-08 | 2007-03-08 | Chao-Lieh Chen | Method for retrieving original intact characteristics of heavily polluted images and its image processing |
| CN103500341A (en) * | 2013-09-16 | 2014-01-08 | 安徽工程大学 | Recognition device used for road signboard |
| CN105411490A (en) * | 2015-10-26 | 2016-03-23 | 曾彦平 | Real-time positioning method of mobile robot and mobile robot |
| CN107506760A (en) * | 2017-08-04 | 2017-12-22 | 西南大学 | Traffic signals detection method and system based on GPS location and visual pattern processing |
| CN108520208A (en) * | 2018-03-23 | 2018-09-11 | 四川意高汇智科技有限公司 | Localize face recognition method |
| CN109230057A (en) * | 2018-07-26 | 2019-01-18 | 佛山市三水区希望火炬教育科技有限公司 | A kind of rubbish divides the small bodyguard robot of Chen Huanbao |
| CN110640741A (en) * | 2018-11-07 | 2020-01-03 | 宁波赛朗科技有限公司 | Grabbing industrial robot with regular-shaped workpiece matching function |
| CN110063694A (en) * | 2019-04-28 | 2019-07-30 | 彭春生 | A kind of binocular sweeping robot and working method |
| CN110264459A (en) * | 2019-06-24 | 2019-09-20 | 江苏开放大学（江苏城市职业学院） | A kind of interstices of soil characteristics information extraction method |
| CN111127402A (en) * | 2019-12-06 | 2020-05-08 | 东莞理工学院 | Visual detection method for welding quality of robot |
| CN111664842A (en) * | 2020-05-07 | 2020-09-15 | 苏州品坤智能科技有限公司 | Instant positioning and map building system of unmanned sweeper |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN105411490B (en) | The real-time location method and mobile robot of mobile robot | |
| CN109900280B (en) | Autonomous navigation-based livestock and poultry information perception robot and map construction method | |
| CN103413313B (en) | The binocular vision navigation system of electrically-based robot and method | |
| CN205219101U (en) | Service robot of family | |
| WO2015024407A1 (en) | Power robot based binocular vision navigation system and method based on | |
| CN101976079B (en) | Intelligent navigation control system and method | |
| CN205905026U (en) | Robot system based on two mesh stereovisions | |
| CN104238566B (en) | Electronic circuit is with inspection robot control system based on image recognition | |
| CN108536145A (en) | A kind of robot system intelligently followed using machine vision and operation method | |
| Wulf et al. | Colored 2D maps for robot navigation with 3D sensor data | |
| CN110928301A (en) | Method, device and medium for detecting tiny obstacles | |
| CN110160543A (en) | The robot of positioning and map structuring in real time | |
| CN105225482A (en) | Based on vehicle detecting system and the method for binocular stereo vision | |
| CN109934108B (en) | Multi-target and multi-type vehicle detection and distance measurement system and implementation method | |
| CN103826105A (en) | Video tracking system and realizing method based on machine vision technology | |
| CN209154263U (en) | A kind of automatic obstacle avoiding intelligent wheel chair based on binocular vision | |
| CN103901895A (en) | Target positioning method based on unscented FastSLAM algorithm and matching optimization and robot | |
| CN107491077A (en) | A kind of indoor floor-sweeping machine device people based on cell phone application control | |
| CN111664842A (en) | Instant positioning and map building system of unmanned sweeper | |
| CN111258311A (en) | Obstacle avoidance method of underground mobile robot based on intelligent vision | |
| CN106934402A (en) | Indoor moving video tracking positions auxiliary shooting method and device | |
| CN111823212A (en) | Garbage bottle cleaning and picking robot and control method | |
| CN107647828A (en) | The sweeping robot of fish-eye camera is installed | |
| CN113031597A (en) | Autonomous obstacle avoidance method based on deep learning and stereoscopic vision | |
| CN110610650A (en) | Point cloud semantic map construction method based on deep learning and depth camera |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20210402 |