CN109650291B

CN109650291B - Vision-based forklift AGV high-precision positioning system and method

Info

Publication number: CN109650291B
Application number: CN201811573543.7A
Authority: CN
Inventors: 叶国云; 田丰; 郑振华; 傅敏
Original assignee: Ningbo Ruyi JSCL
Current assignee: Ningbo Ruyi JSCL
Priority date: 2018-12-21
Filing date: 2018-12-21
Publication date: 2021-07-13
Anticipated expiration: 2038-12-21
Also published as: CN109650291A

Abstract

The invention discloses a vision-based high-precision positioning system for an AGV (automatic guided vehicle), which is used for solving the problems of higher cost and larger system error in identifying the position of the AGV in the prior art and comprises the AGV, a light source, an imaging system and a plurality of image sensors, wherein the light source, the imaging system and the image sensors are arranged at the bottom of the AGV; a plurality of light sources for emitting light having a wavelength within a predetermined range; the imaging system is used for imaging the working pavement of the forklift AGV and generating corresponding image information; the image sensors are used for conducting preset visual feature extraction on the generated image information according to preset sampling time intervals through a preset feature extraction algorithm, calculating the preset visual features according to a preset displacement algorithm, and obtaining displacement information of the current forklift AGV. The system has low cost; the positioning information is acquired based on the image in a non-contact way, so that the reliability of the system can be improved; based on the assistance of the manual mark, the system can work in a very large working space, and brings convenience for designing an AGV path.

Description

Vision-based forklift AGV high-precision positioning system and method

Technical Field

The invention relates to the technical field of image processing and sensors, in particular to a vision-based system and a vision-based method for high-precision positioning of an AGV (automatic guided vehicle).

Background

Agvs (automated Guided vehicles) are transportation vehicles equipped with an electromagnetic or optical automatic guide device, capable of traveling along a predetermined guide path, having safety protection and various transfer functions, and generally use a battery as a power source thereof without requiring a driver for a transportation vehicle in industrial applications. In order to achieve the automatic guidance, the AGV needs to detect its own position in real time and send it to an upper computer or a server. Currently, the positioning methods applicable to AGVs include several typical methods based on magnetic stripes, lasers, and images.

According to the magnetic stripe-based method, the AGV is guided to run by paving a strip containing a magnetic material on the ground, the AGV detects a magnetic field through a Hall device or other components, so that the relative position of the AGV and the magnetic stripe is calculated, and then the AGV is controlled to run along the magnetic stripe without deviation by using a corresponding control algorithm.

Laser-based methods currently exist in two ways: a laser reflecting plate with accurate position is installed around a driving path, a laser scanner emits laser beams, the laser beams reflected by the reflecting plate are collected at the same time, and positioning of an AGV is achieved through continuous triangular geometric operation. One method is to autonomously sense the environment through a laser ranging sensor, autonomously construct a real-time contour map, dynamically adjust the real-time contour map, scan the whole environment once, and determine the current position of the vehicle body in real time through matching the contour once again. There are two approaches to the reflector plate: the requirements on the angle and the position precision of the laying of the reflecting plate are high; requiring the user to have general programming capabilities and operating functions; the path change requires the re-laying of the reflective plate. Scheme without reflector: the system has the capability of sensing the environment and can realize the control of the system; the path is simple to change; there are no other auxiliary facilities except the vehicle body itself. No large area glass windows, stainless steel plates or metal pipes are required in the working area in any method adopting laser.

There are also several ways of image-based methods: one is based on a strip type, the AGV is guided by identifying a color strip laid on the ground, the AGV is closer to a magnetic strip type method in a control method, the other is based on a two-dimensional code type, a two-dimensional code pattern is arranged on the ground at certain intervals, the two-dimensional code pattern can carry position and path information, and when the AGV passes through the two-dimensional code under the assistance of a code disc, the identification code value determines the current position and makes path selection. These methods require great modification and change of the ground, and when the business requirement changes, the layout modification needs to be costly, and in addition, patterns laid on the ground are easy to be stained after being used for a period of time, so that the identification is difficult, which is an inherent defect of the methods.

In summary, a high-precision positioning system and method for an AGV of a forklift with high reliability, low cost and small error are needed to be designed to solve the above problems.

Disclosure of Invention

In order to solve the problems in the background art, the invention provides a vision-based high-precision positioning system and method for an AGV (automatic guided vehicle).

In order to achieve the above purpose, the invention adopts the following technical scheme,

the vision-based high-precision positioning system for the AGV of the forklift comprises the forklift, a light source arranged at the bottom of the forklift, an imaging system and a plurality of image sensors;

a plurality of light sources for emitting light having a wavelength within a predetermined range;

the imaging system is used for imaging the working pavement of the forklift AGV and generating corresponding image information;

the image sensors are used for conducting preset visual feature extraction on the generated image information according to preset sampling time intervals through a preset feature extraction algorithm, calculating the preset visual features according to a preset displacement algorithm, and obtaining displacement information of the current forklift AGV.

Further, the light source is an infrared light source.

Further, the imaging system includes:

the automatic focusing module is used for automatically focusing the working road surface when the working road surface of the forklift AGV is imaged;

and the image generation module is used for generating corresponding image information after automatic focusing.

Further, the plurality of image sensors includes:

the characteristic extraction unit is used for extracting preset visual characteristics in the image information generated by the imaging system through a preset characteristic extraction algorithm;

the characteristic matching unit is used for matching preset visual characteristics of the image information acquired by two adjacent preset sampling time interval points;

and the displacement calculation unit is used for substituting the preset visual feature matching results of the image information acquired at two adjacent preset sampling time interval points into a preset least square algorithm to calculate the displacement information of the current forklift AGV.

Further, still include:

the auxiliary calibration module comprises a plurality of preset key point paving Datamatrix patterns and is used for correcting displacement information calculated by the current forklift AGV at the preset key points through the absolute positions of the preset key point paving Datamatrix patterns.

A vision-based forklift AGV high-precision positioning method comprises the following steps:

s1: emitting light with a wavelength within a preset range through a plurality of light sources arranged at the bottom of the forklift;

s2: imaging the working pavement of the AGV by arranging an imaging system at the bottom of the AGV, and generating corresponding image information;

s3: through a plurality of image sensors, the generated image information is subjected to preset visual feature extraction according to a preset sampling time interval through a preset feature extraction algorithm, the preset visual feature is calculated according to a preset displacement algorithm, and the current displacement information of the AGV is obtained.

Further, step S2 includes:

s21, automatically focusing the working road surface of the forklift AGV when imaging the working road surface through the automatic focusing module of the imaging system;

s22, the corresponding image information is generated after the automatic focusing.

Further, step S3 includes:

s31: extracting preset visual features in image information generated by an imaging system through a preset feature extraction algorithm;

s32: matching preset visual features of image information acquired at two adjacent preset sampling time interval points;

s33: and substituting the preset visual characteristic matching results of the image information acquired at two adjacent preset sampling time interval points into a preset least square algorithm to calculate the displacement information of the current forklift AGV.

Further, the method also comprises the following steps:

and correcting the displacement information calculated by the current AGV at the preset key point through the absolute position of the Datamatrix pattern laid at the preset key points.

The invention has the following advantages:

(1) compared with the traditional forklift AGV laser positioning method, the vision-based forklift AGV high-precision positioning system is low in cost;

(2) the vision-based forklift AGV high-precision positioning system acquires positioning information based on images; the non-contact positioning system increases the reliability of the positioning system;

(3) this fork truck AGV high accuracy location's based on vision system still is provided with supplementary calibration module, based on artifical mark is supplementary to calibrate the fork truck AGV that calculates, can work at very big workspace, offers convenience for designing the AGV route.

Drawings

FIG. 1 is a schematic diagram of the system installation for high precision positioning of an AGV of a vision-based forklift truck according to the present invention;

FIG. 2 is a schematic diagram of a system image sensor for high-precision positioning of an AGV (automatic guided vehicle) of a vision-based forklift truck according to the invention for processing preset visual characteristics;

FIG. 3 is a Datamatrix pattern laid out at a plurality of preset key points of the vision based AGV high precision positioning system;

FIG. 4 is a first flowchart of a method for high-precision positioning of an AGV of a vision-based forklift according to the present invention;

FIG. 5 is a flowchart of a method for high-precision positioning of an AGV (automatic guided vehicle) of a vision-based forklift according to the invention.

In the figure: 1. a light source; 2. an imaging system; 3. image sensor array

Detailed Description

The following are specific embodiments of the present invention and are further described with reference to the drawings, but the present invention is not limited to these embodiments.

Example one

This embodiment provides a system for high-precision positioning of a vision-based forklift AGV, as shown in fig. 1 to 3, the system includes:

the system comprises a forklift, a light source arranged at the bottom of the forklift, an imaging system and a plurality of image sensors;

Further, the light source is an infrared light source.

Wherein under the light source is supplementary, imaging system images the working road surface that fork truck AGV traveles, then predetermines the processing and the extraction of visual characteristic through a plurality of image sensor to imaging system formation of image, and then judges the adjacent matching of predetermineeing the visual characteristic of predetermineeing of sampling time interval point to confirm the displacement of current fork truck AGV's body, realize carrying out fork truck AGV's high accuracy location based on the vision.

The infrared light source adopts an infrared light source LED, and aims to select a light source with a wavelength which is greatly different from that of the working environment of the AGV;

the imaging system that this embodiment provided wherein adopts wide-angle lens combination to form, has adopted the anti-visible light of PMMA material to pass through infrared filter wherein, combines infrared light source, can avoid the interference of fork truck AGV operational environment light.

Further, the imaging system includes:

wherein imaging system need form an image to fork truck AGV's working road surface, utilizes the auto focus module, can avoid the working road surface height not level, causes the fuzzy problem of formation of image.

The plurality of image sensors are arranged according to a preset array.

Wherein image sensor adopts the global exposure mode, can realize that fork truck AGV also can the clear formation of image when high motion, can not produce the jelly effect.

Further, the plurality of image sensors provided in this embodiment are arranged according to a preset array, and the resolution and the arrangement mode of the image sensors are matched with the working speed of the forklift AGV; at the same resolution, the greater the operating speed of the forklift AGV, the larger the array of multiple image sensors.

Further, the plurality of image sensors includes:

The feature extraction unit extracts preset visual features of image information generated by the imaging system by using a preset feature extraction algorithm, and further, the preset feature extraction algorithm provided in this embodiment is: fast image feature extraction algorithm, as shown in fig. 2:

the Fast image feature extraction algorithm principle is that a pixel point P is selected from a picture, the pixel value of the pixel point P is set to be Ip, and circumferential pixel points are numbered on the circumference in the order from 1 to 16 in the clockwise direction.

If the brightness of N continuous pixels on the circumference is larger than the sum of the pixel value Ip of the center pixel point P and the threshold t, or smaller than the sum of the pixel value Ip of the center pixel and the threshold, the center pixel point P is called an angular point, i.e., a feature point.

The specific algorithm is as follows:

1. setting a threshold value: for comparing whether the difference between the surrounding pixel points and the candidate points is large enough. The threshold value selected in this embodiment is t;

2. constructing a moving window: the area with radius of 3 and about 16 pixels is designed in the program and compared with the selected pixel points

3. And comparing the candidate pixel points with the constructed surrounding area: the algorithm examines the pixels at the four

positions

1, 9, 5 and 13 using the position method in the figure, first detecting position 1 and position 9, and then detecting position 5 and position 13 if they are both darker than the threshold or brighter than the threshold. If the pixel point P is an angular point, at least 3 of the four pixel points should be greater than Ip + t or less than Ip + t, because if the pixel point P is an angular point, the part exceeding three quarters of a circle should satisfy the determination condition. If yes, detecting all points in the circle; if not, discard directly.

4. And carrying out non-maximum suppression on the angular points to obtain angular point output.

Through the feature extraction unit, there are many preset visual feature points in the image information generated by the imaging system, so that the method of machine learning and non-maximum suppression is further adopted in the embodiment to perform de-coarsening and refinement on the extracted multiple preset visual feature points, wherein the specific method is as follows:

its score function, V, is calculated for each detected feature point. V is the sum of p and the absolute deviation of its surrounding 16 pixels. Two adjacent feature points are considered and their V values are compared. Points with lower V values will be culled.

Therefore, the accurate preset visual feature point can be obtained.

Furthermore, the preset visual features of the image information acquired at two adjacent preset sampling time interval points are matched through a feature matching unit, namely the feature points extracted by the image array acquired by the sensor array at the current sampling time are matched with the feature points acquired at the next sampling time;

and then, substituting the preset visual feature matching results of the image information acquired at two adjacent preset sampling time interval points into a preset least square algorithm through a displacement calculation unit to calculate the displacement information of the current forklift AGV.

Further, still include:

the auxiliary calibration module comprises a plurality of preset key point paving Datamatrix patterns and is used for correcting displacement information calculated by the current forklift AGV at the preset key points through the absolute positions of the preset key point paving Datamatrix patterns; as shown in FIG. 3, the accumulated error in the positioning of the AGV may be corrected. Thereby expanding the working space of the AGV.

The vision-based high-precision positioning system for the AGV of the forklift is low in cost; acquiring positioning information based on the image; the non-contact positioning system increases the reliability of the positioning system; still be provided with supplementary calibration module, supplementary fork truck AGV based on artifical sign calibrates the fork truck AGV that calculates, can work at very big workspace, offers convenience for designing the AGV route.

Example two

The embodiment provides a method for high-precision positioning of an AGV (automatic guided vehicle) based on vision, as shown in fig. 4 to 5, the method includes:

Further, step S2 includes:

Further, step S3 includes:

Further, the method also comprises the following steps:

The method comprises the steps of firstly imaging a working pavement of the AGV through a light source, an imaging system and an image sensor array which are configured at the bottom of the forklift, extracting visual features in an image, and realizing high-precision positioning of the AGV of the forklift based on the visual features.

The wavelength of the main components of the light source is greatly different from the wavelength of the main components of light in the AGV working environment, so that the interference of the ambient light can be avoided in the imaging process by matching with the image sensor.

Then, the light rays are converged by an imaging system of the optical system, so that the working road surface can be clearly imaged on the image sensor array, and subsystems such as automatic focusing and the like can be added if necessary.

The resolution and the arrangement mode of the image sensor array are matched with the working speed of the AGV, and under the same resolution, the larger the working speed of the AGV is, the larger the array is.

The positioning is realized by extracting visual features from images formed by a working road surface of a forklift AGV, the adopted features are composed of natural features, namely road surface preset visual features, and artificial features, namely Datamatrix patterns are laid at a plurality of preset key points, the natural features are used for realizing displacement estimation in a short time, and the artificial features are used for eliminating errors and marking working paths.

The vision-based high-precision positioning method for the AGV of the forklift is adopted, so that the cost is low; acquiring positioning information based on the image; the non-contact positioning system increases the reliability of the positioning system; still be provided with supplementary calibration module, supplementary fork truck AGV based on artifical sign calibrates the fork truck AGV that calculates, can work at very big workspace, offers convenience for designing the AGV route.

The specific embodiments described herein are merely illustrative of the spirit of the invention. Various modifications or additions may be made to the described embodiments or alternatives may be employed by those skilled in the art without departing from the spirit or ambit of the invention as defined in the appended claims.

Claims

1. System of fork truck AGV high accuracy location based on vision, including fork truck, its characterized in that includes: the system comprises a light source, an imaging system and a plurality of image sensors, wherein the light source, the imaging system and the image sensors are arranged at the bottom of the forklift;

the system comprises a plurality of image sensors, a data processing module and a data processing module, wherein the image sensors are used for extracting preset visual features of generated image information according to a preset sampling time interval through a preset feature extraction algorithm, calculating the preset visual features according to a preset displacement algorithm and acquiring displacement information of the current forklift AGV;

the image sensor includes:

the system comprises a feature extraction unit, a feature extraction unit and a feature extraction unit, wherein the feature extraction unit is used for extracting preset visual features in image information generated by an imaging system through a preset feature extraction algorithm, and the preset feature extraction algorithm comprises a Fast image feature extraction algorithm, a machine learning method and a non-maximum value inhibition method;

the displacement calculation unit is used for substituting the preset visual feature matching results of the image information acquired at two adjacent preset sampling time interval points into a preset least square algorithm to calculate the displacement information of the current forklift AGV;

2. The vision-based system for high-precision positioning of a forklift AGV as claimed in claim 1, wherein the light source is an infrared light source.

3. The vision-based system for high precision positioning of a forklift AGV according to claim 2, characterised in that the imaging system comprises:

4. The vision-based system for high-precision positioning of a forklift AGV as recited in claim 1, wherein the plurality of image sensors are arranged in a predetermined array.

5. A vision-based forklift AGV high-precision positioning method is characterized by comprising the following steps:

s3: through a plurality of image sensors, performing preset visual feature extraction on the generated image information according to a preset sampling time interval through a preset feature extraction algorithm, and calculating the preset visual features according to a preset displacement algorithm to obtain the displacement information of the current forklift AGV;

the step S3 includes:

s33: substituting preset visual characteristic matching results of image information acquired at two adjacent preset sampling time interval points into a preset least square algorithm to calculate displacement information of the current forklift AGV;

further comprising the steps of: and correcting the displacement information calculated by the current AGV at the preset key point through the absolute position of the Datamatrix pattern laid at the preset key points.

6. The method for high accuracy positioning of a vision based forklift AGV according to claim 5, wherein step S2 includes: