CN115035105A - Multidimensional data fusion and decision-making method for AGV trolley steering control - Google Patents

Multidimensional data fusion and decision-making method for AGV trolley steering control Download PDF

Info

Publication number
CN115035105A
CN115035105A CN202210952611.0A CN202210952611A CN115035105A CN 115035105 A CN115035105 A CN 115035105A CN 202210952611 A CN202210952611 A CN 202210952611A CN 115035105 A CN115035105 A CN 115035105A
Authority
CN
China
Prior art keywords
gradient
image
value
obtaining
agv
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202210952611.0A
Other languages
Chinese (zh)
Other versions
CN115035105B (en
Inventor
赵彦燕
袁绪彬
袁绪龙
李栓柱
贺庆壮
王诚善
徐祥琦
龙敏勇
王川
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shandong Ximanke Technology Co ltd
Original Assignee
Shandong Ximanke Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shandong Ximanke Technology Co ltd filed Critical Shandong Ximanke Technology Co ltd
Priority to CN202210952611.0A priority Critical patent/CN115035105B/en
Publication of CN115035105A publication Critical patent/CN115035105A/en
Application granted granted Critical
Publication of CN115035105B publication Critical patent/CN115035105B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/0002Inspection of images, e.g. flaw detection
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/10Segmentation; Edge detection
    • G06T7/13Edge detection
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/10Segmentation; Edge detection
    • G06T7/136Segmentation; Edge detection involving thresholding
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/70Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/762Arrangements for image or video recognition or understanding using pattern recognition or machine learning using clustering, e.g. of similar faces in social networks
    • G06V10/763Non-hierarchical techniques, e.g. based on statistics of modelling distributions
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/70Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/764Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
    • YGENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02PCLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
    • Y02P90/00Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
    • Y02P90/60Electric or hybrid propulsion means for production processes

Abstract

The invention relates to the technical field of image processing, in particular to a multidimensional data fusion and decision method for AGV trolley steering control, which comprises the steps of obtaining a binary image of a road surface image, screening boundary points and obtaining corresponding three-dimensional vectors, classifying all the three-dimensional vectors to obtain a plurality of classes, and calculating a first average gradient of each class; acquiring a clustering distance between every two categories, and performing secondary classification on all the first average gradients by using the clustering distance to obtain two gradient groups; calculating a second average gradient of each gradient group to be respectively used as a high gradient threshold and a low gradient threshold of edge detection, and carrying out edge detection on the road surface image to obtain a track line image; and acquiring visual reliability and magnetic reliability, and selecting a detection result corresponding to the larger value of the visual reliability and the magnetic reliability to control the steering of the AGV. According to the invention, steering control is more accurately carried out on the AGV through mutual assistance of visual detection and the magnetic stripes, so that the steering error is reduced.

Description

Multidimensional data fusion and decision-making method for AGV trolley steering control
Technical Field
The invention relates to the technical field of image processing, in particular to a multidimensional data fusion and decision method for AGV trolley steering control.
Background
The AGV is equipped with an electromagnetic or optical automatic navigation device, and can move along a path defined by an electromagnetic rail attached to a floor or move in accordance with a command from a computer.
Wherein electromagnetic induction guide formula AGV only obtains magnetic induction data through magnetic induction module, and the AGV dolly carries out steering control according to the magnetic signal who acquires. But because the magnetic stripe live time overlength, can appear magnetic stripe itself ageing or external force reason, and then lead to magnetism to weaken for turn to the error appears when turning to, when the accumulative error is too big, turn to the track and exceed the AGV dolly can turn to the limit when probably leading to turning to, perhaps the AGV dolly deviates from the track, can't get back to the positive rail, can't accomplish AGV dolly haulage task. Therefore, in the process of advancing of the AGV trolley, the problem that the steering error is large due to weakening of the magnetism of the magnetic stripes can occur, and the carrying task of the AGV trolley is affected.
Disclosure of Invention
In order to solve the problem that the steering error is large due to the fact that the magnetism of a magnetic strip is weakened in the process of moving the AGV trolley, the invention provides a multidimensional data fusion and decision method for controlling the steering of the AGV trolley, and the adopted technical scheme is as follows:
one embodiment of the invention provides a multidimensional data fusion and decision method for AGV trolley steering control, which comprises the following steps:
collecting a road surface image right in front of an AGV during running, wherein the road surface image comprises magnetic stripe track lines and the ground;
obtaining a binary image of the road surface image through threshold segmentation, obtaining a gradient value of each pixel point in the binary image, and screening out boundary points based on the gradient values; combining the coordinates and the gradient values of the boundary points into three-dimensional vectors corresponding to the boundary points, classifying all the three-dimensional vectors to obtain a plurality of classes, and calculating a first average gradient of each class;
obtaining curvature difference anisotropy of corresponding classes according to the tangential direction of boundary points in each class, obtaining clustering distances between the two corresponding classes based on the difference between the first average gradients of each two classes and the curvature difference anisotropy, and performing secondary classification on all the first average gradients by using the clustering distances to obtain two gradient groups;
calculating a second average gradient of each gradient group to be respectively used as a high gradient threshold and a low gradient threshold of edge detection, and carrying out edge detection on the road surface image to obtain a track line image; extracting break points of the track line image, performing curve fitting on a connected domain containing the break points to obtain a plurality of fitting curves, and acquiring visual reliability based on the distance between the fitting curves in the track line image and the length of the fitting curves;
and selecting a detection result corresponding to a larger value of the visual reliability and the magnetic reliability to control the steering of the AGV by taking the ratio of the magnetic intensity of the magnetic stripe acquired in real time to the standard magnetic intensity as the magnetic reliability at the current moment.
Preferably, the method for acquiring the binary image comprises:
and obtaining an optimal threshold value through the Otsu method, segmenting the road surface image by using the optimal threshold value, setting a pixel value larger than the optimal threshold value as a first preset value, and setting a pixel value smaller than or equal to the optimal threshold value as a second preset value to obtain the binary image.
Preferably, the screening process of the boundary points is as follows:
and taking the difference between the second preset value and the first preset value as a gradient threshold, and when the gradient value of a pixel is the gradient threshold, taking the corresponding pixel as the boundary point.
Preferably, the classifying all three-dimensional vectors to obtain a plurality of classes includes:
and performing density clustering on all three-dimensional vectors by setting the minimum cluster number and the neighborhood radius to obtain the multiple categories.
Preferably, the method for acquiring the curvature difference comprises the following steps:
and for each category, acquiring a Hessian matrix of each boundary point in the binary image, taking a feature vector corresponding to the minimum feature value of the Hessian matrix as the tangential direction of the corresponding boundary point, and taking the variance of all the boundary points in the tangential direction as the curvature difference of the corresponding category.
Preferably, the method for obtaining the clustering distance comprises the following steps:
and calculating the absolute value of the difference between the first average gradients of every two categories, and selecting the larger value of the curvature differences of the two categories to be multiplied by the absolute value of the difference, wherein the obtained result is the clustering distance.
Preferably, the method for acquiring the track line image comprises:
and taking the larger value of the two second average gradients as a high gradient threshold value and the smaller value as a low gradient threshold value, and performing double-threshold edge detection on the road surface image by using a canny edge detection operator to obtain the track line image.
Preferably, the method for extracting the break point comprises the following steps:
and obtaining lines and end points of the lines in the track line image through a connected domain extraction algorithm, removing end points on the image boundary, and taking the end points in the rest image as the break points.
Preferably, the method for acquiring the visual reliability includes:
and taking the ratio of the distance between break points corresponding to the two fitting curves to the sum of the lengths of the two fitting curves as the influence degree of the corresponding track line, and subtracting the average influence degree of all the track lines from a preset value as the visual reliability of the track line image.
The embodiment of the invention at least has the following beneficial effects:
1. the visual reliability is obtained through an image processing method, the magnetic reliability is obtained based on the magnetic strength, steering control is carried out on the detection result with higher reliability in the running process of the AGV trolley, the magnetic strip magnetism is inevitably weakened along with the increase of the using time, when the magnetic strip magnetism is weakened and unavailable, the visual reliability is inevitably greater than the magnetic reliability, steering control is carried out by using the visual detection result, the visual detection result is the result of image processing, the precision cannot be reduced along with the weakening of the magnetic strip magnetism, and the steering error caused by the weakening of the magnetism is avoided. The steering control method achieves a more accurate steering control effect by mutual assistance of visual detection and the magnetic strip, and reduces steering errors caused by weakening of magnetism of the magnetic strip.
2. The method comprises the steps of obtaining a binary image through image threshold segmentation, obtaining a high gradient threshold and a low gradient threshold based on the gradient and curvature difference of boundary points of the binary image, carrying out edge detection on a road surface image through the two thresholds to obtain a track line image, accurately extracting the edge of the track line, and improving the accuracy of steering control of the AGV based on a visual detection result.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions and advantages of the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and other drawings can be obtained by those skilled in the art without creative efforts.
FIG. 1 is a flowchart illustrating steps of a method for fusion and decision-making of multidimensional data for AGV car steering control according to an embodiment of the present invention;
FIG. 2 is an exemplary illustration of a road surface image;
FIG. 3 is another exemplary road surface image;
FIG. 4 is a graph illustrating the effect of edge detection on FIG. 2 using the default threshold of the canny operator;
FIG. 5 is a graph illustrating the effect of edge detection on FIG. 3 using the default threshold of the canny operator;
FIG. 6 is a track line image obtained by edge detection of FIG. 2 using the canny operator with the high and low gradient thresholds obtained by the present invention;
FIG. 7 is an image of the orbit line obtained by edge detection of FIG. 3 using the canny operator with the high and low gradient thresholds obtained by the present invention.
Detailed Description
To further illustrate the technical means and effects of the present invention adopted to achieve the predetermined objects, the following detailed description, structures, features and effects of a multidimensional data fusion and decision method for AGV car steering control according to the present invention are provided with reference to the accompanying drawings and preferred embodiments. In the following description, different "one embodiment" or "another embodiment" refers to not necessarily the same embodiment. Furthermore, the particular features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.
Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.
The following describes a specific scheme of a multidimensional data fusion and decision method for AGV car steering control in detail with reference to the accompanying drawings.
Referring to fig. 1, a flowchart illustrating a method for multidimensional data fusion and decision making for AGV cart steering control according to an embodiment of the present invention is shown, where the method includes the following steps:
and S001, acquiring a road surface image right in front of the AGV during running, wherein the road surface image comprises magnetic stripe track lines and the ground.
The method comprises the following specific steps:
and a gray level camera facing the traveling direction of the AGV trolley is fixed on the AGV trolley and is used for acquiring a road image of the traveling direction of the AGV trolley. The road surface image is a gray level image and comprises a magnetic stripe track line of a part to be driven on the AGV trolley route and the ground, wherein the camera is always in a working state.
Since most of the magnetic stripe tracks are black or yellow, the track lines tend to have a large gray difference with the ground, and the collected road surface images are shown in fig. 2 and 3.
S002, obtaining a binary image of the road surface image through threshold segmentation, obtaining a gradient value of each pixel point in the binary image, and screening out boundary points based on the gradient values; and combining the coordinates and the gradient values of the boundary points to form three-dimensional vectors corresponding to the boundary points, classifying all the three-dimensional vectors to obtain a plurality of classes, and calculating a first average gradient of each class.
The method comprises the following specific steps:
1. and acquiring a binary image of the road surface image.
And obtaining an optimal threshold value through an Otsu algorithm, segmenting the road surface image by using the optimal threshold value, setting the pixel value larger than the optimal threshold value as a first preset value, and setting the pixel value smaller than or equal to the optimal threshold value as a second preset value to obtain a binary image.
As an example, in the embodiment of the present invention, the first preset value is 0, and the second preset value is 1.
And acquiring an optimal threshold value of the road surface image by using an Otsu method for the acquired image, wherein the optimal threshold value is a gray value, binarizing the road surface image by using the optimal threshold value as a segmentation threshold value, setting a pixel point value which is larger than the segmentation threshold value in the road surface image as 0, and setting a pixel point value which is smaller than or equal to the segmentation threshold value in the road surface image as 1 to obtain a binary image of the road surface image.
Since the magnetic stripe is mostly black or has other low-gradation portions with respect to the ground, the portion smaller than the division threshold value is regarded as the portion of 1 in the embodiment of the present invention.
In other embodiments, assignment adjustment after image segmentation can be performed according to the difference between the colors of the magnetic stripe and the ground in a specific implementation scene.
The result of image segmentation of the road surface image through the threshold value obtained by the otus algorithm is a binary image, and after the image is segmented through the otus algorithm, the maximum variance value between two types of the segmented image can be obtained, the difference of the gray value between the two types can be represented, and the gradient at the junction between the two types can represent and evaluate the difference between the two types.
2. And screening boundary points.
And taking the difference between the second preset value and the first preset value as a gradient threshold, and when the gradient value of the pixel point is the gradient threshold, taking the corresponding pixel point as a boundary point.
Establishing a 3-to-3 sliding window, taking each pixel point in the binary image as a sliding window central point, calculating the difference value between the maximum value and the minimum value of the pixel values in the sliding window as the gradient value of the corresponding central point, and recording the gradient value of the ith pixel point as the gradient value of the corresponding central point
Figure 570145DEST_PATH_IMAGE001
. The pixel values in the binary image have only two values, which are 1 and 0 in the embodiment of the present invention, that is, the gradient threshold is 1, and when the gradient value of a pixel point is equal to 1, it is described that two pixel values, 1 and 0, are included in the sliding window, and the pixel point is a boundary point.
3. A plurality of classes are acquired and a first average gradient for each class is calculated.
And combining the coordinates and the gradient values of the boundary points to form three-dimensional vectors corresponding to the boundary points, and performing density clustering on all the three-dimensional vectors by setting the minimum cluster number and the neighborhood radius to obtain a plurality of categories.
Obtaining coordinate information of all boundary points in the binary image in the image, taking the ith pixel point as an example, and recording the coordinate of the ith pixel point as the coordinate of the ith pixel point
Figure 31214DEST_PATH_IMAGE002
(ii) a Gradient information in the gray scale image with the boundary point
Figure 883894DEST_PATH_IMAGE001
Combined into a three-dimensional vector
Figure 63203DEST_PATH_IMAGE003
And further obtaining three-dimensional vectors corresponding to all boundary points in the binary image.
And classifying all three-dimensional vectors by adopting a DBSCAN algorithm, and classifying boundary points with continuous coordinates and similar gradients into one class to obtain D classes.
As an example, when the DBSCAN algorithm is classified in the embodiment of the present invention, the minimum cluster number minport =3, the neighborhood radius r =3, and both minport and r are hyper parameters, which may be adjusted according to a specific implementation scenario.
For each class, calculating the average value of all gradients in the class as the first average gradient of the corresponding class, and recording the first average gradient of the d-th class as the first average gradient of the corresponding class
Figure 259698DEST_PATH_IMAGE004
And S003, acquiring curvature difference anisotropy of corresponding classes according to the tangential direction of boundary points in each class, acquiring clustering distances between the two corresponding classes based on the difference between the first average gradients of the two classes and the curvature difference anisotropy, and performing secondary classification on all the first average gradients by using the clustering distances to obtain two gradient groups.
The method comprises the following specific steps:
1. the curvature difference of each category is obtained.
And for each category, acquiring a Hessian matrix of each boundary point in the binary image, taking a feature vector corresponding to the minimum feature value of the Hessian matrix as the tangential direction of the corresponding boundary point, and taking the variance of all the boundary points in the tangential direction as the curvature difference of the corresponding category.
The Hessian matrix corresponding to each boundary point in the binary image is obtained and is a 2 multiplied by 2 diagonal matrix, the eigenvector corresponding to the minimum eigenvalue of the Hessian matrix is obtained and is used for expressing the direction of the boundary point on the binary image with the minimum gray value change curvature, namely the tangential direction of the boundary point on the boundary line, and then the variance value of all the boundary points in each category corresponding to the tangential direction is obtained and is used as the curvature difference.
The difference in the degree of curvature of the d-th class was recorded as
Figure 575273DEST_PATH_IMAGE005
The larger the variance value is, the more inconsistent the corresponding tangential directions of all pixel points in each category are, and the rail lines are regular lines for ensuring that the AGV can run stably, so that the categories with larger difference in curvature difference are more unlikely to be the rail lines of the AGV.
2. And calculating the clustering distance.
Two gradient thresholds are obtained by performing two classifications on all the first average gradients, and the two classifications in the embodiment of the invention adopt a k-means algorithm, so that the clustering distance is calculated at first.
And calculating the absolute value of the difference between the first average gradients of every two categories, and selecting the larger value of the curvature differences of the two categories to be multiplied by the absolute value of the difference, wherein the obtained result is the clustering distance.
As an example, the clustering distance between the d-th category and the d + 1-th category is calculated by the following formula:
Figure 595925DEST_PATH_IMAGE006
wherein the content of the first and second substances,
Figure 262530DEST_PATH_IMAGE005
indicates the difference in the curvature differences of the d-th class,
Figure 528295DEST_PATH_IMAGE007
the curvature difference of the (d + 1) th class is shown,
Figure 823010DEST_PATH_IMAGE008
presentation selection
Figure 532340DEST_PATH_IMAGE005
And
Figure 905815DEST_PATH_IMAGE007
the maximum value of (a) is,
Figure 726003DEST_PATH_IMAGE004
a first average gradient representing the d-th class,
Figure 999859DEST_PATH_IMAGE009
represents the first average gradient of the d +1 th class.
The larger the difference in curvature is, the less likely it is to be the track line of the AGV car, while the larger the difference between the first average gradients of the two classes is, the more likely it is to be two different classes, and thus the longer the corresponding clustering distance is.
3. And performing secondary classification on all the first average gradients by using the clustering distance to obtain two gradient groups.
And (3) enabling k =2 through a k-means algorithm, carrying out secondary classification on all the first average gradients based on the calculated clustering distance, and classifying the similar first average gradients into one class to obtain two gradient groups.
Step S004, calculating a second average gradient of each gradient group to be respectively used as a high gradient threshold and a low gradient threshold of edge detection, and carrying out edge detection on the road surface image to obtain a track line image; and extracting break points of the track line image, performing curve fitting on the connected domain containing the break points to obtain a plurality of fitting curves, and acquiring visual reliability based on the distance between the fitting curves in the track line image and the length of the fitting curves.
The method comprises the following specific steps:
1. an image of the track line is acquired.
The method comprises the steps of firstly denoising a road surface image through Gaussian filtering, then taking the larger value of two second average gradients as a high gradient threshold value and the smaller value as a low gradient threshold value, and carrying out double-threshold edge detection on the road surface image by using a canny edge detection operator to obtain a track line image.
If the canny operator dual threshold is not properly set, the detection of the orbit line is affected by the interference, as shown in fig. 4 and 5, and fig. 4 is a graph showing the effect of edge detection on fig. 2 by using the default threshold of the canny operator; FIG. 5 is a graph showing the effect of edge detection on FIG. 3 using the default threshold of the canny operator. The double thresholds of the canny operator are default thresholds, and the edge detection result has more interference, so that the track line cannot be accurately obtained.
Taking the larger value of the two second average gradients as a high gradient threshold value of a canny edge detection operator, taking the smaller value as a low gradient threshold value of the canny edge detection operator, and performing double-threshold edge detection on the road surface image to obtain a track line image, as shown in fig. 6 and 7, wherein fig. 6 is the track line image obtained by performing edge detection on fig. 2 by using the canny operator and the high and low gradient threshold values obtained by the method; FIG. 7 is an image of the orbit line obtained by edge detection of FIG. 3 using the canny operator with the high and low gradient thresholds obtained by the present invention.
2. And extracting the break point.
And obtaining lines and end points of the lines in the track line image through a connected domain extraction algorithm, removing the end points on the image boundary, and taking the end points in the rest image as the interruption points.
And obtaining thin-line-shaped each track line connected domain in the track line image through an 8-neighborhood connected domain extraction algorithm, obtaining end points on two sides of each connected domain in the track line image, obtaining the boundary of the track line image, removing the end points on the boundary belonging to the track line image without consideration, only obtaining end points belonging to the interior of the track line image, marking as interruption points, representing the positions of the track line interruption in vision, and obtaining the coordinates of the interruption points.
3. And acquiring visual credibility.
And taking the ratio of the distance between break points corresponding to the two fitting curves to the sum of the lengths of the two fitting curves as the influence degree of the corresponding track line, and subtracting the average influence degree of all the track lines from the preset value as the visual reliability of the track image.
The method comprises the steps of obtaining connected domains with break points, obtaining coordinates of all boundary points in the connected domains, carrying out polynomial fitting on the coordinates of all the boundary points by using a least square method to obtain corresponding fitting curves, calculating the average distance between every two fitting curves, wherein the two fitting curves with the minimum average distance belong to the same track line, namely the two fitting curves belong to the same track line originally.
For two fitting curves belonging to the same track line, the ratio of the distance between break points corresponding to the two fitting curves to the sum of the lengths of the two fitting curves is used as the influence degree of the corresponding track line, if the break distance is short and the track line is long, the influence degree is small at the moment, otherwise, the influence is large. When the influence is small, the steering result obtained by performing steering control by using the visual detection result is good, namely, the visual reliability is high.
The influence degree is in the value range of [0,1 ]]In the embodiment of the invention, the preset value is 1, the influence degree is subtracted from 1 to obtain the visual reliability, and the visual reliability of the c-th track line is recorded as
Figure 880090DEST_PATH_IMAGE010
The value range is [0,1 ]]。
It should be noted that the camera is always in a working state, the visual reliability of the image acquired in real time needs to be calculated in real time, the field of view of the camera is limited, and as the AGV travels, an interrupted point may exist in the same track line in the field of view of the camera at different times, and an interrupted point does not exist in the complete track line at different times, so that the visual reliability of the same track line may be different at different times.
And S005, selecting a detection result corresponding to the larger value of the visual reliability and the magnetic reliability to control the steering of the AGV by taking the ratio of the magnetic strength of the magnetic stripe track line acquired in real time to the standard magnetic strength as the magnetic reliability of the current moment.
The method comprises the following specific steps:
1. and acquiring the magnetic reliability of the current moment.
The magnetic induction module carried by the AGV trolley is used for acquiring and obtaining the magnetic intensity of the AGV trolley on the current running path, the ratio of the magnetic intensity of the magnetic stripe track line acquired in real time to the standard magnetic intensity is used as the magnetic reliability of the current moment, the maximum magnetic reliability is 1, the minimum magnetic reliability is 0, the smaller the magnetic reliability is, the weaker the magnetic signal acquired from the current magnetic stripe track is, and the more unreliable the result is obtained through the magnetic signal.
The magnetic reliability of the c-th magnetic stripe track line is recorded as
Figure 737931DEST_PATH_IMAGE011
Because the AGV moves all the time, the magnetic strengths detected in real time may not be the same, i.e. the magnetic reliabilities of the same magnetic stripe track line at different times may be different.
2. And controlling the steering of the AGV trolley.
After the track line image of the AGV trolley is obtained, due to the fact that the visual use scene is limited, the magnetic stripe use scene is also limited, the data of the visual use scene and the magnetic stripe use scene are combined to be used, the situation that the magnetism of the magnetic stripe AGV trolley is weakened to be obtained when the magnetic stripe AGV trolley is missing can be overcome, and therefore the AGV trolley can be helped to obtain the steering control quantity better.
So if it is
Figure 96231DEST_PATH_IMAGE012
And if the magnetic signal acquired from the magnetic stripe track is better for steering control, the magnetic signal is selected for steering control. Otherwise, the visual detection result is selected for steering control.
Wherein, the magnetic signal is utilized to carry out steering control, namely, the trolley is steered based on the magnetic signal of the ground magnetic stripe.
The process of steering control by using the visual detection result is as follows:
because the AGV trolley camera is fixed, the camera coordinate system and the world coordinate system are calibrated in advance to obtain the position of the coordinates of each pixel point in the camera relative to the real world, and because the deviation of the relative position of the running track and the trolley is not large, a communication domain which is calibrated at the lower boundary of an end point and is close to the middle position can be obtained to be used as a track to be run, a coordinate point of a curve corresponding to the communication domain of the track to be run is obtained to be used as a running position, and steering control is completed.
It should be noted that the range of the track position appearing in the camera may be labeled or changed according to the specific implementation scenario.
Further, a confidence threshold R is set when
Figure 365539DEST_PATH_IMAGE013
And is
Figure 665939DEST_PATH_IMAGE014
And at the moment, the AGV sends out an alarm and stops running, and meanwhile, the running of the part of the associated AGV is stopped or detour running is carried out.
The reliability threshold may be set according to a specific implementation scenario, and as an example, the reliability threshold R =0.5 in the embodiment of the present invention.
In summary, the embodiment of the invention collects road surface images right in front of the AGV during running, wherein the road surface images comprise magnetic stripe track lines and the ground; obtaining a binary image of the road surface image through threshold segmentation, obtaining a gradient value of each pixel point in the binary image, and screening out boundary points based on the gradient values; combining the coordinates and the gradient values of the boundary points into three-dimensional vectors corresponding to the boundary points, classifying all the three-dimensional vectors to obtain a plurality of classes, and calculating a first average gradient of each class; obtaining curvature difference anisotropy of corresponding categories according to the tangential direction of boundary points in each category, obtaining clustering distances between the two corresponding categories based on the difference between the first average gradients of each two categories and the curvature difference anisotropy, and performing secondary classification on all the first average gradients by using the clustering distances to obtain two gradient groups; calculating a second average gradient of each gradient group to be respectively used as a high gradient threshold and a low gradient threshold of edge detection, and carrying out edge detection on the road surface image to obtain a track line image; extracting break points of the track line image, performing curve fitting on a connected domain containing the break points to obtain a plurality of fitting curves, and acquiring visual reliability based on the distance between the fitting curves in the track line image and the length of the fitting curves; and selecting a detection result corresponding to a larger value of the visual reliability and the magnetic reliability to control the steering of the AGV by taking the ratio of the magnetic intensity of the magnetic stripe acquired in real time to the standard magnetic intensity as the magnetic reliability at the current moment. According to the embodiment of the invention, the AGV trolley can achieve a more accurate steering control effect through visual detection and mutual assistance of the magnetic stripes, and the steering error is reduced.
It should be noted that: the precedence order of the above embodiments of the present invention is only for description, and does not represent the merits of the embodiments. And that specific embodiments have been described above. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some embodiments, multitasking and parallel processing may also be possible or may be advantageous.
The embodiments in the present specification are described in a progressive manner, and the same or similar parts in the embodiments are referred to each other, and each embodiment focuses on the differences from the other embodiments.
The above-mentioned embodiments are only used for illustrating the technical solutions of the present application, and not for limiting the same; modifications of the technical solutions described in the foregoing embodiments, or equivalents of some technical features thereof, are not essential to the spirit of the technical solutions of the embodiments of the present application, and are all included in the scope of the present application.

Claims (9)

1. A multidimensional data fusion and decision method for AGV trolley steering control is characterized by comprising the following steps:
collecting a road surface image right in front of an AGV during running, wherein the road surface image comprises magnetic stripe track lines and the ground;
obtaining a binary image of the road surface image through threshold segmentation, obtaining a gradient value of each pixel point in the binary image, and screening out boundary points based on the gradient values; combining the coordinates and the gradient values of the boundary points into three-dimensional vectors corresponding to the boundary points, classifying all the three-dimensional vectors to obtain a plurality of classes, and calculating a first average gradient of each class;
obtaining curvature difference anisotropy of corresponding classes according to the tangential direction of boundary points in each class, obtaining clustering distances between the two corresponding classes based on the difference between the first average gradients of each two classes and the curvature difference anisotropy, and performing secondary classification on all the first average gradients by using the clustering distances to obtain two gradient groups;
calculating a second average gradient of each gradient group to be respectively used as a high gradient threshold and a low gradient threshold of edge detection, and carrying out edge detection on the road surface image to obtain a track line image; extracting break points of the track line image, performing curve fitting on a connected domain containing the break points to obtain a plurality of fitting curves, and acquiring visual reliability based on the distance between the fitting curves in the track line image and the length of the fitting curves;
and selecting a detection result corresponding to a larger value of the visual reliability and the magnetic reliability to control the steering of the AGV by taking the ratio of the magnetic intensity of the magnetic stripe acquired in real time to the standard magnetic intensity as the magnetic reliability at the current moment.
2. The AGV trolley steering control multidimensional data fusion and decision method according to claim 1, wherein the binary image obtaining method comprises the following steps:
and obtaining an optimal threshold value through the Otsu method, segmenting the road surface image by using the optimal threshold value, setting a pixel value larger than the optimal threshold value as a first preset value, and setting a pixel value smaller than or equal to the optimal threshold value as a second preset value to obtain the binary image.
3. The method for fusing and deciding the multidimensional data for AGV car steering control according to claim 2, wherein the process of screening the boundary points comprises:
and taking the difference between the second preset value and the first preset value as a gradient threshold, and when the gradient value of a pixel is the gradient threshold, taking the corresponding pixel as the boundary point.
4. The AGV car steering control multidimensional data fusion and decision method according to claim 1, wherein said classifying all three-dimensional vectors to obtain a plurality of classes comprises:
and performing density clustering on all three-dimensional vectors by setting the minimum cluster number and the neighborhood radius to obtain the multiple categories.
5. The AGV car steering control multidimensional data fusion and decision-making method according to claim 1, wherein the curvature difference obtaining method comprises:
and for each category, acquiring a Hessian matrix of each boundary point in the binary image, taking a feature vector corresponding to the minimum feature value of the Hessian matrix as the tangential direction of the corresponding boundary point, and taking the variance of all the boundary points in the tangential direction as the curvature difference of the corresponding category.
6. The AGV car steering control multidimensional data fusion and decision method according to claim 1, wherein the clustering distance obtaining method comprises:
and calculating the absolute value of the difference between the first average gradients of every two categories, and selecting the larger value of the curvature differences of the two categories to be multiplied by the absolute value of the difference, wherein the obtained result is the clustering distance.
7. The AGV trolley steering control multidimensional data fusion and decision method according to claim 1, wherein the track line image obtaining method comprises the following steps:
and taking the larger value of the two second average gradients as a high gradient threshold value and the smaller value as a low gradient threshold value, and performing double-threshold edge detection on the road surface image by using a canny edge detection operator to obtain the track line image.
8. The AGV car steering control multidimensional data fusion and decision method according to claim 1, wherein the interruption point extraction method comprises:
and obtaining lines and end points of the lines in the track line image through a connected domain extraction algorithm, removing the end points on the image boundary, and taking the end points in the rest image as the interruption points.
9. The AGV car steering control multidimensional data fusion and decision method according to claim 1, wherein the visual reliability obtaining method comprises:
and taking the ratio of the distance between break points corresponding to the two fitting curves to the sum of the lengths of the two fitting curves as the influence degree of the corresponding track line, and subtracting the average influence degree of all the track lines from a preset value as the visual reliability of the track line image.
CN202210952611.0A 2022-08-10 2022-08-10 Multidimensional data fusion and decision-making method for AGV trolley steering control Active CN115035105B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210952611.0A CN115035105B (en) 2022-08-10 2022-08-10 Multidimensional data fusion and decision-making method for AGV trolley steering control

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210952611.0A CN115035105B (en) 2022-08-10 2022-08-10 Multidimensional data fusion and decision-making method for AGV trolley steering control

Publications (2)

Publication Number Publication Date
CN115035105A true CN115035105A (en) 2022-09-09
CN115035105B CN115035105B (en) 2022-10-25

Family

ID=83131266

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210952611.0A Active CN115035105B (en) 2022-08-10 2022-08-10 Multidimensional data fusion and decision-making method for AGV trolley steering control

Country Status (1)

Country Link
CN (1) CN115035105B (en)

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH09245298A (en) * 1996-03-13 1997-09-19 Toyota Motor Corp Automatic running device for vehicle
CN106228138A (en) * 2016-07-26 2016-12-14 国网重庆市电力公司电力科学研究院 A kind of Road Detection algorithm of integration region and marginal information
CN109765905A (en) * 2019-03-01 2019-05-17 航天通用技术(北京)有限公司 A kind of omnidirectional's intelligent three-dimensional carrying control system
CN110081888A (en) * 2019-05-15 2019-08-02 华南师范大学 A kind of indoor positioning algorithms of inertial navigation and earth magnetism fusion based on confidence level
CN112801111A (en) * 2020-12-18 2021-05-14 广东工业大学 Image straight line edge point classification method and device based on gradient direction
CN112882476A (en) * 2021-01-26 2021-06-01 佛山市光华智能设备有限公司 Control method and control device for controlling AGV body steering
CN113200086A (en) * 2021-06-21 2021-08-03 盐城工学院 Intelligent vehicle steering control system and control method thereof
CN114399522A (en) * 2022-01-14 2022-04-26 东南大学 High-low threshold-based Canny operator edge detection method

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH09245298A (en) * 1996-03-13 1997-09-19 Toyota Motor Corp Automatic running device for vehicle
CN106228138A (en) * 2016-07-26 2016-12-14 国网重庆市电力公司电力科学研究院 A kind of Road Detection algorithm of integration region and marginal information
CN109765905A (en) * 2019-03-01 2019-05-17 航天通用技术(北京)有限公司 A kind of omnidirectional's intelligent three-dimensional carrying control system
CN110081888A (en) * 2019-05-15 2019-08-02 华南师范大学 A kind of indoor positioning algorithms of inertial navigation and earth magnetism fusion based on confidence level
CN112801111A (en) * 2020-12-18 2021-05-14 广东工业大学 Image straight line edge point classification method and device based on gradient direction
CN112882476A (en) * 2021-01-26 2021-06-01 佛山市光华智能设备有限公司 Control method and control device for controlling AGV body steering
CN113200086A (en) * 2021-06-21 2021-08-03 盐城工学院 Intelligent vehicle steering control system and control method thereof
CN114399522A (en) * 2022-01-14 2022-04-26 东南大学 High-low threshold-based Canny operator edge detection method

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
JIN- WOO LEE ET AL.: "A Study for AGV Steering Control and Identification using Vision System", 《ISIE 2001》 *
TI-CHUN WANG ET AL.: "AGV navigation analysis based on multi-sensor data fusion", 《PRINGER SCIENCE+BUSINESS MEDIA, LLC, PART OF SPRINGER NATURE 2018》 *
唐金君: "基于传感器信息融合的自动导引车自由路径转向研究", 《中国优秀硕士学位论文全文数据库 信息科技辑》 *

Also Published As

Publication number Publication date
CN115035105B (en) 2022-10-25

Similar Documents

Publication Publication Date Title
JP6587000B2 (en) Image processing apparatus, imaging apparatus, mobile device control system, image processing method, and program
US10943131B2 (en) Image based lane marking classification
US8379928B2 (en) Obstacle detection procedure for motor vehicle
JP5127392B2 (en) Classification boundary determination method and classification boundary determination apparatus
US9626599B2 (en) Reconfigurable clear path detection system
KR101609303B1 (en) Method to calibrate camera and apparatus therefor
WO2012014430A1 (en) Mobile body detection device and mobile body detection method
CN110930459A (en) Vanishing point extraction method, camera calibration method and storage medium
JP6702340B2 (en) Image processing device, imaging device, mobile device control system, image processing method, and program
US8730325B2 (en) Traveling lane detector
CN107909009B (en) Obstacle detection method and device based on road surface learning
WO2017145634A1 (en) Image processing device, imaging device, system for controlling mobile body, image processing method, and program
CN113239733B (en) Multi-lane line detection method
KR20180070258A (en) Method for detecting and learning of objects simultaneous during vehicle driving
US20220342427A1 (en) Information processing device, information processing method, and program
CN115923839A (en) Vehicle path planning method
Ghahremannezhad et al. Robust road region extraction in video under various illumination and weather conditions
CN115035105B (en) Multidimensional data fusion and decision-making method for AGV trolley steering control
Hernández et al. Lane marking detection using image features and line fitting model
US20130034261A1 (en) Road sign detection and tracking within field-of-view (FOV) video data
JP2002175534A (en) Method for detecting road white line
JP2002008019A (en) Railway track recognition device and rolling stock using railway track recognition device
CN105069410A (en) Unstructured road recognition method and device
Yu et al. An improved phase correlation method for stop detection of autonomous driving
EP3905113A1 (en) Camera height calculation method and image processing apparatus

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant