CN110097494B - Fourier-Mellin transform-based cargo positioning method - Google Patents
Fourier-Mellin transform-based cargo positioning method Download PDFInfo
- Publication number
- CN110097494B CN110097494B CN201910345255.4A CN201910345255A CN110097494B CN 110097494 B CN110097494 B CN 110097494B CN 201910345255 A CN201910345255 A CN 201910345255A CN 110097494 B CN110097494 B CN 110097494B
- Authority
- CN
- China
- Prior art keywords
- image
- goods
- current
- fourier
- characteristic image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 45
- 230000009466 transformation Effects 0.000 claims abstract description 29
- PXFBZOLANLWPMH-UHFFFAOYSA-N 16-Epiaffinine Natural products C1C(C2=CC=CC=C2N2)=C2C(=O)CC2C(=CC)CN(C)C1C2CO PXFBZOLANLWPMH-UHFFFAOYSA-N 0.000 claims abstract description 21
- 230000007246 mechanism Effects 0.000 claims abstract description 20
- 239000011159 matrix material Substances 0.000 claims abstract description 10
- 230000009191 jumping Effects 0.000 claims abstract description 4
- 238000001228 spectrum Methods 0.000 claims description 30
- 230000001131 transforming effect Effects 0.000 claims description 10
- 238000006243 chemical reaction Methods 0.000 claims description 4
- 230000008859 change Effects 0.000 description 4
- 230000006870 function Effects 0.000 description 4
- 238000005299 abrasion Methods 0.000 description 3
- 238000001914 filtration Methods 0.000 description 2
- 241000531116 Blitum bonus-henricus Species 0.000 description 1
- 235000008645 Chenopodium bonus henricus Nutrition 0.000 description 1
- 101100001671 Emericella variicolor andF gene Proteins 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 230000008569 process Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/08—Logistics, e.g. warehousing, loading or distribution; Inventory or stock management
- G06Q10/083—Shipping
- G06Q10/0833—Tracking
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/14—Transformations for image registration, e.g. adjusting or mapping for alignment of images
- G06T3/147—Transformations for image registration, e.g. adjusting or mapping for alignment of images using affine transformations
Landscapes
- Business, Economics & Management (AREA)
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Economics (AREA)
- General Physics & Mathematics (AREA)
- Physics & Mathematics (AREA)
- Quality & Reliability (AREA)
- Operations Research (AREA)
- Marketing (AREA)
- Strategic Management (AREA)
- Tourism & Hospitality (AREA)
- Human Resources & Organizations (AREA)
- General Business, Economics & Management (AREA)
- Entrepreneurship & Innovation (AREA)
- Development Economics (AREA)
- Image Analysis (AREA)
- Image Processing (AREA)
Abstract
The invention discloses a cargo positioning method based on Fourier-Mellin transform, which comprises the following steps: establishing an absolute coordinate system, setting a calibration point, and identifying the position information of a positioning goods shelf of the calibration point; step two, installing a camera at a light-shading position; establishing an image feature library, extracting a shot specific frame image of the goods in real time as a feature image of a corresponding goods position, obtaining an affine transformation matrix through the position information of the goods shelf, obtaining a new feature image through affine transformation, and recording the new feature image as a goods feature image library; acquiring a cargo image shot at the current fetching structure position, registering the image based on Fourier-Mellin transformation, and calculating the pose relation of the cargo and the fetching mechanism; step five, updating the characteristic graph library, and jumping to the step four to perform subsequent image registration work; the method realizes high-precision positioning of various goods shelves and goods, updates the calibrated image information in time, and has wide application range and convenient operation.
Description
Technical Field
The invention relates to the technical field of mobile robots, in particular to a Fourier-Mellin transform-based mobile robot cargo positioning method.
Background
In the field of logistics storage, more and more robots are used for the handling and sorting of goods. The positioning of objects and robots using machine vision is a hot spot in the field of mobile robots at present. With the development of robots, application scenes of the robots are increasingly wide.
In this industrial field, there is an important process of grasping a specific cargo on a shelf, and only if the cargo is accurately positioned, the cargo can be correctly grasped. At present, the following positioning modes are mainly adopted for goods: (1) Goods are placed on the goods shelf according to a certain rule, and the goods are positioned by controlling the relative position of the robot and the goods shelf, so that the method excessively depends on the positioning and control precision of the robot, and when the control has deviation, goods taking errors can be caused; (2) Depending on the customized goods tray, the tray larger than goods is often used, and the fault tolerance rate of goods taking is improved through a larger area, but obviously, the utilization rate of the goods shelf is reduced by the method, and the cost is higher; (3) Additional marks are added to goods, such as two-dimensional codes, but some goods cannot be marked or the recognition rate is low after the marks are added, so that the application scene of the method is limited; the market needs a cargo positioning method which does not need to perform additional processing on the cargo, has a wide application range and is convenient to operate, and the invention is used for solving the problem.
Disclosure of Invention
In order to solve the defects of the prior art, the invention aims to provide a Fourier-Mellin transform-based mobile robot goods positioning method, which is characterized in that a goods shelf is marked by using a mark point on the ground or the goods shelf, then goods images are calculated through Fourier-Mellin transform, high-precision positioning of various goods shelves and goods can be realized, and meanwhile, the calibrated image information can be updated when the mark point is passed each time, so that the conditions of self-adaption ground abrasion, change and the like are achieved, extra processing is not needed to be carried out on the goods, and the method is wide in application range and convenient to operate.
In order to achieve the above purpose, the invention adopts the following technical scheme:
a cargo positioning method based on Fourier-Mellin transform comprises the following steps:
step one, establishing an absolute coordinate system,
before the mobile robot is moved,
manually setting a plurality of calibration points in a coordinate system, wherein the calibration points are arranged on a moving path and used for marking the position of a goods shelf, the absolute coordinate of each calibration point is known, and the position relation between each marking point and the goods shelf is also known;
identifying a calibration point and positioning the position information of the goods shelf;
secondly, mounting a camera at a light-shading position of the object taking structure of the mobile robot;
step three, establishing an image feature library,
shooting goods on a goods shelf, and extracting specific frame images in the video in real time to serve as characteristic images of corresponding goods positions so as to form a series of characteristic images of the goods on the goods shelf;
acquiring the pose relationship between the current robot and the goods shelf through the mark point image, calculating an affine matrix, obtaining a new characteristic image through affine transformation of the previously obtained characteristic image, and storing the new characteristic image into a goods characteristic image library;
step four, when the fetching structure starts to position the goods, the goods image shot at the current fetching structure position is obtained,
if the texture of the current characteristic image is not overlapped with the characteristic image library, carrying out image registration based on Fourier-Mellin transform on the current characteristic image and the image of the previous frame with a known position so as to obtain the relative pose of the current fetching structure and the goods;
if the texture of the current characteristic image is overlapped with the characteristic image library, carrying out image registration based on Fourier-Mellin transform on the texture of the current characteristic image and the characteristic image library to obtain the relative pose of the current cargo;
and step five, registering the current position feature image with a feature image library, after the registration is finished, mashup the current position image after affine transformation with the feature image to obtain a new image with the features of the two images as a new known position image, deleting the feature image closest to the current frame, updating the feature image library, and jumping to the step four to perform subsequent cargo positioning work.
In the first step, the specific method for identifying the calibration point and locating the position information of the goods shelf includes: the calibration point is fixed on the ground or the goods shelf, the relative position of the calibration point and the goods shelf is fixed and known, the mobile robot images the calibration point through the camera, and the calibration point is identified to determine the serial number of the goods shelf.
In the first step, the method for locating the goods based on fourier-mellin transform is characterized in that the representation mode of the index point comprises: one-dimensional codes, two-dimensional codes, special symbols, or special textures.
In the foregoing cargo positioning method based on fourier-mellin transform,
in the second step, the camera is arranged at a light-shading position of the mobile robot fetching structure; if the camera can not cover all the ranges
And the shelf is divided into a plurality of sub-areas, and each sub-area is provided with a marking point for the movement of the robot.
In the foregoing cargo positioning method based on fourier-mellin transform,
in the third step, an image feature library is established,
placing goods on a goods shelf, shooting the goods on the goods shelf, extracting specific frame images in a video in real time to serve as characteristic images of corresponding goods positions, and further forming a series of characteristic images of the goods on the goods shelf; the specific frame is extracted according to the following steps that the first frame is a key frame, the image frame when the acquisition time of the first frame exceeds a certain time T is a key frame, and the image frame when the acquisition position of the first frame exceeds a certain range M is a key frame;
acquiring position information of the goods shelf by identifying the mark points, calculating an affine transformation matrix, carrying out affine transformation on an image shot by a camera of the fetching mechanism by using the affine transformation matrix to obtain a new characteristic image, and storing the new characteristic image into a goods characteristic image library; the position information of the goods shelf mainly comprises the relative position and the relative angle between the mobile robot and the goods shelf;
the image that shoots need cover the whole region of current goods shelves, if the coverage of getting the thing mechanism can't cover the whole region of goods shelves, places a plurality of calibration points in front of the goods shelves, divide into a plurality of subregion with the goods shelves, and the removal of cooperation robot realizes the coverage to the goods shelves region.
In the foregoing cargo positioning method based on fourier-mellin transform,
in the fourth step, when the fetching structure starts to position the goods, the goods image shot at the current fetching structure position is obtained,
if the texture of the current characteristic image is not overlapped with the characteristic image library, carrying out image registration based on Fourier-Mellin transform on the current characteristic image and the image of the previous frame with a known position so as to obtain the relative pose of the current fetching structure and the goods;
if the texture of the current characteristic image is overlapped with the characteristic image library, carrying out image registration based on Fourier-Mellin transform on the current characteristic image and the image of the overlapped characteristic image library to obtain the relative pose of the current cargo;
the method for acquiring the transverse moving relation between the goods and the fetching device comprises the following steps: using currently takenThe mutual energy spectrum of the Fourier change of the image and the image at the known position is used for solving the translation relation of the image, and further the position relation of the goods and the fetching structure is deduced; the Fourier spectrum of the known image and the current image is assumed to be F 1 And F 2 Then, the mutual energy spectrum of the two images in the frequency domain is:
inverse transformation is carried out on the mutual energy spectrum to obtain that the peak value is located at (x) 0 ,y 0 ) (ii) the impact function of (x) 0 ,y 0 ) Namely the position deviation between the current image and the known image, and the transverse movement relation between the goods and the fetching mechanism is calculated according to the initial position of the fetching structure and the camera internal reference.
In the foregoing cargo positioning method based on fourier-mellin transform,
in the fourth step, when the fetching structure starts to position the goods, the image of the goods shot at the current fetching structure position is obtained,
if the texture of the current characteristic image is not overlapped with the characteristic image library, carrying out image registration based on Fourier-Mellin transform on the current characteristic image and the image at the known position of the previous frame to obtain the rotation relation of the current fetching structure;
if the texture of the current characteristic image is overlapped with the characteristic image library, carrying out image registration based on Fourier-Mellin transform on the texture of the current characteristic image and the characteristic image library to obtain the rotation relation of the current goods;
the method for acquiring the rotation angle between the goods and the fetching device comprises the following steps: transforming the energy of Fourier spectrums of the currently shot image and the image at the known position into a polar coordinate system, solving the rotation relation of the image, and further deducing the rotation relation of the goods and the fetching structure;
the Fourier spectrum of the known image and the current image is assumed to be F 1 And F 2 And calculating an energy spectrum corresponding to the Fourier, wherein the formula is as follows:
energy conversion to polar coordinate system M 1 (ρ, θ) and M 2 (ρ, θ), for M in polar coordinates 1 And M 2 Solving for phase correlation
Is located at theta 0 The impulse function at the position, i.e. the angle between the current image and the known image, is theta 0 ；
And calculating the position relation of the goods and the fetching mechanism by the initial position of the fetching structure and the internal reference of the camera.
In the foregoing cargo positioning method based on fourier-mellin transform,
in the fourth step, when the fetching structure starts to position the goods, the image of the goods shot at the current fetching structure position is obtained,
if the texture of the current characteristic image does not overlap with the characteristic image library, carrying out image registration based on Fourier-Mellin transform on the current characteristic image texture and the image at the known position of the previous frame to obtain the distance relation of the current fetching structure;
if the texture of the current characteristic image is overlapped with the characteristic image library, carrying out image registration based on Fourier-Mellin transform on the texture of the current characteristic image and the characteristic image library to obtain the distance relation of the current goods;
the method for acquiring the distance between the goods and the fetching device comprises the following steps: transforming the energy of Fourier spectrums of the currently shot image and the image at the known position to a logarithmic coordinate system, solving the proportional relation of the images, further inquiring the distance represented by the current proportion by combining with the established hash table, and further obtaining the distance between the goods and the fetching structure; the Fourier spectrum of the known image and the current image is assumed to be F 1 (xi, eta) and F 2 (xi, eta), transforming them into a logarithmic coordinate system, F 1 (log ξ, log η) andF 2 (log xi, log η), solving for the phase correlation, the formula is as follows:
and obtaining a scale factor s of the current image and the known image, and establishing a Hash lookup table of the scale factor and the real distance in advance to obtain the distance between the goods and the fetching mechanism in the current state.
The invention has the advantages that:
the goods positioning method for the goods shelf enables a mobile robot to obtain the current goods shelf position and serial number through a calibration point, determines that the robot is behind the correct goods shelf, calculates goods images through Fourier-Mellin transformation, and obtains the transverse position, the deflection angle and the distance between the goods and a fetching device through comparison with a characteristic image library; the method has good detection and positioning accuracy on the goods, has small calculation amount, and effectively improves matching accuracy and performance, thereby improving the accuracy of positioning and picking and placing the goods of the mobile robot;
the invention can update the calibrated image information when passing through the calibration point each time, is self-adaptive to the conditions of ground abrasion, change and the like, does not need to carry out additional treatment on goods, and has wide application range and convenient operation.
Drawings
FIG. 1 is a flow chart of one embodiment of the present invention.
Detailed Description
The invention is described in detail below with reference to the figures and the embodiments.
A cargo positioning method based on Fourier-Mellin transform comprises the following steps:
step one, establishing an absolute coordinate system,
before the mobile robot is moved,
manually setting a plurality of calibration points in a coordinate system, wherein the calibration points are arranged on a moving path and used for marking the position of a goods shelf, the absolute coordinate of each calibration point is known, and the position relationship between each marking point and the goods shelf is also known;
identifying a calibration point and positioning the position information of the goods shelf;
the specific method for identifying the index point and positioning the position information of the goods shelf comprises the following steps: the calibration point is fixed on the ground or the goods shelf, the relative position of the calibration point and the goods shelf is fixed and known, the mobile robot images the calibration point through the camera, and the calibration point is identified to determine the serial number of the goods shelf.
The representation mode of the index point comprises the following steps: one-dimensional codes, two-dimensional codes, special symbols, or special textures.
Secondly, mounting a camera at a light-shading position of the object taking structure of the mobile robot;
installing a camera at a light-shading position of a mobile robot fetching structure; if the camera cannot cover all the ranges, the goods shelf is divided into a plurality of sub-areas, and each sub-area is provided with a marking point for the movement of the robot.
Step three, establishing an image feature library,
placing goods on a goods shelf, shooting the goods on the goods shelf, extracting specific frame images in a video in real time to serve as characteristic images of corresponding goods positions, and further forming a series of characteristic images of the goods on the goods shelf; the specific frame is extracted according to the following steps that the first frame is a key frame, the image frame when the acquisition time of the first frame exceeds a certain time T is a key frame, and the image frame when the acquisition position of the first frame exceeds a certain range M is a key frame;
acquiring position information of the goods shelf by identifying the mark points, calculating an affine transformation matrix, carrying out affine transformation on an image shot by a camera of the fetching mechanism by using the affine transformation matrix to obtain a new characteristic image, and storing the new characteristic image into a goods characteristic image library; the position information of the goods shelf mainly comprises the relative position and the relative angle between the mobile robot and the goods shelf;
the shot image needs to cover all the areas of the current goods shelf, if the coverage range of the fetching mechanism cannot cover all the areas of the goods shelf, a plurality of calibration points are placed in front of the goods shelf, the goods shelf is divided into a plurality of sub-areas, and the covering of the goods shelf areas is achieved in cooperation with the movement of the robot.
Step four, when the fetching structure starts to position the goods, the goods image shot at the current fetching structure position is obtained,
if the texture of the current characteristic image is not overlapped with the characteristic image library, carrying out image registration based on Fourier-Mellin transform on the current characteristic image and the image of the previous frame with a known position so as to obtain the relative pose of the current fetching structure and the goods;
if the texture of the current characteristic image is overlapped with the characteristic image library, carrying out image registration based on Fourier-Mellin transform on the texture of the current characteristic image and the characteristic image library to obtain the relative pose of the current cargo;
the pose relationship of the goods and the fetching mechanism has three parts, one is transverse movement, the other is angle, the other is scale, and the solution is separated.
If the position relation between the goods and the fetching mechanism is to be obtained, the fourth step is:
in the fourth step, when the fetching structure starts to position the goods, the goods image shot at the current fetching structure position is obtained,
if the texture of the current characteristic image is not overlapped with the characteristic image library, carrying out image registration based on Fourier-Mellin transform on the current characteristic image and the image of the last frame with the known position to obtain the relative pose of the current fetching structure and the goods;
if the texture of the current characteristic image is overlapped with the characteristic image library, carrying out image registration based on Fourier-Mellin transform on the current characteristic image and the image of the overlapped characteristic image library to obtain the relative pose of the current cargo;
the method for acquiring the transverse moving relation between the goods and the fetching device comprises the following steps: solving the translation relation of the images by using the Fourier transform mutual energy spectrum of the currently shot images and the images at known positions, and further deducing the position relation of goods and an object taking structure; the Fourier spectrum of the known image and the current image is assumed to be F 1 And F 2 Then, the mutual energy spectrum of the two images in the frequency domain is:
inverse transformation is carried out on the mutual energy spectrum to obtain that the peak value is located at (x) 0 ,y 0 ) (ii) the impact function of (x) 0 ,y 0 ) Namely the position deviation between the current image and the known image, and the transverse movement relation between the goods and the fetching mechanism is calculated according to the initial position of the fetching structure and the camera internal reference.
If the rotation relation between the goods and the fetching mechanism is obtained, the fourth step is as follows:
when the fetching structure starts to position the goods, the goods image shot at the current fetching structure position is obtained,
if the texture of the current characteristic image does not overlap with the characteristic image library, carrying out image registration based on Fourier-Mellin transform on the current characteristic image texture and the image at the known position of the previous frame to obtain the rotation relation of the current fetching structure;
if the texture of the current characteristic image is overlapped with the characteristic image library, carrying out image registration based on Fourier-Mellin transform on the texture of the current characteristic image and the characteristic image library to obtain the rotation relation of the current goods;
the method for acquiring the rotation angle between the goods and the fetching device comprises the following steps: transforming the energy of Fourier spectrums of the currently shot image and the image at the known position into a polar coordinate system, solving the rotation relation of the image, and further deducing the rotation relation of the goods and the fetching structure;
the Fourier spectrum of the known image and the current image is assumed to be F 1 And F 2 The fourier corresponding energy spectrum is:
energy conversion to polar coordinate system M 1 (ρ, θ) and M 2 (ρ, θ) for M in polar coordinates 1 And M 2 Solving for phase correlation
Is located at theta 0 The impulse function at the position, i.e. the angle between the current image and the known image, is theta 0 ；
And calculating the position relation of the goods and the fetching mechanism by the initial position of the fetching structure and the internal reference of the camera.
If the distance relationship between the goods and the fetching mechanism is to be obtained, the fourth step is:
when the fetching structure starts to position the goods, the goods image shot at the current fetching structure position is obtained,
if the texture of the current characteristic image is not overlapped with the characteristic image library, carrying out image registration based on Fourier-Mellin transform on the current characteristic image and the image at the known position of the previous frame to obtain the distance relation of the current fetching structure;
if the texture of the current characteristic image is overlapped with the characteristic image library, carrying out image registration based on Fourier-Mellin transform on the texture of the current characteristic image and the characteristic image library to obtain the distance relation of the current goods;
the method for acquiring the distance between the goods and the fetching device comprises the following steps: transforming the energy of Fourier spectrums of the currently shot image and the image at the known position to a logarithmic coordinate system, solving the proportional relation of the images, further inquiring the distance represented by the current proportion by combining with the established hash table, and further obtaining the distance between the goods and the fetching structure; the Fourier spectrum of the known image and the current image is assumed to be F 1 (xi, eta) and F 2 (xi, eta), transforming them into a logarithmic coordinate system, F 1 (log xi, log η) and F 2 (log xi, log η), solving for the phase correlation, the formula is as follows:
and obtaining a scale factor s of the current image and the known image, and establishing a Hash lookup table of the scale factor and the real distance in advance to obtain the distance between the goods and the fetching mechanism in the current state.
And step five, registering the characteristic image of the current position with a characteristic image library, after the registration is finished, mashuping the affine transformed current position image with the characteristic image to obtain a new image with the characteristics of two images as a new known position image, deleting the characteristic image closest to the current frame, updating the characteristic image library, and jumping to the step four to perform subsequent cargo positioning work.
The two-dimensional code is selected as an identification method of a calibration point, a mechanical arm is selected for an object taking structure, and a hardware platform selects zynq 7020 for specific demonstration:
(1) Establishing an absolute coordinate system, manually setting a plurality of calibration points in the coordinate system before the mobile robot moves, wherein the calibration points are arranged on a moving path and used for positioning the position of the goods shelf, and the absolute coordinate of each manually set calibration point is known. Such as but not limited to: posting the two-dimensional code picture on the ground of goods shelf front side, the information that the two-dimensional code contains is the goods shelf number, and its angle is perpendicular with the goods shelf, and the robot removes directly over the two-dimensional code, and camera through the robot bottom forms images to the two-dimensional code, acquires the goods shelf code, acquires the contained angle theta between robot and the goods shelf yaw 。
(2) The camera is arranged at the front light-shading position of the fetching structure of the mobile robot, so that the mobile robot can automatically shoot goods on the goods shelf in front of the goods shelf; such as but not limited to: controlling the robot to move to the front of a goods shelf marked by a certain marking point; a mechanical arm on the robot is used for grabbing goods, and a camera for shooting the goods is arranged at a light-shading position on the mechanical arm; the moving range of the mechanical arm can cover all goods areas on the goods shelf; if the robotic arm is not able to cover all areas, the rack is divided into several sub-areas, so that the robotic arm can cover all cargo areas of each sub-area.
(3) Shooting goods on a goods shelf in advance, extracting specific frame images in a video in real time to serve as characteristic images of corresponding goods positions, further forming a series of characteristic images of the goods on the goods shelf, simultaneously carrying out affine transformation to obtain new characteristic images, and recording the new characteristic images as a goods characteristic image library; such as but not limited to: at a mechanical armSetting equispaced image acquisition points in the covered cargo range, shooting the cargo at the positions of the acquisition points, and utilizing theta yaw And (3) establishing an affine transformation matrix, and storing the acquired cargo image in a memory of the robot after affine transformation to establish a characteristic graph library.
(4) After the characteristic graph library is constructed, assuming that the fetching mechanical arm moves to a certain position, acquiring a current cargo image shot by a mechanical arm camera and recording the image as I 0 (ii) a Selecting a four-adjacent-domain feature pattern I adjacent to the current position in a feature pattern library according to the current mechanical arm position 1 、I 2 、I 2 And I 4 As objects of registration; I.C. A 0 Are each independently of I 1 、I 2 、I 2 And I 4 Carrying out image registration based on Fourier Mellin transform; with I 0 And I 1 For example, for I 0 And I 1 Fourier transform is performed to obtain F 0 (. Epsilon.,. Eta.) and F 1 (ε,η)。
(5) To F 0 (. Epsilon.,. Eta.) and F 1 (epsilon, eta) high-pass filtering is carried out, and the high-pass filtering template is as follows:
H(ε,η)＝(1.0-X(ε,η))(2.0-X(ε,η))，
wherein
X(ε,η)＝cos(πε)cos(πη) -0.5≤ε,η≤0.5；
F is to be 0 (. Epsilon.,. Eta.) and F 1 And (epsilon, eta) is multiplied by the template H (epsilon, eta) to obtain a filtered Fourier transform spectrum.
(6) F is to be 0 (. Epsilon.,. Eta.) and F 1 (ε, η) transformation to polar coordinate F 0 (p, theta) and F 1 (ρ, θ), the ρ radius is transformed to logarithmic coordinates ζ = log ρ, yielding F 0 (ζ, θ) and F 1 (ζ, θ), and obtaining a corresponding mutual energy spectrum, wherein the formula is as follows:
performing inverse Fourier transform on the mutual energy spectrum, wherein the formula is as follows:
finding the peak value of M ', if M' has a plurality of peak values with similar intensity or the intensity of the peak value is lower than the threshold value, considering I 0 And I 1 Is poor in matching degree, continues to be matched with I 2 、I 2 And I 4 And performing calculation until the characteristic graph meeting the condition is found.
(7) If a qualifying characteristic image is found, e.g. I 1 Assuming that the found peak position is (ζ ', θ'), the scale factor between the two images is e ζ ', the angular difference is θ'. Will I 1 Transforming according to the calculated scale factor and angle, and recording the transformed image as I' 1 . According to I 0 And l' 1 Calculating the peak value
The formula is as follows:
the position of the peak is I after transformation 0 And l' 1 To the other.
(8) If at I 1 、I 2 、I 2 And I 4 If no qualified match is found, let I 0 And (4) matching with the image of which the conversion parameter is obtained in the previous frame, wherein the matching method is the same as that in the steps (4) to (7). The current position image and the feature image are mixed to obtain a new image with the features of the two images as a new known position image; finally, adding the current frame into the characteristic graph, and adding I 1 、I 2 、I 2 And I 4 Deleting the feature pattern closest to the current frame, and updating the feature pattern library so as to perform subsequent matching work.
(9) A Hash lookup table of the size and the distance of an object in an image is established in advance, the size of the object in the current frame is obtained according to the scale factor, and the distance from the object in the current frame to the mechanical arm camera is obtained through table lookup.
The invention provides a Fourier-Mellin transform-based mobile robot goods positioning method, which is characterized in that a goods shelf is calibrated by using images on the ground or the goods shelf, then the images of the goods are calculated by Fourier-Mellin transform, so that high-precision positioning of various goods shelves and goods can be realized, and the calibrated image information can be updated each time the goods pass through a calibration point, so that the conditions of ground abrasion, change and the like can be adapted, additional processing on the goods is not required, the application range is wide, and the operation is convenient.
The foregoing illustrates and describes the principles, general features, and advantages of the present invention. It should be understood by those skilled in the art that the above embodiments do not limit the present invention in any way, and all technical solutions obtained by using equivalents or equivalent changes fall within the protection scope of the present invention.
Claims (8)
1. A cargo positioning method based on Fourier-Mellin transform is characterized by comprising the following steps:
step one, establishing an absolute coordinate system,
before the mobile robot is moved,
manually setting a plurality of calibration points in a coordinate system, wherein the calibration points are arranged on a moving path and used for marking the position of a goods shelf, the absolute coordinate of each calibration point is known, and the position relation between each marking point and the goods shelf is also known;
identifying a calibration point and positioning the position information of the goods shelf;
step two, installing a camera at a light-shading position of the mobile robot fetching structure;
step three, establishing an image feature library,
shooting goods on a goods shelf, and extracting specific frame images in the video in real time to serve as characteristic images of corresponding goods positions so as to form a series of characteristic images of the goods on the goods shelf;
acquiring the pose relation between the current robot and the goods shelf through the mark point image, calculating an affine matrix, obtaining a new characteristic image through affine transformation of the previously obtained characteristic image, and storing the new characteristic image into a goods characteristic image library;
step four, when the fetching structure starts to position the goods, the goods image shot at the current fetching structure position is obtained,
if the texture of the current characteristic image is not overlapped with the characteristic image library, carrying out image registration based on Fourier-Mellin transform on the current characteristic image and the image of the previous frame with a known position so as to obtain the relative pose of the current fetching structure and the goods;
if the texture of the current characteristic image is overlapped with the characteristic image library, carrying out image registration based on Fourier-Mellin transform on the texture of the current characteristic image and the characteristic image library to obtain the relative pose of the current cargo;
and step five, registering the current position feature image with a feature image library, after the registration is finished, mashup the current position image after affine transformation with the feature image to obtain a new image with the features of the two images as a new known position image, deleting the feature image closest to the current frame, updating the feature image library, and jumping to the step four to perform subsequent cargo positioning work.
2. The Fourier-Mellin transform-based cargo positioning method as recited in claim 1,
in the first step, the specific method for identifying the index point and positioning the position information of the goods shelf comprises the following steps: the calibration point is fixed on the ground or the goods shelf, the relative position of the calibration point and the goods shelf is fixed and known, the mobile robot images the calibration point through the camera, and the calibration point is identified to determine the serial number of the goods shelf.
3. The fourier-mellin transform-based cargo positioning method as defined in claim 2, wherein in the first step, the indicating manner of the index point comprises: one-dimensional codes, two-dimensional codes, special symbols, or special textures.
4. The Fourier-Mellin transform-based cargo positioning method as recited in claim 1,
in the second step, the camera is arranged at a light-shading position of the mobile robot fetching structure; if the camera cannot cover all the ranges, the goods shelf is divided into a plurality of sub-areas, and each sub-area is provided with a marking point for the movement of the robot.
5. The Fourier-Mellin transform-based cargo positioning method as recited in claim 1,
in the third step, an image feature library is established,
placing goods on a goods shelf, shooting the goods on the goods shelf, extracting specific frame images in the video in real time to serve as characteristic images of corresponding goods positions, and further forming a series of characteristic images of the goods on the goods shelf; the specific frame is extracted according to the following steps that the first frame is a key frame, the image frame when the acquisition time of the first frame exceeds a certain time T is a key frame, and the image frame when the acquisition position of the first frame exceeds a certain range M is a key frame;
acquiring position information of the goods shelf by identifying the mark points, calculating an affine transformation matrix, carrying out affine transformation on an image shot by a camera of the fetching mechanism by using the affine transformation matrix to obtain a new characteristic image, and storing the new characteristic image into a goods characteristic image library; the position information of the goods shelf mainly comprises the relative position and the relative angle between the mobile robot and the goods shelf;
the image that shoots need cover the whole region of current goods shelves, if the coverage of getting the thing mechanism can't cover the whole region of goods shelves, places a plurality of calibration points in front of the goods shelves, divide into a plurality of subregion with the goods shelves, and the removal of cooperation robot realizes the coverage to the goods shelves region.
6. The Fourier-Mellin transform-based cargo positioning method as recited in claim 1,
in the fourth step, when the fetching structure starts to position the goods, the goods image shot at the current fetching structure position is obtained,
if the texture of the current characteristic image is not overlapped with the characteristic image library, carrying out image registration based on Fourier-Mellin transform on the current characteristic image and the image of the previous frame with a known position so as to obtain the relative pose of the current fetching structure and the goods;
if the texture of the current characteristic image is overlapped with the characteristic image library, carrying out image registration based on Fourier-Mellin transform on the current characteristic image and the image of the overlapped characteristic image library to obtain the relative pose of the current cargo;
the method for acquiring the transverse movement relationship between the goods and the fetching device comprises the following steps: solving the translation relation of the images by using the Fourier transform mutual energy spectrum of the currently shot images and the images at known positions, and further pushing out the position relation of the goods and the fetching structure; the Fourier spectrum of the known image and the current image is assumed to be F 1 And F 2 And calculating the mutual energy spectrum of the two images in the frequency domain, wherein the formula is as follows:
inverse transformation is carried out on the mutual energy spectrum to obtain that the peak value is located at (x) 0 ,y 0 ) (ii) the impact function of (x) 0 ,y 0 ) Namely the position deviation between the current image and the known image, and the transverse movement relation between the goods and the fetching mechanism is calculated according to the initial position of the fetching structure and the camera internal reference.
7. The Fourier-Mellin transform-based cargo positioning method as recited in claim 1,
in the fourth step, when the fetching structure starts to position the goods, the goods image shot at the current fetching structure position is obtained,
if the texture of the current characteristic image does not overlap with the characteristic image library, carrying out image registration based on Fourier-Mellin transform on the current characteristic image texture and the image at the known position of the previous frame to obtain the rotation relation of the current fetching structure;
if the texture of the current characteristic image is overlapped with the characteristic image library, carrying out image registration based on Fourier-Mellin transform on the texture of the current characteristic image and the characteristic image library to obtain the rotation relation of the current goods;
the method for acquiring the rotation angle between the goods and the fetching device comprises the following steps: transforming the energy of Fourier spectrums of the currently shot image and the image at the known position into a polar coordinate system, solving the rotation relation of the image, and further deducing the rotation relation of the goods and the fetching structure;
the Fourier spectrum of the known image and the current image is assumed to be F 1 And F 2 And calculating an energy spectrum corresponding to the Fourier, wherein the formula is as follows:
energy conversion to polar coordinate system M 1 (ρ, θ) and M 2 (ρ, θ), for M in polar coordinates 1 And M 2 Solving for the phase correlation, the formula is as follows:
is located at theta 0 The impulse function at the position, i.e. the angle between the current image and the known image, is theta 0 ；
And calculating the position relation of the goods and the fetching mechanism by the initial position of the fetching structure and the internal reference of the camera.
8. The Fourier-Mellin transform-based cargo positioning method according to claim 1,
in the fourth step, when the fetching structure starts to position the goods, the goods image shot at the current fetching structure position is obtained,
if the texture of the current characteristic image is not overlapped with the characteristic image library, carrying out image registration based on Fourier-Mellin transform on the current characteristic image and the image at the known position of the previous frame to obtain a transformation scale relation of the current fetching structure;
if the texture of the current characteristic image is overlapped with the characteristic image library, carrying out image registration based on Fourier-Mellin transformation on the texture of the current characteristic image and the characteristic image library to obtain a transformation scale relation of the current goods;
the method for acquiring the distance between the goods and the fetching device comprises the following steps: transforming the energy of Fourier spectrums of the currently shot image and the image at the known position to a logarithmic coordinate system, solving the relation of the transformation scale of the image, further inquiring the distance represented by the current scale by combining with the established hash table, and further obtaining the distance between the goods and the fetching structure; the Fourier spectrum of the known image and the current image is assumed to be F 1 (xi, eta) and F 2 (xi, eta), transforming them into a logarithmic coordinate system, F 1 (log xi, log eta) and F 2 (log xi, log η), solving for phase correlation:
and obtaining a scale factor s of the current image and the known image, and pre-establishing a Hash lookup table of the scale factor and the real distance to obtain the distance between the goods and the fetching mechanism in the current state.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910345255.4A CN110097494B (en) | 2019-04-26 | 2019-04-26 | Fourier-Mellin transform-based cargo positioning method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910345255.4A CN110097494B (en) | 2019-04-26 | 2019-04-26 | Fourier-Mellin transform-based cargo positioning method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110097494A CN110097494A (en) | 2019-08-06 |
| CN110097494B true CN110097494B (en) | 2023-01-13 |
Family
ID=67446076
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910345255.4A Active CN110097494B (en) | 2019-04-26 | 2019-04-26 | Fourier-Mellin transform-based cargo positioning method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110097494B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111951318B (en) * | 2020-08-10 | 2023-08-04 | 上海科技大学 | Extended Fourier Merlin positioning algorithm applied to multi-depth scene |
| CN115086539B (en) * | 2021-03-15 | 2024-02-02 | 虫极科技（北京）有限公司 | Shooting point positioning method and system |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2004240931A (en) * | 2003-02-05 | 2004-08-26 | Sony Corp | Image collation device, image collation method, and program |
| CN106996777B (en) * | 2017-04-21 | 2019-02-12 | 合肥井松自动化科技有限公司 | A kind of vision navigation method based on ground image texture |
| CN108373001A (en) * | 2018-02-01 | 2018-08-07 | 王学斌 | A kind of intelligent industrial robot automation warehousing system |
-
2019
- 2019-04-26 CN CN201910345255.4A patent/CN110097494B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN110097494A (en) | 2019-08-06 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP7292829B2 (en) | Systems and methods for combining machine vision coordinate spaces in a guided assembly environment | |
| JP2018169403A5 (en) | ||
| CN108044627B (en) | Method and device for detecting grabbing position and mechanical arm | |
| CN102640185B (en) | The method and apparatus of the combined tracking that object represents in real time in image sequence | |
| CN109159114A (en) | The accuracy method of SCARA manipulator fixed camera vision system hand and eye calibrating | |
| CN108109174A (en) | A kind of robot monocular bootstrap technique sorted at random for part at random and system | |
| CN111738261A (en) | Pose estimation and correction-based disordered target grabbing method for single-image robot | |
| US20040172164A1 (en) | Method and apparatus for single image 3D vision guided robotics | |
| US20050002555A1 (en) | Image processing apparatus | |
| CN110097494B (en) | Fourier-Mellin transform-based cargo positioning method | |
| CN106845354B (en) | Part view library construction method, part positioning and grabbing method and device | |
| CN113781561B (en) | Target pose estimation method based on self-adaptive Gaussian weight quick point feature histogram | |
| CA2354301A1 (en) | Geometric hashing for model-based recognition of an object | |
| Sehgal et al. | Real-time scale invariant 3D range point cloud registration | |
| CN113329179B (en) | Shooting alignment method, device, equipment and storage medium | |
| CN116958146B (en) | Acquisition method and device of 3D point cloud and electronic device | |
| CN113744409B (en) | Workpiece positioning method, device, system, equipment and medium | |
| WO2022002149A1 (en) | Initial localization method, visual navigation device, and warehousing system | |
| Han et al. | Target positioning method in binocular vision manipulator control based on improved canny operator | |
| CN117115105A (en) | Workpiece processing method, device, equipment and storage medium | |
| Fan et al. | An automatic robot unstacking system based on binocular stereo vision | |
| CN110986916A (en) | Indoor positioning method and device, electronic equipment and storage medium | |
| CN116188540A (en) | Target identification and pose estimation method based on point cloud information | |
| Alpatov et al. | An estimation algorithm of the multispectral image geometric transformation parameters based on multiple reference area tracking | |
| CN108550168A (en) | The multiple target quick high accuracy of Intelligentized mechanical arm view-based access control model identifies and localization method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |