CN110189267B - Real-time positioning device and system based on machine vision - Google Patents
Real-time positioning device and system based on machine vision Download PDFInfo
- Publication number
- CN110189267B CN110189267B CN201910420567.7A CN201910420567A CN110189267B CN 110189267 B CN110189267 B CN 110189267B CN 201910420567 A CN201910420567 A CN 201910420567A CN 110189267 B CN110189267 B CN 110189267B
- Authority
- CN
- China
- Prior art keywords
- module
- marki
- image
- real
- stereoscopic vision
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000012545 processing Methods 0.000 claims abstract description 34
- 238000004364 calculation method Methods 0.000 claims abstract description 20
- 238000007667 floating Methods 0.000 claims abstract description 13
- 230000001360 synchronised effect Effects 0.000 claims abstract description 13
- 238000004422 calculation algorithm Methods 0.000 claims description 18
- 238000000605 extraction Methods 0.000 claims description 13
- 238000007781 pre-processing Methods 0.000 claims description 12
- 230000004927 fusion Effects 0.000 claims description 11
- 238000005259 measurement Methods 0.000 claims description 9
- 239000013598 vector Substances 0.000 claims description 7
- 238000001914 filtration Methods 0.000 claims description 6
- 238000004891 communication Methods 0.000 claims description 3
- 230000004044 response Effects 0.000 abstract description 3
- 238000013461 design Methods 0.000 abstract description 2
- 238000005516 engineering process Methods 0.000 description 23
- 101001059443 Homo sapiens Serine/threonine-protein kinase MARK1 Proteins 0.000 description 18
- 102100028921 Serine/threonine-protein kinase MARK1 Human genes 0.000 description 18
- 238000010586 diagram Methods 0.000 description 8
- 230000003287 optical effect Effects 0.000 description 6
- 238000000034 method Methods 0.000 description 5
- 238000003384 imaging method Methods 0.000 description 3
- 238000011160 research Methods 0.000 description 3
- 230000000007 visual effect Effects 0.000 description 3
- 230000001133 acceleration Effects 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 101001059454 Homo sapiens Serine/threonine-protein kinase MARK2 Proteins 0.000 description 1
- 102100028904 Serine/threonine-protein kinase MARK2 Human genes 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- QBWCMBCROVPCKQ-UHFFFAOYSA-N chlorous acid Chemical compound OCl=O QBWCMBCROVPCKQ-UHFFFAOYSA-N 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000006073 displacement reaction Methods 0.000 description 1
- 238000007499 fusion processing Methods 0.000 description 1
- 238000005286 illumination Methods 0.000 description 1
- 239000011159 matrix material Substances 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000012821 model calculation Methods 0.000 description 1
- 230000035484 reaction time Effects 0.000 description 1
- 238000011524 similarity measure Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/70—Denoising; Smoothing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T19/00—Manipulating 3D models or images for computer graphics
- G06T19/006—Mixed reality
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
- G06T7/33—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
- G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
- G06T7/75—Determining position or orientation of objects or cameras using feature-based methods involving models
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
- G06V10/75—Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries
- G06V10/757—Matching configurations of points or features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20024—Filtering details
- G06T2207/20032—Median filtering
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Software Systems (AREA)
- Computing Systems (AREA)
- General Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Evolutionary Computation (AREA)
- Databases & Information Systems (AREA)
- Artificial Intelligence (AREA)
- Health & Medical Sciences (AREA)
- Multimedia (AREA)
- Computer Graphics (AREA)
- Computer Hardware Design (AREA)
- General Engineering & Computer Science (AREA)
- Length Measuring Devices By Optical Means (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a real-time positioning device and a system based on a machine vision system, wherein the device comprises: the device comprises an FPGA hardware logic module and a high-speed DSP floating point calculation module. The system comprises a stereoscopic vision camera, a VR helmet unit, a stereoscopic vision processing board and a computer. According to the invention, the position of the MARK point is identified through the image, and the MARK point is subjected to synchronous control design, so that the real-time response and the reliability of the system are improved.
Description
Technical Field
The invention belongs to the field of machine vision, and particularly relates to a real-time positioning device and system based on machine vision.
Background
Machine vision aims to identify and understand content in images/videos with computers. The positioning and attitude-fixing technology is a technology for digitizing the position and attitude of motion by physical means such as inertia, optics and the like, and belongs to a branch of machine vision. The current mainstream motion capture technologies include optical motion capture technologies and inertial motion capture technologies. The mainstream optical motion capture technology research in the world mainly comprises Oxford Metrics Limited company, MotionNalisis company and Natural Point company in the UK, which respectively develop a Vicon system, a Raptor system and an OptiTrack system, wherein the tracking frequency of the systems is more than 60Hz, and the tracking precision can reach the millimeter level. The research on the inertial kinetic capture technology mainly comprises the Holland Xsens company and the Beijing Nordic technologies, which respectively research and develop an Xsens MVN system and a Legacy system, and the systems can realize multi-person motion capture with 10ms delay and 60Hz frame rate.
The existing motion capture technology is continuously developed and perfected, the requirements of people on the usability, the calculation precision, the real-time property, the economy and the like of positioning and posture fixing are continuously increased, and the optical vision positioning technology and the inertial positioning and posture fixing technology have advantages and disadvantages respectively and are difficult to meet all requirements. Although the optical vision positioning technology has high position precision, the optical vision positioning technology has poor usability and high use cost and can not directly measure the posture. The inertial positioning technology does not use space limitation, can directly measure acceleration and angular velocity, but has low position precision and is easily interfered by an external magnetic field. The combination of various technical means, especially measurement means mainly based on inertia and vision, is a main development direction of future positioning and attitude determination technology.
However, there are many difficulties in the prior art that have not yet been overcome. For example, a vision sensor has no way to work for non-textured areas. An Inertial Measurement Unit (IMU) can measure angular velocity and acceleration through a built-in gyroscope and accelerometer to estimate the attitude of the camera, but the estimated attitude has accumulated errors. How to fuse the measurement information of the vision sensor and the IMU to realize real-time positioning is an urgent technical problem to be solved in the industry.
Disclosure of Invention
The system is based on VIO (Visual Inertial odometer) technology, and MARK points and MARK synchronous control design are added on the basis of the VIO technology, so that real-time response and reliability of the system are improved. The technical scheme provided by the application is as follows:
a real-time positioning device based on a machine vision system, comprising: the FPGA hardware logic module and the high-speed DSP floating point calculation module;
the FPGA hardware logic module comprises a video acquisition preprocessing module, a first group manager and a MARKi synchronous control module; video data acquired by a stereoscopic vision camera is input into a video acquisition preprocessing module; the output data of the video acquisition preprocessing module is sent to a first group manager; the first group manager is controlled by a MARKi synchronous control module; the MARKi synchronous control module is connected to a real-time control bus of the system through a real-time control bus communication protocol;
the high-speed DSP floating point calculation module comprises a system calibration module, a calibration parameter module, a second group manager and a VIO information fusion module which are sequentially connected;
the first group of managers comprises a synchronization frame MARKi image taking module, a MARKi feature extraction (SURF algorithm) module, a MARKi image matching module and a MARKi image coordinate calculating module;
the second group of manager modules comprise a MARKi space coordinate resolving module, an HTi space coordinate and attitude resolving module and an IMUi attitude information module;
data of a MARKi image coordinate calculating module in the first group of managers is sent to a MARKi space coordinate calculating module in the second group of managers; the MARKi space coordinate resolving module in the second group of managers sends the processed data to the HTi space coordinate and attitude resolving module; the IMUi attitude information module acquires IMUi attitude information from a system real-time control bus; the VIO information fusion module simultaneously acquires output data of the HTi space coordinate and attitude calculation module and the IMUi attitude information module; and the VIO information fusion module outputs the reported information.
A real-time positioning system based on a machine vision system, comprising: a stereoscopic vision camera, a VR headset unit, a stereoscopic vision processing board and a computer, the stereoscopic vision processing board comprising a real-time positioning device based on a machine vision system as claimed in claim 1.
According to one aspect of the application, the system sends data collected by an IMU module and a MARK synchronizer arranged on each VR helmet unit to a stereoscopic vision processing board through a real-time control bus of the system; the stereoscopic vision processing board obtains the video acquisition information through the stereoscopic vision camera and fuses IMU information obtained by the IMU module to obtain pose information.
According to one aspect of the application, the system filters digital images acquired by a stereoscopic vision camera through a fast median filtering algorithm.
According to one aspect of the application, the system takes a MARKi image of a synchronous frame to perform Feature extraction, integrates the Feature region image by using a Speeded Up Robust Feature (SURF) algorithm, performs convolution operation to obtain harr features of a MARK point, and calculates multi-dimensional vectors and principal Feature vectors of the harr features to form a Feature descriptor of the MARK point.
According to one aspect of the application, the system adopts Euclidean distance as similarity measurement, one feature point in an image is taken, N feature points which are nearest neighbor in reference Euclidean distance are found according to a threshold value, wherein N is a natural number, comparison is carried out, and if more than 60% of feature points are smaller than the threshold value, the feature points are considered to be matched; and resolving the three-dimensional space position of the MARK point according to the relation between the image coordinate and the space coordinate of the matching point.
According to one aspect of the application, the stereoscopic vision camera is a binocular camera assembly.
According to one aspect of the application, the stereoscopic vision processing board sends the pose information to the computer through the USB interface.
Compared with the prior art, the invention has the following characteristics:
1) FPGA and DSP based rapid SURF algorithm
The feature extraction algorithm adopts a SURF algorithm with robust features, and improves harr feature operators, so that the feature region images can be quickly integrated in FPGA hardware logic, while convolution operation related to floating points is completed in a DSP.
2) Fast feature matching based on FPGA and synchronization technology
Image feature recognition and matching are technical difficulties in stereo measurement and are also a main part influencing system delay. The algorithms are realized by adopting FPGA hardware logic, so that the reaction time of the system is greatly improved, the video frame rate is 24 ms/frame, and meanwhile, the resource pressure brought to the FPGA by distinguishing the characteristic points ID is reduced by adopting a synchronization technology, so that the processing time is less than 100ms while the multi-target three-dimensional attitude calculation can be processed, and the real-time requirement is met.
According to the invention, the precision of the attitude angle is less than 0.01 degree within the range of not less than 1 meter, and the precision of the displacement is less than 1 mm. And the target number is not less than 5, and the corresponding time is less than 100 ms. High-precision real-time data can be provided for the virtual simulation scene.
Drawings
FIG. 1 is a schematic diagram of the overall system scheme;
FIG. 2 is a schematic diagram of a MARK point synchronization control flow;
FIG. 3 is a block diagram of the composition of a stereoscopic processing panel;
FIG. 4 is a schematic diagram of a binocular stereo vision technique;
FIG. 5 is a schematic diagram of a feature descriptor;
fig. 6 is a software processing module configuration diagram of system data.
Detailed Description
The technical content of the invention is further explained by combining the embodiment; the following examples are illustrative and not intended to be limiting, and are not intended to limit the scope of the invention.
1. Overall system scheme
As shown in fig. 1, the system components include a stereoscopic vision camera, a VR headset unit, a stereoscopic vision processing board, and a computer. Wherein the VR helmet unit includes an IMU module, a MARK synchronizer. The computer includes a processor and a memory. Preferably, the stereoscopic camera is a binocular camera assembly.
And arranging N VR helmets in the visual range of the stereoscopic vision camera, wherein N is a natural number. And the data collected by the IMU module and the MARK synchronizer on each VR helmet unit is sent to a stereoscopic vision processing board through a real-time control bus of the system. The stereoscopic vision processing board obtains video acquisition information through a stereoscopic vision camera. The stereoscopic vision processing board sends the pose information to the computer through the USB interface.
A MARK point with identifiable characteristics, i.e., MARK1, MARK2 · MARKi, is placed on each VR helmet. The image can be accurately and stably captured, the image processing pressure is reduced, and the reliability of the system is improved.
Because the accumulated error of the accelerometer is large, the IMU attitude accuracy can meet the application. The system thus uses the pose angle information of the IMU to assist in the pose measurement for stereoscopic vision. And (3) carrying out high-precision positioning and attitude determination on the VR helmet attitude by adopting a stereoscopic vision technology, and carrying out attitude auxiliary fusion processing by adopting a micro inertial sensor IMU.
In the existing DSP image processing technology, when a plurality of feature points of a plurality of targets are processed, the feature points are identified, feature extraction, stereo matching and the like, so that the resource consumption is very high, real-time calculation cannot be realized, and image processing cannot be completed within 24ms (video frame interval time), thereby causing video response jamming. The present application uses FPGA hardware logic to accomplish these tasks.
Specifically, the system adopts a MARK point synchronization control technology. And generating a synchronization signal by the FPGA, and sending the synchronization signal to a synchronization controller of the MARK point. The VR helmet MARK point is identified and processed by the FPGA, and calculation can be completed within 5ms according to past results. And at this moment, the ID of each VR helmet is known, so that the identification misjudgment and resource consumption of a plurality of VR helmets processed simultaneously are avoided. The flow of MARK point synchronization control is shown in fig. 2.
2. Hardware platform
As shown in fig. 1-3, the hardware platform of the present invention includes a binocular camera assembly, a VR headset, a stereoscopic vision processing board, and a computer. As shown in fig. 3, the stereoscopic vision processing board is a stereoscopic vision processing board configured based on an FPGA + DSP platform. The stereo vision processing plate comprises the following main parts: the device comprises a video acquisition circuit, an FPGA logic processing module, a DSP floating point operation module and a USB interface. The FPGA logic processing module and the DSP floating point operation module are simultaneously connected to a real-time control bus of the system. The stereo vision processing board has a main stereo measurement function, and only the visual positioning calculation software needs to be modified according to different characteristics of the light spots.
The FPGA is used as a logic circuit of high-capacity and flexible hardware with programmable logic, integrates a plurality of hardware resources, such as an embedded DSP block, an embedded RAM block, a high-speed external memory interface (DDR) and the like, and a logic processing platform, is very suitable for real-time processing of high-performance videos and images, can realize the parallel processing capability of the images, and mainly completes filtering, stereo matching, feature point marking and tracking algorithms.
The DSP uses a high performance 32-bit floating point DSP chip. The DSP adopts a Harvard bus structure, has a password protection mechanism, and can perform double 16 multiplied by 16 and 32 multiplied by 32 addition operations in one period, thereby having double functions of control and quick operation. The calculation speed is incomparable with that of a general-purpose computer and a single chip microcomputer. The method mainly completes the three-dimensional positioning calculation and the attitude matrix model calculation.
The stereoscopic vision processing board adopts a USB interface to output the attitude information.
3. Software processing of system data
The software processing process of the system data is based on the binocular stereo vision technology. Binocular Stereo Vision (Binocular Stereo Vision) consists of left and right cameras. As shown in fig. 4, the corresponding parameters of the left and right cameras are denoted by the subscripts l and r, respectively. The image points of a point A (X, Y, Z) in the world space on the imaging planes Cl and Cr of the left and right cameras are al (ul, vl) and ar (ur, vr), respectively. These two image points are images of the same object point a in world space, and are called "conjugate points". Knowing these two conjugate image points, we respectively make their connection lines with the optical centers Ol and Or of the respective cameras, i.e. the projection lines alOl and arOr, and their intersection points are the object points a (X, Y, Z) in world space.
The system software processing process comprises image acquisition preprocessing, feature extraction and image matching. The corresponding software processing module comprises an FPGA hardware logic module and a high-speed DSP floating point calculation module. The FPGA hardware logic module comprises a video acquisition preprocessing module, a first group of managers and a MARKi synchronous control module. The high-speed DSP floating point calculation module comprises a system calibration module, a calibration parameter module, a second group of managers and a VIO information fusion module.
Image acquisition preprocessing:
the acquisition of digital images is a source of information for stereoscopic vision. When a spatial three-dimensional scene is transformed into a two-dimensional image through Perspective Projection (Perspective Projection), the imaging of the same scene on camera image planes of different viewpoints can be distorted and deformed to different degrees, and the influences of various factors such as the illumination condition in the scene, the geometric shape and surface characteristics of a detected object, noise interference and distortion, camera characteristics and the like are collectively reflected in a single image gray value, and the image needs to be further filtered, such as unclear or distorted, which is not beneficial to image feature extraction.
The invention carries out filtering processing on the digital image acquired by the stereoscopic vision camera through a rapid median filtering algorithm. The edge characteristics of the image are kept, noise can be effectively removed, and meanwhile, the algorithm is designed by adopting an FPGA (field programmable gate array), so that the real-time performance of processing can be guaranteed.
Feature extraction:
and taking the filtered digital image as a synchronous frame MARKi image for feature extraction. The Feature extraction algorithm adopts a speedup Robust Feature (SURF) algorithm. The algorithm integrates the images in the characteristic region, then carries out convolution operation to obtain harr characteristics of the MARK point, and calculates the multidimensional vector and the principal characteristic vector of the harr characteristics to form the characteristic descriptor of the characteristic point. A schematic of the feature descriptors is shown in fig. 5. The center of the left image in fig. 5 represents a feature point. The 64 small squares near the feature points represent pixel points in their local regions. In the image local gradient direction, one or more directions are assigned to the feature points. The length of the arrow represents the gradient modulus of the pixel point, and the direction of the arrow represents the gradient direction of the pixel point. The area where the 64 small squares are located is divided into four equal parts as shown in the left diagram of fig. 5, then gradient direction histograms in eight directions of each module are calculated as shown in the right diagram of fig. 5, and an accumulated value in each gradient direction is obtained to form a feature point. Therefore, each feature point contains coordinate information. The actual sub-matching of the descriptors of the gradient vectors around the MARK point is the subsequent image matching. The SURF algorithm adopted by the feature extraction algorithm can be realized by using an FPGA. Therefore, the robustness of the features of the MARK points can be ensured on the basis of ensuring the real-time performance, and a foundation is laid for the subsequent image matching.
Image matching:
as shown in fig. 4, in the stereoscopic vision, image matching is performed by associating image points al (ul, vl) and ar (ur, vr) of a point a (X, Y, Z) in the three-dimensional space on the imaging planes Cl and Cr of the left and right cameras. I.e. depth information can be obtained from the two-dimensional image. Image matching is the most important and difficult problem in stereo vision.
After the feature points are detected and described, the feature points between the two images are matched, namely, the relationship of the feature points between the images shot by the left camera and the right camera of the binocular camera, namely C1 and C2, is found. When the feature points in the MARKi image are matched, the Euclidean distance is used as a similarity measure, one feature point in the image is taken, N feature points (wherein N is a natural number) nearest to the reference Euclidean distance are found out according to a threshold value, comparison is carried out, and if more than 60% of the feature points are smaller than the threshold value, the feature points are considered to be matched. The software can adjust the threshold value, when the threshold value is set to be reduced to a certain value, the number of the matching points is reduced, and when group judgment can be carried out, the matching is more stable and reliable.
And finally, resolving the three-dimensional space position of the MARK point according to the relation between the image coordinate and the space coordinate of the matching point.
Specifically, as shown in fig. 6, the software processing module of the system data of the present invention includes an FPGA hardware logic module and a high-speed DSP floating point calculation module. The FPGA hardware logic module comprises a video acquisition preprocessing module, a first group of managers and a MARKi synchronous control module. And video data acquired by the binocular camera is input into the video acquisition preprocessing module. And the output data of the video acquisition preprocessing module is sent to the first group manager. The first group manager is controlled by the MARKi synchronization control module. The MARKi synchronous control module is connected to a real-time control bus of the system through a real-time control bus communication protocol. The first group of managers comprises a synchronization frame taking MARKi image module, a MARKi feature extraction (SURF algorithm) module, a MARKi image matching module and a MARKi image coordinate calculating module.
The high-speed DSP floating point calculation module comprises a system calibration module, a calibration parameter module, a second group of managers and a VIO information fusion module. The system calibration module, the calibration parameter module, the second group manager and the VIO information fusion module are sequentially connected. The second group of manager modules comprise a MARKi space coordinate resolving module, an HTi space coordinate and attitude resolving module and an IMUi attitude information module.
And the data of the MARKi image coordinate calculating module in the first group of managers is sent to the MARKi space coordinate calculating module in the second group of managers. And the MARKi space coordinate calculation module in the second group of managers sends the processed data to the HTi space coordinate and attitude calculation module. The MARKi feature points constitute the HTi space. And the IMUi attitude information module acquires IMUi attitude information from a system real-time control bus. And the VIO information fusion module simultaneously acquires the output data of the HTi space coordinate and attitude calculation module and the IMUi attitude information module. And the VIO information fusion module outputs the reported information.
The described embodiments are susceptible to various modifications and alternative forms, and specific embodiments thereof have been shown by way of example in the drawings and are herein described in detail. It should be understood, however, that the embodiments are not to be limited to the particular forms or methods disclosed, but to the contrary, the intention is to cover all modifications, equivalents, and alternatives.
Claims (8)
1. A real-time positioning device based on a machine vision system, comprising: the FPGA hardware logic module and the high-speed DSP floating point calculation module;
the FPGA hardware logic module comprises a video acquisition preprocessing module, a first group manager and a MARKi synchronous control module; video data acquired by a stereoscopic vision camera is input into a video acquisition preprocessing module; the output data of the video acquisition preprocessing module is sent to a first group manager; the first group manager is controlled by a MARKi synchronous control module; the MARKi synchronous control module is connected to a real-time control bus of the system through a real-time control bus communication protocol;
the high-speed DSP floating point calculation module comprises a system calibration module, a calibration parameter module, a second group manager and a VIO information fusion module which are sequentially connected;
the first group of managers comprises a synchronization frame MARKi image taking module, a MARKi feature extraction module, a MARKi image matching module and a MARKi image coordinate calculating module;
the second group of manager modules comprise a MARKi space coordinate resolving module, an HTi space coordinate and attitude resolving module and an IMUi attitude information module; the MARKi characteristic points form an HTi space;
data of a MARKi image coordinate calculating module in the first group of managers is sent to a MARKi space coordinate calculating module in the second group of managers; the MARKi space coordinate resolving module in the second group of managers sends the processed data to the HTi space coordinate and attitude resolving module; the IMUi attitude information module acquires IMUi attitude information from a system real-time control bus; the VIO information fusion module simultaneously acquires output data of the HTi space coordinate and attitude calculation module and the IMUi attitude information module; and the VIO information fusion module outputs the reported information.
2. A real-time positioning system based on a machine vision system, comprising: a stereoscopic vision camera, a VR headset unit, a stereoscopic vision processing board and a computer, the stereoscopic vision processing board comprising a real-time positioning device based on a machine vision system as claimed in claim 1.
3. The system of claim 2, wherein: the system sends data acquired by an IMU (inertial measurement unit) and a MARK (MARK synchronization unit) arranged on each VR (virtual reality) helmet unit to a stereoscopic vision processing board through a real-time control bus of the system; the stereoscopic vision processing board obtains the video acquisition information through the stereoscopic vision camera and fuses IMU information obtained by the IMU module to obtain pose information.
4. The system of claim 2, wherein: the system carries out filtering processing on the digital image acquired by the stereoscopic vision camera through a rapid median filtering algorithm.
5. The system of claim 2, wherein: the system takes a synchronization frame MARKi image to carry out feature extraction, integrates the image of a feature region by adopting an SURF algorithm, carries out convolution operation to obtain the harr feature of the MARK point, calculates the multidimensional vector and the main feature vector of the harr feature and forms the feature descriptor of the MARK point.
6. The system of claim 2, wherein: the system adopts Euclidean distance as similarity measurement, one feature point in an image is taken, N feature points which are nearest to the reference Euclidean distance are found out according to a threshold value, wherein N is a natural number, comparison is carried out, and if more than 60% of feature points are smaller than the threshold value, the feature points are considered to be matched; and resolving the three-dimensional space position of the MARK point according to the relation between the image coordinate and the space coordinate of the matching point.
7. The system of claim 2, wherein: the stereoscopic vision camera is a binocular camera assembly.
8. The system of claim 2, wherein: the stereoscopic vision processing board sends the pose information to the computer through the USB interface.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910420567.7A CN110189267B (en) | 2019-05-21 | 2019-05-21 | Real-time positioning device and system based on machine vision |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910420567.7A CN110189267B (en) | 2019-05-21 | 2019-05-21 | Real-time positioning device and system based on machine vision |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110189267A CN110189267A (en) | 2019-08-30 |
| CN110189267B true CN110189267B (en) | 2020-12-15 |
Family
ID=67716883
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910420567.7A Active CN110189267B (en) | 2019-05-21 | 2019-05-21 | Real-time positioning device and system based on machine vision |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110189267B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110926463A (en) * | 2019-11-20 | 2020-03-27 | 深圳市优必选科技股份有限公司 | Helmet and positioning monitoring management system |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20150375091A1 (en) * | 2012-07-16 | 2015-12-31 | Fleet Us Llc | Line marking apparatus with distance measurement |
| CN108627157A (en) * | 2018-05-11 | 2018-10-09 | 重庆爱奇艺智能科技有限公司 | A kind of head based on three-dimensional marking plate shows localization method, device and three-dimensional marking plate |
| CN109115204A (en) * | 2018-09-30 | 2019-01-01 | 四川福德机器人股份有限公司 | A kind of fine positioning system and method for navigation vehicle |
-
2019
- 2019-05-21 CN CN201910420567.7A patent/CN110189267B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20150375091A1 (en) * | 2012-07-16 | 2015-12-31 | Fleet Us Llc | Line marking apparatus with distance measurement |
| CN108627157A (en) * | 2018-05-11 | 2018-10-09 | 重庆爱奇艺智能科技有限公司 | A kind of head based on three-dimensional marking plate shows localization method, device and three-dimensional marking plate |
| CN109115204A (en) * | 2018-09-30 | 2019-01-01 | 四川福德机器人股份有限公司 | A kind of fine positioning system and method for navigation vehicle |
Non-Patent Citations (2)
| Title |
|---|
| "基于FPGA的图像采集系统的设计与实现";杨超宇;《中国优秀硕士学位论文全文数据库 信息科技辑》;20160215(第 02 期);第11，18，41页 * |
| "复杂室内环境下仿人机器人定位与运动规划研究";许宪东;《中国博士学位论文全文数据库 信息科技辑》;20190115(第 01 期);第17，33-35，40，49，68，72页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110189267A (en) | 2019-08-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109166149B (en) | Positioning and three-dimensional line frame structure reconstruction method and system integrating binocular camera and IMU | |
| CN110426051B (en) | Lane line drawing method and device and storage medium | |
| CN110411441B (en) | System and method for multi-modal mapping and localization | |
| CN110555889B (en) | CALTag and point cloud information-based depth camera hand-eye calibration method | |
| CN104463108B (en) | A kind of monocular real time target recognitio and pose measuring method | |
| WO2017077925A1 (en) | Method and system for estimating three-dimensional pose of sensor | |
| EP3114647A2 (en) | Method and system for 3d capture based on structure from motion with simplified pose detection | |
| CN111768449B (en) | Object grabbing method combining binocular vision with deep learning | |
| CN111709973A (en) | Target tracking method, device, equipment and storage medium | |
| CN112419497A (en) | Monocular vision-based SLAM method combining feature method and direct method | |
| CN112541973B (en) | Virtual-real superposition method and system | |
| CN108876852A (en) | A kind of online real-time object identification localization method based on 3D vision | |
| CN114119739A (en) | Binocular vision-based hand key point space coordinate acquisition method | |
| CN105809664B (en) | Method and device for generating three-dimensional image | |
| CN114022560A (en) | Calibration method and related device and equipment | |
| Islam et al. | Stereo vision-based 3D positioning and tracking | |
| Grest et al. | Single view motion tracking by depth and silhouette information | |
| CN109474817B (en) | Optical sensing device, method and optical detection module | |
| EP3035242B1 (en) | Method and electronic device for object tracking in a light-field capture | |
| JP6922348B2 (en) | Information processing equipment, methods, and programs | |
| CN110189267B (en) | Real-time positioning device and system based on machine vision | |
| CN110197104B (en) | Distance measurement method and device based on vehicle | |
| JP6924455B1 (en) | Trajectory calculation device, trajectory calculation method, trajectory calculation program | |
| CN116580107A (en) | Cross-view multi-target real-time track tracking method and system | |
| CN105447007B (en) | A kind of electronic equipment and data processing method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |