CN113532424A - Integrated equipment for acquiring multidimensional information and cooperative measurement method - Google Patents
Integrated equipment for acquiring multidimensional information and cooperative measurement method Download PDFInfo
- Publication number
- CN113532424A CN113532424A CN202110911933.6A CN202110911933A CN113532424A CN 113532424 A CN113532424 A CN 113532424A CN 202110911933 A CN202110911933 A CN 202110911933A CN 113532424 A CN113532424 A CN 113532424A
- Authority
- CN
- China
- Prior art keywords
- geomagnetic
- indoor
- information
- reference point
- user
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000691 measurement method Methods 0.000 title abstract description 6
- 238000000034 method Methods 0.000 claims abstract description 33
- 238000001514 detection method Methods 0.000 claims abstract description 19
- 238000005259 measurement Methods 0.000 claims abstract description 19
- 238000010219 correlation analysis Methods 0.000 claims abstract description 9
- 230000000694 effects Effects 0.000 claims description 35
- 238000012545 processing Methods 0.000 claims description 12
- 238000007499 fusion processing Methods 0.000 claims description 5
- 238000003062 neural network model Methods 0.000 claims description 5
- 230000009467 reduction Effects 0.000 claims description 5
- 230000009471 action Effects 0.000 claims description 4
- 238000013500 data storage Methods 0.000 claims description 4
- 238000002372 labelling Methods 0.000 claims description 4
- 238000007781 pre-processing Methods 0.000 claims description 4
- 230000008569 process Effects 0.000 claims description 4
- 238000004364 calculation method Methods 0.000 claims description 3
- 238000013528 artificial neural network Methods 0.000 claims description 2
- 238000010276 construction Methods 0.000 claims description 2
- 238000005520 cutting process Methods 0.000 claims description 2
- 238000003384 imaging method Methods 0.000 claims description 2
- 230000010354 integration Effects 0.000 claims 2
- 238000003860 storage Methods 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 3
- 238000010586 diagram Methods 0.000 description 2
- 230000004927 fusion Effects 0.000 description 2
- 230000005358 geomagnetic field Effects 0.000 description 2
- 238000013507 mapping Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000004590 computer program Methods 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 238000009826 distribution Methods 0.000 description 1
- 239000000835 fiber Substances 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 230000002452 interceptive effect Effects 0.000 description 1
- 239000012528 membrane Substances 0.000 description 1
- 239000002184 metal Substances 0.000 description 1
- 229910052751 metal Inorganic materials 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000003825 pressing Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
- G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
- G01C21/005—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 with correlation of navigation data from several sources, e.g. map or contour matching
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
- G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
- G01C21/04—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by terrestrial means
- G01C21/08—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by terrestrial means involving use of the magnetic field of the earth
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
- G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
- G01C21/20—Instruments for performing navigational calculations
- G01C21/206—Instruments for performing navigational calculations specially adapted for indoor navigation
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
- G01S17/02—Systems using the reflection of electromagnetic waves other than radio waves
- G01S17/06—Systems determining position data of a target
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
- G01S17/02—Systems using the reflection of electromagnetic waves other than radio waves
- G01S17/50—Systems of measurement based on relative movement of target
- G01S17/58—Velocity or trajectory determination systems; Sense-of-movement determination systems
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
- G01S17/86—Combinations of lidar systems with systems other than lidar, radar or sonar, e.g. with direction finders
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/11—Region-based segmentation
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Remote Sensing (AREA)
- Radar, Positioning & Navigation (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Electromagnetism (AREA)
- Automation & Control Theory (AREA)
- Life Sciences & Earth Sciences (AREA)
- Software Systems (AREA)
- Computer Networks & Wireless Communication (AREA)
- Computational Linguistics (AREA)
- General Health & Medical Sciences (AREA)
- Mathematical Physics (AREA)
- General Engineering & Computer Science (AREA)
- Computing Systems (AREA)
- Molecular Biology (AREA)
- Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Evolutionary Computation (AREA)
- Data Mining & Analysis (AREA)
- Environmental & Geological Engineering (AREA)
- Computer Graphics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Life Sciences & Earth Sciences (AREA)
- Geology (AREA)
- Geometry (AREA)
- Optical Radar Systems And Details Thereof (AREA)
Abstract
The application provides an integrated device and a cooperative measurement method for acquiring indoor multidimensional information, wherein the device comprises: the System comprises a spatial information scanning module, a GNSS (Global Navigation Satellite System) signal receiving module, a geomagnetic signal detection module and a data memory; a method for cooperatively measuring space three-dimensional information and geomagnetic information based on a laser radar and geomagnetic integrated device comprises the steps of establishing an indoor three-dimensional model, establishing a geomagnetic database on a user movement path based on the indoor three-dimensional model, carrying out correlation analysis on a geomagnetic reference point geomagnetic intensity information sequence measured in real time and a geomagnetic reference point information sequence in the geomagnetic database, and estimating position coordinates and movement tracks of a user by readjusting the geomagnetic and the laser radar to verify the geomagnetic reference point geomagnetic intensity information sequence and the geomagnetic reference point information sequence mutually when the measured indoor environment is easily interfered by the outside, so that indoor movement measurement is realized, and the efficiency and accuracy of scanning of the indoor space three-dimensional information and the geomagnetic information are improved.
Description
Technical Field
The invention relates to the field of laser radar and geomagnetic measurement, in particular to a method and equipment for measuring three-dimensional space information and geomagnetic information cooperatively based on laser radar and geomagnetic integrated equipment.
Background
With the development of modern society, the demand for personalized location information services has grown substantially. The outdoor multidimensional information can be used for accurately positioning to the meter through a Global Navigation Satellite System (GNSS) such as a GPS and a Beidou; however, GNSS signals in an indoor environment cannot be directly acquired, and thus cannot be accurately located. For modern people, more than 80% of the activity space is concentrated indoors. Therefore, the need for accuracy and efficiency based indoor positioning techniques is more and more pressing. Among a plurality of technologies capable of providing indoor positioning information, the geomagnetic positioning is utilized to have the advantages of no need of laying a positioning information source, low cost and wide application to large buildings. But geomagnetism may change due to the passing of the vehicle, resulting in inaccurate measurement.
Disclosure of Invention
In view of this, the present application provides an integrated device and a cooperative measurement method for acquiring indoor multidimensional information, when a measured indoor environment is easily interfered by the outside: for example, when adjacent rooms are used, the geomagnetic and the laser radar are readjusted to be mutually verified, so that the efficiency and the accuracy of scanning of indoor space three-dimensional information and geomagnetic information are improved.
A method for cooperatively measuring space three-dimensional information and geomagnetic information based on a laser radar and geomagnetic integrated device comprises the steps of establishing an indoor three-dimensional model, establishing a geomagnetic database on a user movement path based on the indoor three-dimensional model, carrying out correlation analysis on a geomagnetic reference point geomagnetic intensity information sequence measured in real time and a geomagnetic reference point information sequence in the geomagnetic database, and estimating position coordinates and movement tracks of a user, so that indoor movement measurement is realized.
The method specifically comprises the following steps:
step 1, acquiring indoor point cloud data and a high-resolution indoor image by using the laser radar and geomagnetic integrated equipment;
step 2, performing multi-mode fusion processing on the point cloud data and the high-resolution indoor image to acquire image data which has both depth value information and high resolution;
step 3, acquiring an image data set of indoor common articles, and constructing a deep neural network model RCNN; carrying out target detection on the high-resolution indoor image through an RCNN model so as to obtain indoor article information;
step 4, preprocessing the point cloud data and reconstructing a real three-dimensional model of an indoor scene;
step 5, according to the image data which has both depth value information and high resolution, marking the article at the corresponding position of the three-dimensional model in the step 4, namely endowing article information to the indoor three-dimensional model;
step 6, constructing a geomagnetic database on a user activity path;
and 7, carrying out correlation analysis on the geomagnetic intensity information sequence of the geomagnetic reference point measured in real time and the geomagnetic reference point information sequence in the geomagnetic database in the step 6, and estimating the position coordinates and the action track of the user so as to realize indoor movement measurement.
The application provides an integrated equipment of indoor multidimension information obtains, includes: the System comprises a spatial information scanning module, a GNSS (Global Navigation Satellite System) signal receiving module, a geomagnetic signal detection module and a data memory;
the output ends of the spatial information scanning module, the GNSS signal receiving module and the geomagnetic signal detection module are respectively connected with the data memory; the signal processing module is also connected with an input/output interface and a dual-computer cooperative interface module.
The beneficial technical effects are as follows: the invention provides an integrated device for acquiring multi-dimensional information and a cooperative measurement method, which solve the problems that the geomagnetic field is inaccurate in measurement due to external interference, such as passing of a vehicle, and the like in the existing geomagnetic detection technology, readjust the geomagnetic field and the laser radar and verify the geomagnetic radar and the geomagnetic radar mutually, so that the geomagnetic measurement is more convenient, errors of the geomagnetic measurement are avoided, and the efficiency and the accuracy of scanning of indoor space three-dimensional information and geomagnetic information are improved.
Drawings
FIG. 1 is a flow chart of a method provided by an embodiment of the invention
FIG. 2 is a schematic structural diagram of an apparatus according to an embodiment of the present invention
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is further described in detail with reference to the following embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
The invention adopts the geomagnetic matching and positioning technology to carry out correlation analysis on the geomagnetic intensity information sequence of the geomagnetic reference point measured in real time and the geomagnetic reference point information sequence in the pre-established geomagnetic database, and estimates the position coordinates and the action track of the user, thereby realizing indoor movement measurement. The indoor mobile measurement method provided by the invention avoids errors of geomagnetic measurement based on the geomagnetic database on the pre-constructed user activity path and the geomagnetic intensity information sequence of the geomagnetic reference point measured in real time, and improves the efficiency and accuracy of scanning indoor space three-dimensional information and geomagnetic information.
Fig. 1 is a flowchart of a method for cooperatively measuring three-dimensional information and geomagnetic information in a space based on a lidar and a geomagnetic integrated device according to an embodiment of the present invention, where the flowchart of the present application is further explained by a structure diagram of an integrated device for acquiring indoor multidimensional information in fig. 2, where the integrated device for acquiring indoor multidimensional information includes: the System comprises a spatial information scanning module, a GNSS (Global Navigation Satellite System) signal receiving module, a geomagnetic signal detection module and a data memory; the output ends of the spatial information scanning module, the GNSS signal receiving module and the geomagnetic signal detection module are respectively connected with the data memory; the signal processing module is also connected with an input/output interface and a dual-computer cooperative interface module.
The method for cooperatively measuring space three-dimensional information and geomagnetic information based on laser radar and geomagnetic integrated equipment comprises the steps of establishing an indoor three-dimensional model, constructing a geomagnetic database on a user movement path based on the indoor three-dimensional model, carrying out correlation analysis on a geomagnetic reference point geomagnetic intensity information sequence measured in real time and a geomagnetic reference point information sequence in the geomagnetic database, and estimating position coordinates and movement tracks of a user, so that indoor movement measurement is realized, and specifically comprises the following steps:
step 1, acquiring indoor point cloud data and a high-resolution indoor image by using the laser radar and geomagnetic integrated equipment;
wherein the step 1 comprises:
step S11, placing the laser radar and geomagnetic integrated equipment at an indoor central position, and adjusting the height of the laser radar and geomagnetic integrated equipment by adjusting a base tripod of the laser radar and geomagnetic integrated equipment;
step S22, scanning and imaging a target object by using a spatial information scanning module in the laser radar and geomagnetic integrated equipment to obtain high-density and high-precision laser point cloud data and a target image with high target resolution; and storing the three-dimensional point cloud data and the image data in a data storage.
Step 2, performing multi-mode fusion processing on the point cloud data and the high-resolution indoor image to acquire image data which has both depth value information and high resolution;
with the current intensive research on indoor positioning, a single sensor has no way to fully cope with a complex real-world environment, and in order to solve the problems of detection accuracy and robustness, a multi-modal method is the most optimal solution at present. The multi-modal fusion is a place where a plurality of kinds of sensor data are taken as input of a deep learning network, and information acquired by a plurality of kinds of sensors is complemented by the multi-modal fusion. The invention carries out multi-mode fusion processing on the point cloud data acquired in the step 1 and the target image with high resolution to acquire image data which has both depth value information and high resolution. The method realizes the accurate matching of corresponding RGB values on corresponding pixels of the point cloud and the image based on a collinearity equation; the collinearity equation is:
wherein, (X, Y, Z) is the coordinates of the laser point on the image plane, and (X, Y) to be solved is the corresponding pixel coordinates of the laser point under the coordinate system of the image plane; xs, Ys and Zs are coordinates of a projection center in an image plane coordinate system, and f is a camera main distance; a1-a3, b1-b3 and c1-c3 are transform coefficients.
Step 3, acquiring an image data set of indoor common articles, and constructing a deep neural network model RCNN; carrying out target detection on the high-resolution indoor image through an RCNN model so as to obtain indoor article information;
wherein the step 3 comprises:
step S31, generating candidate regions: dividing an image into small regions, and combining two regions with highest probability in the small regions according to a combination rule: the merging rule is that areas with similar color and texture are merged preferentially, and the total area of the areas is reduced after merging; and repeating the merging operation until the whole image is merged into one region position, generating 2000-3000 candidate regions by one image, and outputting the candidate region generation.
Step S32, extracting features of the candidate regions, normalizing the image into 224 × 224 input neural network model, wherein the extracted features of the neural network are 4096 dimensions;
step S33, sending 4096-dimensional features output by the deep network into a full connection layer for classification; and (4) independently allocating a linear SVM classifier for each class of targets to judge and output a classification result.
Step 4, preprocessing the point cloud data and reconstructing a real three-dimensional model of an indoor scene;
the unprocessed point cloud data is scattered and disordered and cannot be modeled, and the data can be imported into 3D Max software for modeling only after the point cloud splicing, merging, noise reduction and other processing. The existing ground three-dimensional laser scanner generally has post-processing software of data, the data processing software HD _3LS _ SCENE of the Zhonghaida scanner is adopted in the invention, the data is imported into the HD _3LS _ SCENE software, and the corresponding point cloud splicing, merging, noise reduction and other processing are carried out, so that high-quality point cloud modeling data can be obtained.
In the processing process, the cloud of each station is spliced, and the splicing precision among the stations is 0.05 m. And secondly, point cloud merging, wherein an overlapping area of more than 10% exists between two adjacent stations, and the data volume can be effectively reduced under the condition that the structure is not changed by point cloud merging. And finally, point cloud noise reduction is carried out, and point clouds irrelevant to the object are deleted and removed, so that the point cloud quality is improved.
The method comprises the step of S41, point cloud splicing, wherein the three-dimensional laser scanner adopts a substation type operation mode, each acquired station data has an independent coordinate system, and if all-directional point cloud data are acquired, the point cloud data need to be registered to a uniform coordinate system. After the point cloud data obtained by field scanning is imported into software, data splicing is firstly carried out, namely the point cloud data obtained by multi-station scanning is converted into the same coordinate system, and point cloud splicing in a public overlapping area is adopted to form a whole. Commonly used splicing methods are: target splicing, homonymous point splicing, view splicing, mixed splicing of a target and a homonymous point, and splicing of known control points. The method adopts mixed splicing of targets and homologous points, and then splicing is carried out in HD _3LS _ SCENE software. When splicing, two survey stations are spliced firstly, then the spliced result is spliced with the spliced result of the other two stations,
and step S42, point cloud merging, wherein a certain overlapping area is generated due to substation type scanning, the distance between a scanner and a target also influences the scanned data volume, generally, the closer the distance is, the larger the point cloud density is, the farther the distance is, the smaller the point cloud density is, the data volume is not reduced in the data splicing process, and data redundancy can also be caused. And the acquired point cloud data volume is too large, the requirement on hardware facilities is high, and the data processing efficiency is reduced. Therefore, it is necessary to reduce the point cloud data, i.e., merge the overlapping areas, without affecting the data accuracy. In HD _3LS _ SCENE software, a merging command is executed, the system automatically generates simplified point cloud data, and after point cloud merging operation, the original structure is not influenced, the data storage space is reduced, and the data processing efficiency is improved.
And step S43, carrying out point cloud noise reduction, wherein the original point cloud data obtained by the three-dimensional laser scanner is a scattered spatial data set. During scanning, the obtained point cloud data is influenced by many factors of the surrounding environment. Although the point cloud data is greatly deleted after point cloud merging processing, the original texture structure of the original data is not changed by merging, and the original noise points still exist. The noise points not only occupy the storage space and increase the processing time of a computer, but also have influence on the segmentation, identification, modeling and the like of the point cloud and are rejected. In HD _3LS _ SCENE software, large-range noise point clouds are deleted artificially, scattered noise points are deleted by a method for automatically identifying the noise points through the software, different visual angles are switched through the interactive operation of the manual work and computer software, and all the noise points are removed.
The three-dimensional modeling mainly comprises two parts, namely three-dimensional geometric modeling and texture mapping, wherein the three-dimensional geometric modeling constructs an indoor integral structure, and the texture mapping can construct a three-dimensional model with a sense of reality. The mosaic modeling is different from the traditional measurement size modeling, and in a high version of 3D Max 2015 or above, the spliced point cloud data can be imported into a working surface, and then a reasonable mosaic module is designed according to the structural characteristics of the spliced point cloud to perform block modeling. Block modeling can construct details of complex modules well without destroying the overall structure. After the mosaic and block modeling, all block models are combined to construct a complete three-dimensional model, and finally the point cloud data is hidden, so that a clear white membrane can be obtained.
And 5, marking the article at the corresponding position of the three-dimensional model in the step 4 according to the image data which has both the depth value information and the high resolution, namely endowing the article information to the indoor three-dimensional model.
Wherein the step 5 comprises:
step S51, correcting the high-resolution indoor image by using Photoshop software, converting the image into an orthographic projection image, and performing processing such as cutting, removing mottle, balancing tone and the like;
and step S52, pasting the picture on the white film of the corresponding article according to the target detection result through 3D Max software, correspondingly adding UVW (ultraviolet ray) picture pasting coordinates if the picture is distorted and dislocated, and adjusting texture coordinates and model coordinates to enable the geometric model to be matched with the texture data.
Step 6, constructing a geomagnetic database on a user activity path;
wherein step 6 comprises:
step S61, based on the indoor three-dimensional model position data information, labeling the activity path of the indoor user;
based on the indoor three-dimensional model position information, counting all possible activity paths of the user, wherein the user activity path data expression is as follows:
wherein Index is the unique identification number of the user activity path,
different nodes on the user activity path, I nodes are shared by the user activity paths under Index labelsAnd (4) point.
The method also comprises the step of determining a geomagnetic reference point k of a geomagnetic database between two adjacent nodes according to a preset parameter d, firstly obtaining the total number N of the geomagnetic reference points between the two adjacent nodes and the distance L between the two adjacent nodes, and calculating the formula as follows,
wherein (A), (B), (C), (D), (C), (B), (C)
) And (a)
) Coordinates of two adjacent nodes on two user activity paths;
the spatial position coordinates of the geomagnetic reference point should (
) Comprises the following steps:
step S62, generating a geomagnetic reference point according to the activity path, calculating the position coordinates of the geomagnetic reference point, and acquiring the geomagnetic intensity of the point;
generating a geomagnetic reference point according to the moving path, and calculating a position coordinate of the geomagnetic reference point, wherein acquiring the geomagnetic intensity of the point comprises: after acquiring the space coordinates of the geomagnetic reference point, simulating the indoor activities of the user, and acquiring geomagnetic intensity signals at the geomagnetic reference point on the user activity path through the carried geomagnetic signal detection module
Step S63, constructing a geomagnetic database on a user activity path based on the position coordinates of the geomagnetic reference point and the geomagnetic intensity;
the constructing of the geomagnetic database on the user activity path includes: and binding the space coordinates and the geomagnetic strength of the geomagnetic reference point, and storing the space coordinates and the geomagnetic strength into a geomagnetic database of a server by taking the user activity path as a unit.
And 7, carrying out correlation analysis on the geomagnetic intensity information sequence of the geomagnetic reference point measured in real time and the geomagnetic reference point information sequence in the geomagnetic database in the step 6, and estimating the position coordinates and the action track of the user so as to realize indoor movement measurement.
When a user carries the smart phone to move indoors, a built-in geomagnetic detection module of the smart phone collects geomagnetic intensity information K (t) in real time. Since the geomagnetic signal itself is easily interfered by metal objects, if a vehicle suddenly passes by the user, the geomagnetic intensity is inevitably affected, and therefore the collected geomagnetic intensity k (t) may not be matched with the geomagnetic sequence in the geomagnetic database constructed in step 6. Then, the interference is instantaneous, and the interference of a driving automobile and the like cannot exist for a long time, that is, the collected geomagnetic intensity sequence k (t) is similar to a certain sequence in the geomagnetic database in rough distribution. Based on the method, the KL divergence is used for calculating the similarity between the geomagnetic intensity sequence K (t) detected in real time and the sequence M (t) in the geomagnetic database, so that the space coordinates of the corresponding sequence in the database are obtained, and the indoor movement measurement is realized.
Wherein step 7 comprises:
step S71, collecting geomagnetic intensity information K (t) in real time by using a built-in geomagnetic detection module of the carried smart phone in the indoor moving process of the user carrying the smart phone;
step S72, establishing a geomagnetic strength construction sequence m (t) of a geomagnetic reference point between two nodes according to a user activity path in the geomagnetic database, a node on each path, and a geomagnetic reference point between each two nodes, and performing KL divergence calculation on the geomagnetic strength sequences k (t) and m (t) collected in real time:
and N is the number of geomagnetic reference points between the two nodes, corresponding nodes are obtained according to the KL divergence values, the nodes are connected to form a path, and the possible activity path of the user where the accumulated KL divergence value is minimum is determined as the actual path of the current user, so that indoor mobile measurement is realized. When the KL divergence value is smaller than a preset threshold value g, judging that the current user movement belongs to the positions of two nodes under the corresponding path in the geomagnetic database; otherwise, the device server side continuously searches and selects two node positions with the minimum KL divergence value as corresponding nodes.
In the above embodiments, the implementation may be wholly or partially realized by software, hardware, firmware, or any combination thereof. When used in whole or in part, can be implemented in a computer program product that includes one or more computer instructions. When loaded or executed on a computer, cause the flow or functions according to embodiments of the invention to occur, in whole or in part. The computer may be a general purpose computer, a special purpose computer, a network of computers, or other programmable device. The computer instructions may be stored in a computer readable storage medium or transmitted from one computer readable storage medium to another, for example, the computer instructions may be transmitted from one website site, computer, server, or data center to another website site, computer, server, or data center via wire (e.g., coaxial cable, fiber optic, Digital Subscriber Line (DSL), or wireless (e.g., infrared, wireless, microwave, etc.)). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device, such as a server, a data center, etc., that includes one or more of the available media. The usable medium may be a magnetic medium (e.g., floppy Disk, hard Disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., Solid State Disk (SSD)), among others.
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents and improvements made within the spirit and principle of the present invention are intended to be included within the scope of the present invention.
Claims (10)
1. A method for cooperatively measuring space three-dimensional information and geomagnetic information based on a laser radar and geomagnetic integrated device comprises the steps of establishing an indoor three-dimensional model, constructing a geomagnetic database on a user movement path based on the indoor three-dimensional model, carrying out correlation analysis on a geomagnetic reference point geomagnetic intensity information sequence measured in real time and a geomagnetic reference point information sequence in the geomagnetic database, and estimating position coordinates and movement tracks of a user, so that indoor movement measurement is realized.
The building of the indoor three-dimensional model comprises the following steps:
step 1, acquiring indoor point cloud data and a high-resolution indoor image by using the laser radar and geomagnetic integrated equipment;
step 2, performing multi-mode fusion processing on the point cloud data and the high-resolution indoor image to acquire image data which has both depth value information and high resolution;
step 3, acquiring an image data set of indoor common articles, and constructing a deep neural network model RCNN; carrying out target detection on the high-resolution indoor image through an RCNN model so as to obtain indoor article information;
step 4, preprocessing the point cloud data and reconstructing a real three-dimensional model of an indoor scene;
step 5, according to the image data which has both depth value information and high resolution, marking the article at the corresponding position of the three-dimensional model in the step 4, namely endowing article information to the indoor three-dimensional model;
step 6, constructing a geomagnetic database on a user activity path;
step 7, performing correlation analysis on the geomagnetic intensity information sequence of the geomagnetic reference point measured in real time and the geomagnetic reference point information sequence in the geomagnetic database in the step 6, when a vehicle passes through the geomagnetic intensity variation, establishing a geomagnetic intensity construction sequence M (t) of the geomagnetic reference point between two nodes according to the user activity path in the geomagnetic database, the nodes on each path and the geomagnetic reference point between every two nodes, and performing KL divergence calculation on the geomagnetic intensity sequences K (t) and M (t) collected in real time:
the method comprises the following steps that N is the number of geomagnetic reference points between two nodes, corresponding nodes are obtained according to KL divergence values, the nodes are connected to form a path, and a possible activity path of a user where the accumulated KL divergence value is the smallest is determined as an actual path of the current user, wherein the method for obtaining the nodes comprises the following steps: when the KL divergence value is smaller than a preset threshold value g, judging that the current user movement belongs to the positions of two nodes under the corresponding path in the geomagnetic database; otherwise, the equipment server side continuously searches and selects two node positions with the minimum KL divergence value as corresponding nodes; and finally, estimating the position coordinates and the action track of the user, thereby realizing indoor movement measurement.
2. The method for measuring three-dimensional spatial information and geomagnetic information based on cooperation of lidar and geomagnetic integration equipment according to claim 1, wherein the step 1 of acquiring the indoor point cloud data and the high-resolution indoor image by using the lidar and geomagnetic integration equipment comprises:
step S11, placing the laser radar and geomagnetic integrated equipment at an indoor central position, and adjusting the height of the laser radar and geomagnetic integrated equipment by adjusting a base tripod of the laser radar and geomagnetic integrated equipment;
step S12, scanning and imaging a target object by using a spatial information scanning module in the laser radar and geomagnetic integrated equipment to obtain high-density and high-precision laser point cloud data and a target image with high target resolution; and storing the three-dimensional point cloud data and the image data in a data storage.
3. The method according to claim 1, wherein the step 2 of performing multi-modal fusion processing on the point cloud data and the high-resolution indoor image to obtain image data having both depth value information and high resolution includes:
based on a collinearity equation, realizing the accurate matching of corresponding RGB values on corresponding pixels of the point cloud and the image; the collinearity equation is:
wherein, (X, Y, Z) is the coordinates of the laser point on the image plane, and (X, Y) to be solved is the corresponding pixel coordinates of the laser point under the coordinate system of the image plane; xs, Ys and Zs are coordinates of a projection center in an image plane coordinate system, and f is a camera main distance;
a1-a3, b1-b3 and c1-c3 are transform coefficients.
4. The method for measuring three-dimensional spatial information and geomagnetic information based on cooperation of lidar and geomagnetic integrated equipment according to claim 1, wherein the acquiring indoor article information in step 3 comprises:
step S31, generating candidate regions: dividing an image into small regions, and combining two regions with highest probability in the small regions according to a combination rule: the merging rule is that areas with similar color and texture are merged preferentially, and the total area of the areas is reduced after merging; and repeating the merging operation until the whole image is merged into one region position, generating 2000-3000 candidate regions by one image, and outputting the candidate region generation.
Step S32, extracting features of the candidate regions, normalizing the image into 224 × 224 input neural network model, wherein the extracted features of the neural network are 4096 dimensions;
step S33, sending 4096-dimensional features output by the deep network into a full connection layer for classification; and (4) independently allocating a linear SVM classifier for each class of targets to judge and output a classification result.
5. The method for measuring three-dimensional spatial information and geomagnetic information in cooperation based on lidar and geomagnetic integrated equipment according to claim 1, wherein the preprocessing the point cloud data and reconstructing the real three-dimensional model of the indoor scene in step 4 comprises:
step S41, point cloud splicing;
step S42, point cloud merging;
and step S43, point cloud noise reduction.
6. The method for measuring three-dimensional spatial information and geomagnetic information in cooperation based on lidar and geomagnetic integrated equipment according to claim 1, wherein the step 5 of assigning the article information to the three-dimensional model comprises:
step S51, correcting the high-resolution indoor image by using Photoshop software, converting the image into an orthographic projection image, and performing processing such as cutting, removing mottle, balancing tone and the like;
and step S52, pasting the picture on the white film of the corresponding article according to the target detection result through 3D Max software, correspondingly adding UVW (ultraviolet ray) picture pasting coordinates if the picture is distorted and dislocated, and adjusting texture coordinates and model coordinates to enable the geometric model to be matched with the texture data.
7. The method for measuring three-dimensional spatial information and geomagnetic information based on the lidar and geomagnetic integrated equipment and the geomagnetic integrated equipment according to claim 1, wherein the step 6 comprises:
step S61, based on the indoor three-dimensional model position data information, labeling the activity path of the indoor user;
step S62, generating a geomagnetic reference point according to the activity path, calculating the position coordinates of the geomagnetic reference point, and acquiring the geomagnetic intensity of the point;
step S63, constructing a geomagnetic database on a user activity path based on the position coordinates of the geomagnetic reference point and the geomagnetic intensity;
the step S61 of labeling the activity path of the user indoors includes:
based on the indoor three-dimensional model position information, counting all possible activity paths of the user, wherein the user activity path data expression is as follows:
{Index,{(xi,yi)node,i＝1,2...I}}
where Index is the unique identification number of the user's active path, (x)i,yi)nodeThe nodes are different nodes on the user activity path, and the user activity path under Index labels has I nodes.
8. The method for cooperatively measuring spatial three-dimensional information and geomagnetic information based on lidar and geomagnetic integrated equipment according to claim 7, further comprising:
the labeling of the activity path of the user indoors in step S61 includes acquiring a geomagnetic reference point:
determining a geomagnetic reference point k of a geomagnetic database between two adjacent nodes according to a preset parameter d, first obtaining a total number N of the geomagnetic reference points between the two adjacent nodes and a distance L between the two adjacent nodes, wherein a calculation formula is as follows,
wherein (x)node1,ynode1) And (x)node1,ynode1) Is two usersCoordinates of two adjacent nodes on the active path;
spatial position coordinates of geomagnetic reference point should (x)k,yk) Comprises the following steps:
in step S62, generating a geomagnetic reference point according to the motion path, and calculating a position coordinate of the geomagnetic reference point, wherein acquiring the geomagnetic intensity of the point includes: after the space coordinates of the geomagnetic reference point are obtained, the indoor activities of the user are simulated, and geomagnetic intensity signals are collected at the geomagnetic reference point on the user activity path through equipment of a carried geomagnetic signal detection module;
in step S63, constructing a geomagnetic database on a user movement path based on the position coordinates of the geomagnetic reference point and the geomagnetic intensity includes: and binding the space coordinates and the geomagnetic strength of the geomagnetic reference point, and storing the space coordinates and the geomagnetic strength into a geomagnetic database of a server by taking the user activity path as a unit.
9. The method of claim 1, wherein the step 7 is to perform correlation analysis on the real-time measured geomagnetic reference point and the geomagnetic reference point information sequence in the geomagnetic database to estimate the position coordinates and the movement track of the user, so as to implement indoor movement measurement, and the method includes:
step S71, in the process of moving indoors with the smart phone carried by the user, the geomagnetic intensity information k (t) is collected in real time by using the built-in geomagnetic detection module of the smart phone carried by the user.
10. An integrated device for acquiring multi-dimensional information and a cooperative measurement device, which adopt the method as claimed in claims 1-9, the integrated device for acquiring indoor multi-dimensional information comprises: the System comprises a spatial information scanning module, a GNSS (Global Navigation Satellite System) signal receiving module, a geomagnetic signal detection module and a data memory;
the output ends of the spatial information scanning module, the GNSS signal receiving module and the geomagnetic signal detection module are respectively connected with the data memory; the signal processing module is also connected with an input/output interface and a dual-computer cooperative interface module.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110911933.6A CN113532424B (en) | 2021-08-10 | 2021-08-10 | Integrated equipment for acquiring multidimensional information and cooperative measurement method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110911933.6A CN113532424B (en) | 2021-08-10 | 2021-08-10 | Integrated equipment for acquiring multidimensional information and cooperative measurement method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113532424A true CN113532424A (en) | 2021-10-22 |
| CN113532424B CN113532424B (en) | 2024-02-20 |
Family
ID=78091381
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110911933.6A Active CN113532424B (en) | 2021-08-10 | 2021-08-10 | Integrated equipment for acquiring multidimensional information and cooperative measurement method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113532424B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115291767A (en) * | 2022-08-01 | 2022-11-04 | 北京奇岱松科技有限公司 | Control method and device of Internet of things equipment, electronic equipment and storage medium |
| CN117872389A (en) * | 2024-01-19 | 2024-04-12 | 武汉万曦智能科技有限公司 | Wireless speed measuring method and system for field vehicle |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10024664B1 (en) * | 2014-09-30 | 2018-07-17 | The United States Of America As Represented By The Administrator Of The National Aeronautics And Space Administration | Range and intensity image-based terrain and vehicle relative pose estimation system |
| CN108801265A (en) * | 2018-06-08 | 2018-11-13 | 武汉大学 | Multidimensional information synchronous acquisition, positioning and position service apparatus and system and method |
| KR101971734B1 (en) * | 2018-07-26 | 2019-04-23 | 영남대학교 산학협력단 | Apparatus and method for indoor positioning |
| CN109737968A (en) * | 2019-03-07 | 2019-05-10 | 中山大学 | Indoor fusion and positioning method based on two-dimentional LiDAR and smart phone |
| CN111126304A (en) * | 2019-12-25 | 2020-05-08 | 鲁东大学 | Augmented reality navigation method based on indoor natural scene image deep learning |
| CN111156983A (en) * | 2019-11-19 | 2020-05-15 | 石化盈科信息技术有限责任公司 | Target equipment positioning method and device, storage medium and computer equipment |
| WO2020237693A1 (en) * | 2019-05-31 | 2020-12-03 | 华南理工大学 | Multi-source sensing method and system for water surface unmanned equipment |
| KR20210003065A (en) * | 2020-04-14 | 2021-01-11 | 네이버랩스 주식회사 | Method and system for collecting data |
| CN113066162A (en) * | 2021-03-12 | 2021-07-02 | 武汉大学 | Urban environment rapid modeling method for electromagnetic calculation |
-
2021
- 2021-08-10 CN CN202110911933.6A patent/CN113532424B/en active Active
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10024664B1 (en) * | 2014-09-30 | 2018-07-17 | The United States Of America As Represented By The Administrator Of The National Aeronautics And Space Administration | Range and intensity image-based terrain and vehicle relative pose estimation system |
| CN108801265A (en) * | 2018-06-08 | 2018-11-13 | 武汉大学 | Multidimensional information synchronous acquisition, positioning and position service apparatus and system and method |
| KR101971734B1 (en) * | 2018-07-26 | 2019-04-23 | 영남대학교 산학협력단 | Apparatus and method for indoor positioning |
| CN109737968A (en) * | 2019-03-07 | 2019-05-10 | 中山大学 | Indoor fusion and positioning method based on two-dimentional LiDAR and smart phone |
| WO2020237693A1 (en) * | 2019-05-31 | 2020-12-03 | 华南理工大学 | Multi-source sensing method and system for water surface unmanned equipment |
| CN111156983A (en) * | 2019-11-19 | 2020-05-15 | 石化盈科信息技术有限责任公司 | Target equipment positioning method and device, storage medium and computer equipment |
| CN111126304A (en) * | 2019-12-25 | 2020-05-08 | 鲁东大学 | Augmented reality navigation method based on indoor natural scene image deep learning |
| KR20210003065A (en) * | 2020-04-14 | 2021-01-11 | 네이버랩스 주식회사 | Method and system for collecting data |
| CN113066162A (en) * | 2021-03-12 | 2021-07-02 | 武汉大学 | Urban environment rapid modeling method for electromagnetic calculation |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115291767A (en) * | 2022-08-01 | 2022-11-04 | 北京奇岱松科技有限公司 | Control method and device of Internet of things equipment, electronic equipment and storage medium |
| CN117872389A (en) * | 2024-01-19 | 2024-04-12 | 武汉万曦智能科技有限公司 | Wireless speed measuring method and system for field vehicle |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113532424B (en) | 2024-02-20 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Chen et al. | Applicability of personal laser scanning in forestry inventory | |
| CN109285220B (en) | Three-dimensional scene map generation method, device, equipment and storage medium | |
| US10043097B2 (en) | Image abstraction system | |
| US10297074B2 (en) | Three-dimensional modeling from optical capture | |
| CN111709981A (en) | Registration method of laser point cloud and analog image with characteristic line fusion | |
| US20190026400A1 (en) | Three-dimensional modeling from point cloud data migration | |
| CN111028358B (en) | Indoor environment augmented reality display method and device and terminal equipment | |
| Chen et al. | Transforming a 3-d lidar point cloud into a 2-d dense depth map through a parameter self-adaptive framework | |
| CN112818925B (en) | Urban building and crown identification method | |
| CN102959946A (en) | Augmenting image data based on related 3d point cloud data | |
| CN111915723A (en) | Indoor three-dimensional panorama construction method and system | |
| CN111915517B (en) | Global positioning method suitable for RGB-D camera under indoor illumination unfavorable environment | |
| CN113532424B (en) | Integrated equipment for acquiring multidimensional information and cooperative measurement method | |
| Kaiser et al. | Automatic co-registration of photogrammetric point clouds with digital building models | |
| Özdemir et al. | A multi-purpose benchmark for photogrammetric urban 3D reconstruction in a controlled environment | |
| Jiang et al. | Determination of construction site elevations using drone technology | |
| CN113160292B (en) | Laser radar point cloud data three-dimensional modeling device and method based on intelligent mobile terminal | |
| CN110021041B (en) | Unmanned scene incremental gridding structure reconstruction method based on binocular camera | |
| WO2020124091A1 (en) | Automatic fine-grained radio map construction and adaptation | |
| CN117830772A (en) | Local map generation method and system based on point cloud image fusion | |
| CN116863083A (en) | Method and device for processing three-dimensional point cloud data of transformer substation | |
| CN115019167A (en) | Fusion positioning method, system, equipment and storage medium based on mobile terminal | |
| CN109813321B (en) | Indoor map drawing method and system based on user behavior analysis | |
| Habib et al. | Standards and specifications for the calibration and stability of amateur digital camaras for close-range mapping applications | |
| WO2023281797A1 (en) | Facility recognition system and facility recognition method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |