CN113532424B - Integrated equipment for acquiring multidimensional information and cooperative measurement method - Google Patents
Integrated equipment for acquiring multidimensional information and cooperative measurement method Download PDFInfo
- Publication number
- CN113532424B CN113532424B CN202110911933.6A CN202110911933A CN113532424B CN 113532424 B CN113532424 B CN 113532424B CN 202110911933 A CN202110911933 A CN 202110911933A CN 113532424 B CN113532424 B CN 113532424B
- Authority
- CN
- China
- Prior art keywords
- geomagnetic
- indoor
- information
- user
- laser radar
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000691 measurement method Methods 0.000 title abstract description 6
- 238000000034 method Methods 0.000 claims abstract description 34
- 230000000694 effects Effects 0.000 claims abstract description 30
- 238000005259 measurement Methods 0.000 claims abstract description 18
- 238000001514 detection method Methods 0.000 claims abstract description 16
- 230000009471 action Effects 0.000 claims abstract description 9
- 238000010219 correlation analysis Methods 0.000 claims abstract description 9
- 238000013500 data storage Methods 0.000 claims abstract description 6
- 238000012545 processing Methods 0.000 claims description 10
- 238000013507 mapping Methods 0.000 claims description 8
- 238000007499 fusion processing Methods 0.000 claims description 5
- 238000003062 neural network model Methods 0.000 claims description 5
- 230000008569 process Effects 0.000 claims description 5
- 238000004364 calculation method Methods 0.000 claims description 4
- 238000002372 labelling Methods 0.000 claims description 4
- 238000007781 pre-processing Methods 0.000 claims description 4
- 238000013528 artificial neural network Methods 0.000 claims description 2
- 238000010276 construction Methods 0.000 claims description 2
- 230000001186 cumulative effect Effects 0.000 claims description 2
- 238000003384 imaging method Methods 0.000 claims description 2
- 238000005516 engineering process Methods 0.000 description 3
- 230000009467 reduction Effects 0.000 description 3
- 230000008901 benefit Effects 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 230000004927 fusion Effects 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 230000000295 complement effect Effects 0.000 description 1
- 238000004590 computer program Methods 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 239000000835 fiber Substances 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 230000002452 interceptive effect Effects 0.000 description 1
- 230000005923 long-lasting effect Effects 0.000 description 1
- 239000002184 metal Substances 0.000 description 1
- 229910052751 metal Inorganic materials 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
- G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
- G01C21/005—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 with correlation of navigation data from several sources, e.g. map or contour matching
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
- G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
- G01C21/04—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by terrestrial means
- G01C21/08—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by terrestrial means involving use of the magnetic field of the earth
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
- G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
- G01C21/20—Instruments for performing navigational calculations
- G01C21/206—Instruments for performing navigational calculations specially adapted for indoor navigation
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
- G01S17/02—Systems using the reflection of electromagnetic waves other than radio waves
- G01S17/06—Systems determining position data of a target
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
- G01S17/02—Systems using the reflection of electromagnetic waves other than radio waves
- G01S17/50—Systems of measurement based on relative movement of target
- G01S17/58—Velocity or trajectory determination systems; Sense-of-movement determination systems
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
- G01S17/86—Combinations of lidar systems with systems other than lidar, radar or sonar, e.g. with direction finders
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/11—Region-based segmentation
Abstract
The application provides an integrated device and a cooperative measurement method for acquiring indoor multidimensional information, wherein the device comprises: the system comprises a space information scanning module, a GNSS (Global Navigation Satellite System ) signal receiving module, a geomagnetic signal detection module and a data storage; the method for cooperatively measuring the space three-dimensional information and the geomagnetic information based on the laser radar and geomagnetic integrated equipment comprises the steps of establishing an indoor three-dimensional model, constructing a geomagnetic database on a user activity path based on the indoor three-dimensional model, carrying out correlation analysis on geomagnetic reference point geomagnetic intensity information sequences measured in real time and geomagnetic reference point information sequences in the geomagnetic database, and estimating the position coordinates and action tracks of a user by readjusting the geomagnetic and the laser radar and mutually verifying the geomagnetic and the laser radar when the measured indoor environment is easily interfered by the outside, so that indoor mobile measurement is realized, and the scanning efficiency and accuracy of the indoor space three-dimensional information and the geomagnetic information are improved.
Description
Technical Field
The invention relates to the field of laser radar and geomagnetic measurement, in particular to a method and equipment for cooperatively measuring space three-dimensional information and geomagnetic information based on laser radar and geomagnetic integrated equipment.
Background
With the development of modern society, the demand for personalized location information services has grown greatly. The outdoor multidimensional information can accurately position the precision to the meter through a GPS, beidou and other satellite navigation systems (Global Navigation Satellite System, GNSS); the GNSS signals in the indoor environment cannot be directly acquired, and thus cannot be accurately located. For modern people, more than 80% of the active space is concentrated indoors. Thus, there is an increasing need for accuracy and efficiency based indoor positioning techniques. In a plurality of technologies capable of providing indoor positioning information, the geomagnetic positioning is utilized, so that positioning information sources are not required to be arranged, the cost is low, and the geomagnetic positioning system is widely applicable to large-scale buildings. Geomagnetism may change due to the passage of a vehicle, resulting in inaccurate measurement.
Disclosure of Invention
In view of this, the present application proposes an integrated device and a collaborative measurement method for acquiring indoor multidimensional information, when the measured indoor environment is susceptible to external interference: when adjacent rooms are formed, geomagnetism and laser radar are readjusted to mutually verify, so that the efficiency and accuracy of scanning of indoor space three-dimensional information and geomagnetism information are improved.
The method for cooperatively measuring the space three-dimensional information and the geomagnetic information based on the laser radar and geomagnetic integrated equipment comprises the steps of establishing an indoor three-dimensional model, constructing a geomagnetic database on a user activity path based on the indoor three-dimensional model, carrying out correlation analysis on geomagnetic reference point geomagnetic intensity information sequences measured in real time and geomagnetic reference point information sequences in the geomagnetic database, and estimating position coordinates and action tracks of a user, so that indoor mobile measurement is realized.
Specifically comprising:
step 1, acquiring indoor point cloud data and high-resolution indoor images by using the laser radar and geomagnetic integrated equipment;
step 2, carrying out multi-mode fusion processing on the point cloud data and the high-resolution indoor image to obtain image data with depth value information and high resolution;
step 3, acquiring an image dataset of an indoor common object, and constructing a deep neural network model RCNN; performing target detection on the high-resolution indoor image through an RCNN model, so as to acquire indoor article information;
step 4, preprocessing the point cloud data and reconstructing a real three-dimensional model of the indoor scene;
step 5, marking the object at the corresponding position of the three-dimensional model in the step 4 according to the image data with the depth value information and the high resolution, namely, giving the object information to the indoor three-dimensional model;
step 6, constructing a geomagnetic database on the user activity path;
and 7, carrying out correlation analysis on the geomagnetic reference point geomagnetic intensity information sequence measured in real time and the geomagnetic reference point information sequence in the geomagnetic database in the step 6, and estimating the position coordinates and the action track of the user, thereby realizing indoor mobile measurement.
The application provides an integrated equipment for acquiring indoor multidimensional information, which comprises: the system comprises a space information scanning module, a GNSS (Global Navigation Satellite System ) signal receiving module, a geomagnetic signal detection module and a data storage;
the output ends of the space information scanning module, the GNSS signal receiving module and the geomagnetic signal detecting module are respectively connected with the data memory; the signal processing module is also connected with an input/output interface and a dual-computer cooperative interface module.
The beneficial technical effects are as follows: the invention provides integrated equipment for acquiring multidimensional information and a cooperative measurement method, which solve the problems that geomagnetism in the existing geomagnetic detection technology is inaccurate in measurement and the like caused by external interference, such as passing of vehicles, readjust geomagnetism and laser radar, mutually verify, enable geomagnetism measurement errors to be more convenient and avoided, and improve the scanning efficiency and accuracy of indoor space three-dimensional information and geomagnetism information.
Drawings
FIG. 1 is a flow chart of a method provided by an embodiment of the invention
FIG. 2 is a schematic diagram of a device according to an embodiment of the present invention
Detailed Description
The present invention will be described in further detail with reference to the following examples in order to make the objects, technical solutions and advantages of the present invention more apparent. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the invention.
The invention adopts geomagnetic matching positioning technology, carries out correlation analysis on geomagnetic reference point geomagnetic intensity information sequences measured in real time and geomagnetic reference point information sequences in a geomagnetic database established in advance, and estimates the position coordinates and action tracks of a user, thereby realizing indoor mobile measurement. The indoor mobile measurement method provided by the invention is based on the geomagnetic database on the pre-constructed user activity path and the geomagnetic reference point geomagnetic intensity information sequence measured in real time, so that errors of geomagnetic measurement are avoided, and the efficiency and accuracy of scanning of indoor space three-dimensional information and geomagnetic information are improved.
Fig. 1 is a flowchart of a method for cooperatively measuring spatial three-dimensional information and geomagnetic information based on a laser radar and geomagnetic integrated apparatus according to an embodiment of the present invention, and the flowchart of the present application is further explained by a block diagram of the integrated apparatus for acquiring indoor multidimensional information in fig. 2, where the integrated apparatus for acquiring indoor multidimensional information includes: the system comprises a space information scanning module, a GNSS (Global Navigation Satellite System ) signal receiving module, a geomagnetic signal detection module and a data storage; the output ends of the space information scanning module, the GNSS signal receiving module and the geomagnetic signal detecting module are respectively connected with the data memory; the signal processing module is also connected with an input/output interface and a double-machine cooperative interface module.
The method for cooperatively measuring the space three-dimensional information and the geomagnetic information based on the laser radar and geomagnetic integrated equipment comprises the steps of establishing an indoor three-dimensional model, establishing a geomagnetic database on a user activity path based on the indoor three-dimensional model, carrying out correlation analysis on a geomagnetic reference point geomagnetic intensity information sequence measured in real time and a geomagnetic reference point information sequence in the geomagnetic database, and estimating the position coordinates and the action track of a user, thereby realizing indoor mobile measurement, and specifically comprises the following steps:
step 1, acquiring indoor point cloud data and high-resolution indoor images by using the laser radar and geomagnetic integrated equipment;
wherein step 1 comprises:
step S11, placing the laser radar and geomagnetic integrated equipment at an indoor central position, and adjusting the height of the laser radar and geomagnetic integrated equipment by adjusting a base tripod of the laser radar and geomagnetic integrated equipment;
s22, scanning and imaging a target object by using a spatial information scanning module in the laser radar and geomagnetic integrated equipment to obtain high-density and high-precision laser point cloud data and a target image with high target resolution; and storing the three-dimensional point cloud data and the image data in a data memory.
Step 2, carrying out multi-mode fusion processing on the point cloud data and the high-resolution indoor image to obtain image data with depth value information and high resolution;
with the continuous and intensive research of indoor positioning at present, a single sensor cannot fully cope with a complex real environment, and in order to solve the problems of detection precision and robustness, a multi-mode method is a currently optimal solution. The multi-modal fusion refers to taking various sensor data as the input of the deep learning network, and the information acquired by various sensors is complementary to the benefits of multi-modal fusion. The method carries out multi-mode fusion processing on the point cloud data acquired in the step 1 and the high-resolution target image to acquire image data with depth value information and high resolution. The invention realizes the accurate matching of the corresponding RGB values on the point cloud and the corresponding pixels of the image based on the collineation equation; the collinearity equation is:
wherein (X, Y, Z) is the coordinate of the laser point on the image plane, and (X, Y) to be solved is the corresponding pixel coordinate of the laser point under the image plane coordinate system; xs, ys and Zs are coordinates of a projection center in an image plane coordinate system, and f is a main distance of the camera; a1-a3, b1-b3, c1-c3 are transform coefficients.
Step 3, acquiring an image dataset of an indoor common object, and constructing a deep neural network model RCNN; performing target detection on the high-resolution indoor image through an RCNN model, so as to acquire indoor article information;
wherein step 3 comprises:
step S31, generating candidate areas: dividing an image into small areas, and merging two areas with highest possibility in the small areas according to a merging rule: the merging rule is to preferentially merge areas with similar color textures, and the total area of the merged areas is reduced; and repeatedly executing the merging operation until the whole image is merged into a region position, generating 2000-3000 candidate regions for one image, and outputting the candidate region generation.
Step S32, extracting features of the candidate areas, normalizing the images into 224 x 224 and inputting the images into a neural network model, wherein the extracted features of the neural network are 4096 dimensions;
step S33, the 4096-dimensional characteristics output by the depth network are sent into a full-connection layer for classification; and independently distributing a linear SVM classifier for each class of targets to judge and output classification results.
Step 4, preprocessing the point cloud data and reconstructing a real three-dimensional model of the indoor scene;
the unprocessed point cloud data are arranged in a random and unordered way, modeling cannot be performed, and the data can be imported into 3DMax software for modeling only after the processing such as point cloud splicing, merging, noise reduction and the like. The current ground three-dimensional laser scanner generally has post-processing software of data, the data is imported into HD_3LS_SCENE software by adopting data processing software HD_3LS_SCENE of a Zhonghai scanner, and corresponding point cloud splicing, merging, noise reduction and other processes are performed, so that high-quality point cloud modeling data can be obtained.
In the processing process, all site clouds are spliced, and the splicing precision among the sites is up to 0.05m. And secondly, point cloud merging, namely more than 10% of overlapping areas exist between two adjacent stations, so that the data volume can be effectively reduced under the condition that the structure is unchanged by the point cloud merging. And finally, carrying out point cloud noise reduction, deleting and eliminating the point cloud irrelevant to the object, and improving the quality of the point cloud.
The method comprises the steps of S41 and point cloud splicing, wherein a three-dimensional laser scanner adopts a substation type operation mode, each station of acquired data has an independent coordinate system, and all-dimensional point cloud data must be registered under a unified coordinate system in order to acquire the data. After the software is imported into the point cloud data obtained by field scanning, firstly, data splicing is carried out, namely, the multi-station scanned point cloud data are converted into the same coordinate system, and the point cloud splicing of a common overlapping area is adopted to form a whole. The common splicing methods are as follows: target splicing, homonymous point splicing, view splicing, mixed splicing of targets and homonymous points, and splicing of known control points. Hybrid tiling of targets and homonyms is employed herein, followed by tiling in hd_3ls_scene software. When in splicing, the two measuring stations are adopted for splicing, then the spliced result is spliced with the spliced result of the other two stations,
in step S42, the point clouds are combined, and a certain overlapping area is generated due to the substation type scanning, the distance between the scanner and the target also affects the scanned data volume, the closer the distance is, the larger the point cloud density is, the farther the distance is, the smaller the point cloud density is, the data volume is not reduced in the data splicing process, and the data redundancy is also caused. And the acquired point cloud data volume is too large, so that the requirement on hardware facilities is high, and the data processing efficiency is reduced. Therefore, it is necessary to streamline the point cloud data, i.e., merge overlapping regions, without affecting the accuracy of the data. In the HD_3LS_SCENE software, a merging command is executed, the system automatically generates simplified point cloud data, and after the point cloud merging operation, the original structure is not affected, the data storage space is reduced, and the data processing efficiency is improved.
In step S43, the original point cloud data obtained by the three-dimensional laser scanner is a scattered spatial data set. In scanning, the resulting point cloud data may be affected by many factors of the surrounding environment. After the point cloud merging processing, the point cloud data are greatly deleted, but the texture structure of the original data is not changed by merging, and the original noise point still exists. Noise points not only occupy storage space and increase processing time of a computer, but also have influence on segmentation, identification, modeling and the like of point cloud, and are rejected. In the HD_3LS_SCENE software, a large-range noise point cloud is manually deleted, scattered noise points are deleted by a method of automatically identifying the noise points by the software, and all the noise points are removed by switching different visual angles through the interactive operation of manual work and computer software.
The three-dimensional modeling mainly comprises two parts, namely three-dimensional geometric modeling and texture mapping, wherein the three-dimensional geometric modeling constructs an indoor overall structure, and the texture mapping can construct a realistic three-dimensional model. Unlike traditional measurement size modeling, in the high version of 3DMax 2015 and above, the mosaic modeling can lead the spliced point cloud data into the working surface, and then the mosaic module with reasonable design is designed according to the spliced point cloud structural characteristics to perform block modeling. The block modeling can well construct the details of complex modules without destroying the overall structure. After mosaic block modeling, combining all block models to construct a complete three-dimensional model, and finally hiding point cloud data to obtain a clear white film.
And 5, marking the object at the corresponding position of the three-dimensional model in the step 4 according to the image data with the depth value information and the high resolution, namely, giving the object information to the indoor three-dimensional model.
Wherein step 5 comprises:
s51, correcting the high-resolution indoor image by utilizing Photoshop software, converting the image into a front projection image, and performing cutting, variegation, peace and tone treatment;
and S52, mapping the target detection result to the white film of the corresponding object through 3DMax software, and if the mapping distortion dislocation phenomenon occurs, correspondingly adding UVW mapping coordinates, and adjusting texture coordinates and model coordinates to enable the geometric model to be matched with texture data.
Step 6, constructing a geomagnetic database on the user activity path;
wherein step 6 comprises:
step S61, labeling the activity path of the indoor user based on the position data information of the indoor three-dimensional model;
based on the indoor three-dimensional model position information, counting all possible activity paths of a user, wherein the user activity path data expression is as follows:
where Index is the unique identification number of the user's active path,is a different node on the user active path, and the user active path under Index number has I nodes in total.
The method further comprises determining geomagnetic reference points k of geomagnetic databases between two adjacent nodes according to a preset parameter d, firstly obtaining the total number N of geomagnetic reference points between the two adjacent nodes and the distance L between the two adjacent nodes, and adopting the following calculation formula,
wherein, the method comprises the following steps of) And%) Is the coordinates of two adjacent nodes on two user activity paths;
spatial position coordinates of geomagnetic reference points should be [ ]) The method comprises the following steps:
，k=1,2,…[N]
, k=1,2,…[N]。
step S62, generating a geomagnetic reference point according to the moving path, calculating the position coordinates of the geomagnetic reference point, and obtaining geomagnetic intensity of the geomagnetic reference point;
generating a geomagnetic reference point according to the activity path, calculating the position coordinates of the geomagnetic reference point, and acquiring geomagnetic intensity of the geomagnetic reference point comprises: after the space coordinates of the geomagnetic reference points are acquired, simulating the indoor activities of users, and acquiring geomagnetic intensity signals at the geomagnetic reference points on the user activity path through the carried geomagnetic signal detection module equipment
Step S63, constructing a geomagnetic database on a user activity path based on the position coordinates of the geomagnetic reference point and the geomagnetic intensity;
constructing a geomagnetic database on a user activity path includes: binding the space coordinates of the geomagnetic reference points with geomagnetic intensity, and storing the space coordinates and the geomagnetic intensity into a geomagnetic database of a server by taking a user moving path as a unit.
And 7, carrying out correlation analysis on the geomagnetic reference point geomagnetic intensity information sequence measured in real time and the geomagnetic reference point information sequence in the geomagnetic database in the step 6, and estimating the position coordinates and the action track of the user, thereby realizing indoor mobile measurement.
When a user carries the smart phone to move indoors, a geomagnetic detection module built in the smart phone collects geomagnetic intensity information K (t) in real time. In view of the fact that geomagnetic signals are easily interfered by metal objects, if a car which is driven by a user suddenly appears around, geomagnetic intensity is likely to be affected, and therefore collected geomagnetic intensity K (t) may not be matched with geomagnetic sequences in a geomagnetic database constructed in the step 6 in an agreed manner. The disturbance is then instantaneous, and the disturbance such as a car driving over is not long-lasting, i.e. the collected geomagnetic intensity sequence K (t) should be similar to a certain sequence in the geomagnetic database in rough distribution. Based on the method, the KL divergence is utilized to calculate the similarity between the geomagnetic intensity sequence K (t) detected in real time and the sequence M (t) in the geomagnetic database, and further the space coordinates of the corresponding sequence in the database are obtained, so that indoor mobile measurement is realized.
Wherein step 7 comprises:
step S71, in the indoor moving process of the smart phone carried by the user, geomagnetic intensity information K (t) is collected in real time by utilizing a geomagnetic detection module built in the carried smart phone;
step S72, a geomagnetic intensity construction sequence M (t) of a geomagnetic reference point between every two nodes is established according to a user activity path in a geomagnetic database, the nodes on each path and the geomagnetic reference point between every two nodes, and KL divergence calculation is carried out on the geomagnetic intensity sequences K (t) and M (t) collected in real time:
n is the number of geomagnetic reference points between two nodes, corresponding nodes are obtained according to KL divergence values, the nodes are connected to form a path, a user possible active path where the cumulative KL divergence value is minimum is determined as a current user actual path, and indoor mobile measurement is achieved. When the KL divergence value is smaller than a preset threshold value g, judging that the movement of the current user belongs to two node positions under the corresponding path in the geomagnetic database; otherwise, the device server end continuously searches and selects two node positions with the smallest KL divergence value as corresponding nodes.
In the above embodiments, it may be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When used in whole or in part, is implemented in the form of a computer program product comprising one or more computer instructions. When loaded or executed on a computer, produces a flow or function in accordance with embodiments of the present invention, in whole or in part. The computer may be a general purpose computer, a special purpose computer, a computer network, or other programmable apparatus. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center by a wired (e.g., coaxial cable, fiber optic, digital Subscriber Line (DSL), or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. that contains an integration of one or more available media. The usable medium may be a magnetic medium (e.g., floppy Disk, hard Disk, tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid State Disk (SSD)), etc.
The foregoing description of the preferred embodiments of the invention is not intended to be limiting, but rather is intended to cover all modifications, equivalents, and alternatives falling within the spirit and principles of the invention.
Claims (10)
1. A method for cooperatively measuring space three-dimensional information and geomagnetic information based on laser radar and geomagnetic integrated equipment comprises the steps of establishing an indoor three-dimensional model, constructing a geomagnetic database on a user activity path based on the indoor three-dimensional model, carrying out correlation analysis on geomagnetic reference point geomagnetic intensity information sequences measured in real time and geomagnetic reference point information sequences in the geomagnetic database, and estimating position coordinates and action tracks of a user so as to realize indoor movement measurement;
the building of the indoor three-dimensional model comprises the following steps:
step 1, acquiring indoor point cloud data and high-resolution indoor images by using the laser radar and geomagnetic integrated equipment;
step 2, carrying out multi-mode fusion processing on the point cloud data and the high-resolution indoor image to obtain image data with depth value information and high resolution;
step 3, acquiring an image dataset of an indoor common object, and constructing a deep neural network model RCNN; performing target detection on the high-resolution indoor image through an RCNN model, so as to acquire indoor article information;
step 4, preprocessing the point cloud data and reconstructing a real three-dimensional model of the indoor scene;
step 5, marking the object at the corresponding position of the three-dimensional model in the step 4 according to the image data with the depth value information and the high resolution, namely, giving the object information to the indoor three-dimensional model;
step 6, constructing a geomagnetic database on the user activity path;
step 7, carrying out correlation analysis on a geomagnetic reference point geomagnetic intensity information sequence measured in real time and a geomagnetic reference point information sequence in a geomagnetic database in step 6, when the geomagnetic intensity variation is caused by the passing of a vehicle, establishing a geomagnetic intensity construction sequence M (t) of a geomagnetic reference point between two nodes according to a user activity path in the geomagnetic database, nodes on each path and geomagnetic reference points between every two nodes, and carrying out KL divergence calculation on a geomagnetic intensity sequence K (t) and the geomagnetic intensity sequence M (t) collected in real time:
n is the number of geomagnetic reference points between two nodes, corresponding nodes are obtained according to KL divergence values, the nodes are connected to form a path, a user possible active path where the cumulative KL divergence value is minimum is determined as a current user actual path, and the method for obtaining the nodes comprises the following steps: when the KL divergence value is smaller than a preset threshold value g, judging that the movement of the current user belongs to two node positions under the corresponding path in the geomagnetic database; otherwise, the equipment server end continuously searches and selects two node positions with the smallest KL divergence value as corresponding nodes; finally, the position coordinates and the action track of the user are estimated, so that indoor movement measurement is realized.
2. The method for cooperatively measuring spatial three-dimensional information and geomagnetic information based on the integrated laser radar and geomagnetic equipment as set forth in claim 1, wherein the acquiring indoor point cloud data and high-resolution indoor images by using the integrated laser radar and geomagnetic equipment in step 1 includes:
step S11, placing the laser radar and geomagnetic integrated equipment at an indoor central position, and adjusting the height of the laser radar and geomagnetic integrated equipment by adjusting a base tripod of the laser radar and geomagnetic integrated equipment;
step S12, scanning and imaging a target object by using a spatial information scanning module in the laser radar and geomagnetic integrated equipment, and obtaining high-density and high-precision laser point cloud data and a target image with high target resolution; and storing the three-dimensional point cloud data and the image data in a data memory.
3. The method for cooperatively measuring spatial three-dimensional information and geomagnetic information based on a laser radar and geomagnetic integrated apparatus according to claim 1, wherein in the step 2, the point cloud data and the high-resolution indoor image are subjected to multi-mode fusion processing, and obtaining image data with depth value information and high resolution includes:
based on a collineation equation, realizing accurate matching of the point cloud and corresponding RGB values on corresponding pixels of the image; the collinearity equation is:
wherein (X, Y, Z) is the coordinate of the laser point on the image plane, and (X, Y) to be solved is the corresponding pixel coordinate of the laser point under the image plane coordinate system; xs, ys and Zs are coordinates of a projection center in an image plane coordinate system, and f is a main distance of the camera;
a1, a2, a3, b1, b2, b3, c1, c2, c3 are transform coefficients.
4. The method for cooperatively measuring spatial three-dimensional information and geomagnetic information based on integrated laser radar and geomagnetic equipment as set forth in claim 1, wherein the acquiring indoor article information in step 3 includes:
step S31, generating candidate areas: dividing an image into small areas, and merging two areas with highest possibility in the small areas according to a merging rule: the merging rule is to preferentially merge areas with similar color textures, and the total area of the merged areas is reduced; repeatedly executing merging operation until the whole image is merged into a region position, generating 2000-3000 candidate regions by one image, and outputting the candidate region generation;
step S32, extracting features of the candidate areas, normalizing the images into 224 x 224 and inputting the images into a neural network model, wherein the extracted features of the neural network are 4096 dimensions;
step S33, the 4096-dimensional characteristics output by the depth network are sent into a full-connection layer for classification; and independently distributing a linear SVM classifier for each class of targets to judge and output classification results.
5. The method for cooperatively measuring spatial three-dimensional information and geomagnetic information based on the laser radar and geomagnetic integrated equipment according to claim 1, wherein the preprocessing of the point cloud data and the reconstruction of the real three-dimensional model of the indoor scene in the step 4 comprise the following steps:
s41, point cloud splicing;
step S42, merging point clouds;
and S43, reducing noise of the point cloud.
6. The method for cooperatively measuring spatial three-dimensional information and geomagnetic information based on a laser radar and geomagnetic integrated apparatus as set forth in claim 1, wherein the assigning of the article information to the three-dimensional model in the step 5 includes:
s51, correcting the high-resolution indoor image by utilizing Photoshop software, converting the image into a front projection image, and performing clipping, variegation removal and peace hue treatment;
and S52, mapping the target detection result to the white film of the corresponding object through 3DMax software, and if the mapping distortion dislocation phenomenon occurs, correspondingly adding UVW mapping coordinates, and adjusting texture coordinates and model coordinates to enable the geometric model to be matched with texture data.
7. The method for cooperatively measuring spatial three-dimensional information and geomagnetic information based on the integrated laser radar and geomagnetic equipment of claim 1, wherein the step 6 includes:
step S61, labeling the activity path of the indoor user based on the position data information of the indoor three-dimensional model;
step S62, generating a geomagnetic reference point according to the moving path, calculating the position coordinates of the geomagnetic reference point, and obtaining geomagnetic intensity of the geomagnetic reference point;
step S63, constructing a geomagnetic database on a user activity path based on the position coordinates of the geomagnetic reference point and the geomagnetic intensity;
the labeling of the activity path of the indoor user in step S61 includes:
based on the indoor three-dimensional model position information, counting all possible activity paths of a user, wherein the user activity path data expression is as follows:
{Index,{(x i ,y i ) node ，i＝1,2...I}}
where Index is the unique identification number of the user's active path, (x) i ，y i ) node Is a different node on the user active path, and the user active path under Index number has I nodes in total.
8. The method for cooperatively measuring spatial three-dimensional information and geomagnetic information based on a laser radar and geomagnetic integrated apparatus as set forth in claim 7, further comprising:
the labeling the activity path of the indoor user in step S61 includes obtaining geomagnetic reference points:
determining geomagnetic reference points k of a geomagnetic database between two adjacent nodes according to a preset parameter d, firstly obtaining the total number N of geomagnetic reference points between the two adjacent nodes and the distance L of the two adjacent nodes, and adopting the following calculation formula,
wherein, (x) node1 ,y node1 ) And (x) node2 ,y node2 ) Is the coordinates of two adjacent nodes on two user activity paths;
the spatial position coordinates of the geomagnetic reference point should be (x k ,y k ) The method comprises the following steps:
in the step S62, generating a geomagnetic reference point according to the movement path, calculating a position coordinate of the geomagnetic reference point, and obtaining geomagnetic intensity of the geomagnetic reference point includes: after the space coordinates of the geomagnetic reference points are obtained, simulating the indoor activities of users, and collecting geomagnetic intensity signals at the geomagnetic reference points on the user activity path through the carried geomagnetic signal detection module;
in the step S63, constructing a geomagnetic database on the user activity path based on the position coordinates of the geomagnetic reference point and the geomagnetic intensity includes: binding the space coordinates of the geomagnetic reference points with geomagnetic intensity, and storing the space coordinates and the geomagnetic intensity into a geomagnetic database of a server by taking a user moving path as a unit.
9. The method for cooperatively measuring spatial three-dimensional information and geomagnetic information based on a laser radar and geomagnetic integrated device according to claim 1, wherein in the step 7, correlation analysis is performed on a geomagnetic reference point geomagnetic intensity information sequence measured in real time and a geomagnetic reference point information sequence in the geomagnetic database, and position coordinates and action tracks of a user are estimated, so that indoor movement measurement is realized, and the method comprises the following steps:
in step S71, in the indoor moving process of the smart phone carried by the user, geomagnetic intensity information K (t) is collected in real time by using a geomagnetic detection module built in the carried smart phone.
10. An integrated apparatus for acquiring multi-dimensional information and a cooperative measurement apparatus, adopting the method as claimed in any one of claims 1 to 9, said integrated apparatus for acquiring indoor multi-dimensional information comprising: the system comprises a space information scanning module, a GNSS (GlobalNavigation Satellite System ) signal receiving module, a geomagnetic signal detection module, a data storage and a signal processing module;
the output ends of the space information scanning module, the GNSS signal receiving module and the geomagnetic signal detecting module are respectively connected with the data memory; the signal processing module is also connected with an input/output interface and a dual-computer cooperative interface module.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110911933.6A CN113532424B (en) | 2021-08-10 | 2021-08-10 | Integrated equipment for acquiring multidimensional information and cooperative measurement method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110911933.6A CN113532424B (en) | 2021-08-10 | 2021-08-10 | Integrated equipment for acquiring multidimensional information and cooperative measurement method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113532424A CN113532424A (en) | 2021-10-22 |
| CN113532424B true CN113532424B (en) | 2024-02-20 |
Family
ID=78091381
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110911933.6A Active CN113532424B (en) | 2021-08-10 | 2021-08-10 | Integrated equipment for acquiring multidimensional information and cooperative measurement method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113532424B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115291767A (en) * | 2022-08-01 | 2022-11-04 | 北京奇岱松科技有限公司 | Control method and device of Internet of things equipment, electronic equipment and storage medium |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10024664B1 (en) * | 2014-09-30 | 2018-07-17 | The United States Of America As Represented By The Administrator Of The National Aeronautics And Space Administration | Range and intensity image-based terrain and vehicle relative pose estimation system |
| CN108801265A (en) * | 2018-06-08 | 2018-11-13 | 武汉大学 | Multidimensional information synchronous acquisition, positioning and position service apparatus and system and method |
| KR101971734B1 (en) * | 2018-07-26 | 2019-04-23 | 영남대학교 산학협력단 | Apparatus and method for indoor positioning |
| CN109737968A (en) * | 2019-03-07 | 2019-05-10 | 中山大学 | Indoor fusion and positioning method based on two-dimentional LiDAR and smart phone |
| CN111126304A (en) * | 2019-12-25 | 2020-05-08 | 鲁东大学 | Augmented reality navigation method based on indoor natural scene image deep learning |
| CN111156983A (en) * | 2019-11-19 | 2020-05-15 | 石化盈科信息技术有限责任公司 | Target equipment positioning method and device, storage medium and computer equipment |
| WO2020237693A1 (en) * | 2019-05-31 | 2020-12-03 | 华南理工大学 | Multi-source sensing method and system for water surface unmanned equipment |
| KR20210003065A (en) * | 2020-04-14 | 2021-01-11 | 네이버랩스 주식회사 | Method and system for collecting data |
| CN113066162A (en) * | 2021-03-12 | 2021-07-02 | 武汉大学 | Urban environment rapid modeling method for electromagnetic calculation |
-
2021
- 2021-08-10 CN CN202110911933.6A patent/CN113532424B/en active Active
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10024664B1 (en) * | 2014-09-30 | 2018-07-17 | The United States Of America As Represented By The Administrator Of The National Aeronautics And Space Administration | Range and intensity image-based terrain and vehicle relative pose estimation system |
| CN108801265A (en) * | 2018-06-08 | 2018-11-13 | 武汉大学 | Multidimensional information synchronous acquisition, positioning and position service apparatus and system and method |
| KR101971734B1 (en) * | 2018-07-26 | 2019-04-23 | 영남대학교 산학협력단 | Apparatus and method for indoor positioning |
| CN109737968A (en) * | 2019-03-07 | 2019-05-10 | 中山大学 | Indoor fusion and positioning method based on two-dimentional LiDAR and smart phone |
| WO2020237693A1 (en) * | 2019-05-31 | 2020-12-03 | 华南理工大学 | Multi-source sensing method and system for water surface unmanned equipment |
| CN111156983A (en) * | 2019-11-19 | 2020-05-15 | 石化盈科信息技术有限责任公司 | Target equipment positioning method and device, storage medium and computer equipment |
| CN111126304A (en) * | 2019-12-25 | 2020-05-08 | 鲁东大学 | Augmented reality navigation method based on indoor natural scene image deep learning |
| KR20210003065A (en) * | 2020-04-14 | 2021-01-11 | 네이버랩스 주식회사 | Method and system for collecting data |
| CN113066162A (en) * | 2021-03-12 | 2021-07-02 | 武汉大学 | Urban environment rapid modeling method for electromagnetic calculation |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113532424A (en) | 2021-10-22 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Chen et al. | Applicability of personal laser scanning in forestry inventory | |
| CN111563442B (en) | Slam method and system for fusing point cloud and camera image data based on laser radar | |
| US10043097B2 (en) | Image abstraction system | |
| US20190303725A1 (en) | Neural network training system | |
| CN109285220A (en) | A kind of generation method, device, equipment and the storage medium of three-dimensional scenic map | |
| CN111709981A (en) | Registration method of laser point cloud and analog image with characteristic line fusion | |
| US20190026400A1 (en) | Three-dimensional modeling from point cloud data migration | |
| US20190026938A1 (en) | Three-dimensional modeling from optical capture | |
| CN111028358B (en) | Indoor environment augmented reality display method and device and terminal equipment | |
| Chen et al. | Transforming a 3-d lidar point cloud into a 2-d dense depth map through a parameter self-adaptive framework | |
| CN112818925B (en) | Urban building and crown identification method | |
| Lin et al. | Measurement and calculation of crown projection area and crown volume of individual trees based on 3D laser-scanned point-cloud data | |
| CN112489099A (en) | Point cloud registration method and device, storage medium and electronic equipment | |
| CN113340312A (en) | AR indoor live-action navigation method and system | |
| CN113532424B (en) | Integrated equipment for acquiring multidimensional information and cooperative measurement method | |
| CN106709432B (en) | Human head detection counting method based on binocular stereo vision | |
| Kaiser et al. | Automatic co-registration of photogrammetric point clouds with digital building models | |
| CN113160292B (en) | Laser radar point cloud data three-dimensional modeling device and method based on intelligent mobile terminal | |
| Jiang et al. | Determination of construction site elevations using drone technology | |
| Fasiolo et al. | Comparing LiDAR and IMU-based SLAM approaches for 3D robotic mapping | |
| CN112446844B (en) | Point cloud feature extraction and registration fusion method | |
| CN110580468B (en) | Single wood structure parameter extraction method based on image matching point cloud | |
| CN116863083A (en) | Method and device for processing three-dimensional point cloud data of transformer substation | |
| CN113724381B (en) | Dynamic three-dimensional scene rapid reconstruction method based on high-resolution remote sensing image | |
| CN109118576A (en) | Large scene three-dimensional reconstruction system and method for reconstructing based on BDS location-based service |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |