CN107194991A - A kind of three-dimensional global visualization monitoring system construction method updated based on skeletal point local dynamic - Google Patents
A kind of three-dimensional global visualization monitoring system construction method updated based on skeletal point local dynamic Download PDFInfo
- Publication number
- CN107194991A CN107194991A CN201710347732.1A CN201710347732A CN107194991A CN 107194991 A CN107194991 A CN 107194991A CN 201710347732 A CN201710347732 A CN 201710347732A CN 107194991 A CN107194991 A CN 107194991A
- Authority
- CN
- China
- Prior art keywords
- dimensional
- scene
- camera
- global
- point
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
- G06T17/05—Geographic models
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/22—Matching criteria, e.g. proximity measures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/13—Edge detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/50—Depth or shape recovery
- G06T7/55—Depth or shape recovery from multiple images
- G06T7/593—Depth or shape recovery from multiple images from stereo images
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
- H04N7/181—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast for receiving images from a plurality of remote sources
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2200/00—Indexing scheme for image data processing or generation, in general
- G06T2200/08—Indexing scheme for image data processing or generation, in general involving all processing steps from image acquisition to 3D model generation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10028—Range image; Depth image; 3D point clouds
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20212—Image combination
- G06T2207/20221—Image fusion; Image merging
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30232—Surveillance
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Data Mining & Analysis (AREA)
- Software Systems (AREA)
- Geometry (AREA)
- Bioinformatics & Computational Biology (AREA)
- Evolutionary Computation (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Artificial Intelligence (AREA)
- Remote Sensing (AREA)
- Life Sciences & Earth Sciences (AREA)
- Computer Graphics (AREA)
- Image Processing (AREA)
Abstract
A kind of three-dimensional global visualization monitoring system construction method updated based on skeletal point local dynamic, this method mainly includes mobile robot visual angle scene three-dimensional reconstruction system, a wide range of view scene three-dimensional reconstruction system and skeletal point three-dimensional localization and matching system, and implementation step mainly includes the real-time update of mobile robot visual angle scene three-dimensional reconstruction, a wide range of view scene three-dimensional reconstruction, global coordinate system lower skeleton point three-dimensional localization, the control of accumulated error and a wide range of three-dimensional scenic.The present invention combines the multiple view angle video camera of fixed placement（Camera）And the array camera of mobile robot carrying（Camera）Pass through skeletal point three-dimensional localization and matching process, to complete lower real-time three-dimensional scene rebuilding on a large scale, and combine the method that local differentiation updates, only changing unit in three-dimensional scenic is updated, renewal efficiency is greatly improved, the global three-dimensional map of good a wide range of scene has both been obtained, and real-time update can be carried out to it.
Description
Technical field
The present invention relates to a kind of three-dimensional global visualization monitoring system construction method updated based on skeletal point local dynamic,
The three-dimensional global visualization monitoring realized to a wide range of scene can be updated by the local dynamic of skeletal point, belongs to computer vision
Field.
Background technology
The three-dimensional global visualization monitoring of a wide range of three-dimensional scenic is a popular research direction of computer vision field,
It suffers from being widely applied prospect in many fields such as industrial monitoring, traces archaeology, virtual realities.Based on being arranged
Apply, current major factories, transformer station and some communal facilitys, deploy video camera, quantity from several to up to a hundred,
Whether run well to monitoring site each several part, but current monitoring image is that manpower is completed, and two that video camera is obtained
Tie up image and lose larger and relatively low with scene relating degree for scene information, for duty personnel and manager, ceaselessly
The video information for watching such magnanimity is one very big burden, is also a unsustainable job.But if being regarded by machine
The method of feel, the image real-time reconstructing that these video cameras are shot is a three dimensions similar to real scene, is passed through
Threedimensional model after monitoring reconstruction, understands the dynamic in scene, so just can greatly reduce the workload of staff in real time,
Field condition can be more truly grasped again, and the problem of scene there may be can also be found in time, is responded in time.
For these problems, in recent years, with the development of computer vision, based on mobile platform（Vehicular）Take the photograph more
The technology and software systems that three potential field scapes of structure are maked an inspection tour in camera region gradually grow up.But, because its foundation is whole three-dimensional
Scene needs to make an inspection tour whole scene areas, could gather the image data acquiring of whole, for a wide range of three-dimensional scenic, takes
It is longer, it is impossible to realize the reconstruction of whole three-dimensional scenic in the short period of time, wanting for three-dimensional scenic real-time update can not be met
Ask, and do not make full use of the existing multiple camera video information in scene.But regarded using the existing multiple video cameras in scene
Frequency information carries out three-dimensional reconstruction, can only realize that the three-dimensional scenic of live part is quickly rebuild again, and can not obtain the overall situation dimensionally
Figure, if three-dimensional point cloud splicing is carried out to local message obtains global dynamic 3 D map, can not meet the need of real-time update again
Ask.For problem above, the scene reconstruction method of this programme combination mobile robot, and propose a kind of based on skeleton Point matching
The method for reconstructing three-dimensional scene of realization.
The content of the invention
The present invention provides a kind of three-dimensional global visualization monitoring system construction method updated based on skeletal point local dynamic,
This method is by the three-dimensional reconstruction of stereoscopic vision, using the perspective view based on multiple-camera, with reference to mobile robot
Regional three-dimensional reconstruction, and using the method for local difference update, realize that real-time three-dimensional is global visual under a wide range of scene
Change monitoring.The region of each camera acquired image of transformer station is identified and to the weight of local scene for convenience
Build, therefore, the coding circle of handmarking is added in the field range region of each camera of transformer station as skeletal point, to cog region
Domain position.Technical scheme is as follows：It is a kind of that system is monitored based on the three-dimensional global visualization that skeletal point local dynamic updates
System construction method, this method mainly includes：
（1）Vehicle-carried mobile formula video camera 3 D scene rebuilding system：The main battle array for including mobile robot platform and its being carried
Row video camera（Camera）, it is main to complete the regional area reconstruction that fixed monitoring site video camera be rebuild；
（2）A wide range of view scene three-dimensional reconstruction system：Mainly the multiple fixed placements containing the public visual field are taken the photograph in scene
Camera（Camera）Constitute, the main scene three-dimensional reconstruction for completing fixed monitor area；
（3）Skeletal point three-dimensional localization and matching system：Fixation is put in the main coded markings artificially set in scene by having, scene
The array camera for video camera and the mobile robot carrying put is collectively formed, the main three-dimensional localization for completing scene middle skeleton point,
And as above（1）And（2）Three-dimensional scenic fusion after reconstruction.
Technical scheme is as follows：It is a kind of that system is monitored based on the three-dimensional global visualization that skeletal point local dynamic updates
System construction method, this programme includes offline global three-dimensional scenic map reconstruction and online three-dimensional scene area quasi real time updates two
Point.Offline 3 D scene rebuilding part is realized jointly based on vehicle-carried mobile formula video camera and monitoring site video camera, realization pair
Fixed monitor area is rebuild, and vehicle-carried mobile formula video camera realizes other regional reconstructions, and the two gathers the coding bone in image simultaneously
Frame point is extracted and code identification, and carries out three-dimensional reconstruction to skeletal point, and the coordinate value of local skeletal point is transformed into the overall situation
Under coordinate system.Finally, it is with reference to the position under global coordinate system of result and skeletal point of local three-dimensional reconstruction, each local is three-dimensional
The map of reconstruction is spliced and merged, so as to realize the reconstruction of static panorama three-dimensional map.Put simultaneously by fixation in scene
The picture frame for the camera acquisition put carries out locality Three-Dimensional Dynamic renewal.Main working process comprises the following steps：
（1）A wide range of view scene three-dimensional reconstruction：This step is based primarily upon the video camera of fixed placement in scene（Camera）, it is complete
Extracted into IMAQ, general areas three-dimensional reconstruction, skeletal point；
（2）Mobile robot visual angle scene three-dimensional reconstruction：This step mainly completes the array camera mark of mobile robot carrying
Fixed, IMAQ, local three-dimensional reconstruction, mobile robot visual angle skeletal point are extracted；
（3）Global coordinate system lower skeleton point three-dimensional localization：According to such as step（1）And step（2）The skeletal point extracted, by it
Align under normalizing to global coordinate system, in order to which the three-dimensional scene information rebuild according to this skeletal point is merged；
（4）The real-time update of a wide range of three-dimensional scenic：Three-dimensional scenic after for having rebuild, the shooting to fixed placement in scene
The picture frame of machine collection carries out picture frame sequence analysis and local updating mode, on the three-dimensional scenic basis set up, enters
Row locality three-dimensional updates, and in the case of ensureing that three-dimensional scenic is effective in real time, renewal efficiency is substantially improved；
（5）The control of accumulated error：The main Reconstruction Strategy using common view carries out cumulative errors with adjustment optimization method
Control, to obtain the three-dimensional reconstruction effect that robustness is stronger；
Beneficial effects of the present invention, the present invention combines the multiple view angle video camera of fixed placement（Camera）And mobile robot carrying
Array camera（Camera）, to complete to descend real-time three-dimensional scene rebuilding on a large scale, and the method that local differentiation updates is combined, both obtained
The global three-dimensional map of scene was obtained, and real-time update can be carried out to it.
Brief description of the drawings
Fig. 1 is a kind of three-dimensional global visualization monitoring system construction method updated based on skeletal point local dynamic of the present invention
Handmarking's point design diagram；
Fig. 2 is a kind of mark of the three-dimensional global visualization monitoring system construction method updated based on skeletal point local dynamic of the present invention
Note point overhaul flow chart；
Fig. 3 is a kind of base of the three-dimensional global visualization monitoring system construction method updated based on skeletal point local dynamic of the present invention
In the style of shooting schematic diagram of common view；
Fig. 4 is a kind of base of the three-dimensional global visualization monitoring system construction method updated based on skeletal point local dynamic of the present invention
In the reconstruction mode schematic diagram of common view；
Fig. 5 is the three of a kind of three-dimensional global visualization monitoring system construction method updated based on skeletal point local dynamic of the present invention
Tie up scene fusion schematic diagram；
Fig. 6 is a kind of light of the three-dimensional global visualization monitoring system construction method updated based on skeletal point local dynamic of the present invention
Beam method adjustment effects schematic diagram；
Fig. 7 is a kind of difference of the three-dimensional global visualization monitoring system construction method updated based on skeletal point local dynamic of the present invention
Different region minimum enclosed rectangle extends schematic diagram.
Embodiment
Technical scheme is described in more detail with reference to the accompanying drawings and detailed description.
Shown in reference picture 1, Fig. 2, Fig. 3, Fig. 4, Fig. 5, Fig. 6 and Fig. 7, a kind of three-dimensional updated based on skeletal point local dynamic
Global visualization monitoring system construction method, mainly includes mobile robot visual angle scene three-dimensional reconstruction system, regards on a large scale more
Angle scene three-dimensional reconstruction system, skeletal point three-dimensional localization and matching system.
Real-time three-dimensional global visualization monitors a wide range of three-dimensional reconstruction firstly the need of completion scene under a wide range of scene, greatly
Scope three-dimensional reconstruction is a kind of based on image digitization up short reconstruction technique, the photo shot from different perspectives using multiple,
By handling the image of characteristic target edge, artificial target in testee etc., then characteristic target image is positioned
To calculate the accurate three-dimensional position of object space point.Digitized video will recognize these targets, it is necessary to constitute this by means of extracting
The feature of the so-called image of a little targets.Feature extraction is the basis of image analysing computer and Image Matching.These features can be using individual
The coded markings of engineer distinguish different scenes, and coded markings and scene characteristic point collectively form scene skeletal point.
Coded markings give a unique encoded radio to each point in image, and the coded markings with obvious characteristic are as a wide range of
The skeletal point of scene.
The encoded point of handmarking according to being designed as shown in Figure 1.The equal extracting waste of artificial target is foreground, and black is
Background colour.There is a circle inside of mark, and circular center is that the center of circle is the target point detected.The mesh of coding and non-coding point
Punctuate correspondence diameter of a circle is typically consistent, is easy to disposably extract encoding target point.It is segmentation around the target point of coded markings
Annular section, for being encoded to encoded point, it is only determine some encoded point characteristic information, be referred to as " coding-belt ".
" coding-belt " of coded markings angularly every 24 degree it is a, be divided into 15 parts, every part can all be equal to bit.It is selected
Counterclockwise or clockwise direction is as the order of coding, and coding records with binary digit, and each can take prospect respectively
Or background colour, represented with binary code " 1 " or " 0 ".
According to image characteristics extraction algorithm, the image coordinate in the center of circle that mark point feature is met after camera imaging is extracted.This
Invention first carries out image segmentation with Canny operators to shooting picture, extracts the profile information of different zones, because the circle shot exists
Ellipse is rendered as in image, searches for possible then in conjunction with the feature such as characteristic information such as size, shape and position distribution of mark point
Oval border, for possible oval border picture point using gradient mean value method carry out sub-pix amendment, use least square
Method fitting algorithm obtains the information such as elliptical center and major and minor axis.The coding collar out-of-bounds enclosed to its elliptical side carries out gray scale sampling, obtains
To the coding of coded markings point, if there is no encoded radio, then it is assumed that be non-coded target.The testing process of mark point is as schemed
Shown in 2.
The matching process of mark point is using two kinds of solutions, if for coding maker, can respectively be visited on each image
Recognized that it is encoded when surveying coded target, and an ID is identified to each coding maker, had on different pictures
Identical ID point is exactly same place.If for other marks, one is excluded using the consistent geometrical constraint of space point topological structure
The match point of a little mistakes.
The inner parameter of camera（Intrinsic parameter）It can be obtained by camera calibration, but in three-dimensional reconstruction system, except
The inner parameter of each camera is needed outside demarcating, it is necessary to while knowing each camera in the overall situation（The world）Outside under coordinate
Parameter.Here by global coordinate system set up wherein a camera coordinates fasten, then the external parameter of other cameras namely this
Posture and relative position between two video cameras.The external parameter of camera is mainly with camera translation matrix and spin matrix come table
Show.
Camera translation matrix amount refers to the relative translation amount between two camera coordinates systems in binocular vision system.According to camera
The coordinate that pin-hole model and every camera are set up, the vector that three translational movements are constituted is actually second camera coordinates
Coordinate value of the origin of system under first camera coordinates system.Global coordinate system is set up on first camera coordinate system.It is determined that
Camera external parameter contains two processes, i.e. relative orientation and absolute orientation.Relative orientation refers to relative between two cameras
Position and azimuth information.Absolute orientation refers to that all camera coordinates systems are believed relative to the position under global coordinate system and orientation
Breath.A wide range of multiview three-dimensional rebuilds the relativeness that camera two-by-two is determined often through the relative orientation between camera, finally
By coordinate transform gradually by under all camera coordinates systems unification to global coordinate system, absolute orientation is completed.
The present invention is obtained the projection matrix of left and right cameras using encoded point and sets up the three-dimensional coordinate of skeletal point, except this
Outside, often there is many features point in testee three-D space structure, it is determined that the position of the three dimensions of these characteristic points,
It just can substantially determine the three-dimensional structure of this object.Three-dimensional reconstruction Main Basiss are principle of triangulation, and triangulation is needed
Stereo matching is carried out to the same place in two field pictures.Stereo matching is mainly the characteristic point for extracting multiple image, seeks same
Matching corresponding relation of one testee between the image pixel under different visual angles, sets up more accurate matching relationship.This
Invention uses feature matching method, in the matching process, using epipolar line restriction relation, by the hole heart image of least square fitting
Coordinate, is matched with reference to nominal data and epipolar geom etry principle.If a mark point is imaged on more than two camera, profit
With redundancy more Exact Reconstruction spatial point.
Rebuild at one in scene, multiple common point shoot multi-frame images in the region are surrounded centered on a region, such as
Shown in Fig. 3.In shooting so that the view that camera 1 is shotEnergy and cameraThe view of shootingDual-view can be constituted, i.e.,Exist with other multiple views more than 5 non-co-planar common points
Image, nowReferred to as common view.Now directly with viewCentered on each viewAll divide
Dual-view is not constituted,Camera coordinates where view, which are fastened, sets up global coordinate system, other views by relative orientation all
It is directed toCamera coordinates system where view, willCamera coordinates system where view this as global coordinate system, to the shooting
The relative orientation of the reconstruction of area three-dimensional point is exactly the process of absolute orientation, as shown in Figure 4.
Its process of reconstruction is as follows：According to the information of each image common point, image network is set up, including multiple public
Sets of views and a reference view group；Directional relation of each view with respect to common view is determined by public encoded point；Try to achieve each
Three-dimensional point coordinate under individual sets of views coordinate system；Find and do not orient successful camera around each sets of views, if unoriented figure
As there is common point with directional images（>5）, then carry out relative orientations by 5 algorithms and calculate public between them
3-dimensional encoding point, while realizing the pre-matching of non-coding point with epipolar geom etry etc.；New camera and new 3-dimensional encoding point are directed to
In common set；By the common point of each common view group and benchmark group, each common view group is subjected to absolute orientation to base
In the global coordinate system of quasi- sets of views.This method is much larger than single group dual-view, and common view due to the region of common view group
Group interior orientation be not present coordinate transform, institute can greatly reduce in this approach large-sized object measure coordinate transform number of times so that
Reduce the accumulation of error.
The present invention uses the Reconstruction Strategy based on common view, can effectively reduce Coordinate Conversion number of times, so that by mistake
Difference is accumulative to diminish, and is conducive to the reconstruction precision of raising system.Simultaneously as the visual field presence of the video camera of fixed placement can not be complete
The whole region to be reconstructed of all standing, or because the public visual field between camera is not enough and reconstruction effect that cause is bad asks
Topic, therefore completed in the camera using fixed placement in scene on general areas three-dimensional reconstruction basis, it is flat with reference to mobile robot
Platform carries out locality three-dimensional reconstruction, perfect to be carried out to whole three-dimensional scenic on a large scale, to reach the mesh of global three-dimensional reconstruction
's.
The video camera array of mobile robot carrying includes common-depth camera and common CCD array, passes through depth camera
The depth map of low resolution is got, the scene information that fusion common CCD array is got obtains high-resolution depth map,
That is depth information of scene.Utilize obtained depth information of scene；Combining camera calibrating parameters and principle of triangulation, by depth map
It is mapped as scene surface point cloud and gauging surface normal map；An overall situation is spliced into continuous surface point cloud chart and normal map
Scene surface model；For the problem of ICP algorithm iteration efficiency is low during surface merging, using one kind in multiple yardstick tables
The method of Optimized Iterative splicing from coarse to fine under face；In order to solve ICP nonlinear optimizations, to calculate time efficiency low
Problem, is proposed using the less characteristic of relative motion between two continuous surfaces, by the nonlinear optimal problem in splicing
Approximately it is transformed into linear optimization problem, so that improving the optimizing phase calculates time efficiency；Finally, with reference to GPU computation capabilities,
Above step is further speeded up, realizes and is based on mobile robot platform quick-speed large-scale 3 D scene rebuilding in real time.
The three-dimensional scenic that the three-dimensional scenic and robot platform rebuild for the camera using fixed placement are rebuild, passes through
Skeletal point three-dimensional localization is fused into global three-dimensional scenic, and specific schematic diagram is as shown in Figure 5.Wherein, Py is by robot
Key point in the scene that platform bearer camera is obtained, remaining P1, P2, Pi and Px, Pz etc. are that fixed placement camera is got
Scene key point（Skeletal point）, in the vision area of robot, by three-dimensional reconstruction, the three-dimensional scene information of the vision area can be obtained
And the position orientation relation of the key point such as Px, Py, Pz, and simultaneously for the general areas of such as polygonal internal reconstructed completion
Three-dimensional scenic, Px, Pz world coordinates are, it is known that therefore by key points such as Px, Py, Pz（Skeletal point）Relative pose close
System, can obtain the corresponding relation of the three-dimensional scenic and the three-dimensional scenic of fixed placement camera reconstruction of robot platform reconstruction, and
The fusion of scene information is completed using it.
For the accumulated error inevitably occurred during three-dimensional reconstruction, the present invention is using the side for first rebuilding re-optimization
Method, is adjusted to reconstruction precision.The main method for taking bundle adjustment, video camera is repeatedly shot in diverse location, passes through close shot
Bundle adjustment in videographic measurment, can accurately obtain the coordinate of these index points under global coordinate system.Error compensation method is
Nonlinear equation is linearized by Taylor expansion, the nonlinear problem using amount to be solved as unknown number is converted into given first
Using the correction of amount to be solved as the linear problem of unknown number in the case of value, so as to use various linear problems
Ripe algorithm, such as least square scheduling algorithm solve problem.In imaging measurement task, bundle adjustment method is with dimensional target point
Imaging collinearity equation be adjustment constraints optimized algorithm, first try to achieve the initial value of parameter to be measured, substitute into object function, no
Disconnected adjustment parameter successive ignition to be measured, makes difference constantly reduction more than object function, so as to try to achieve the optimal value of undetermined parameter, it is illustrated
Figure is as shown in Figure 6.
The self-calibration bundle adjustment that the present invention is used.Difference with general bundle adjustment is：General flux of light method
Adjustment is to be based on collinearity condition equation, the whole of picpointed coordinate, tested point picpointed coordinate and other data of external pelivimetry in the industry
Or part substitutes into the calculation method integrally solved in error equation as observation simultaneously.And self-calibration bundle adjustment
Also by camera intrinsic parameter, and distortion parameter also serves as unknown number and substitutes into optimization simultaneously in error equation.Added without extra
Observation achieves that the automatic compensation and accurate solution of systematic error with self calibration light-stream adjustment.
Because static global three-dimensional scenic is in fixed video camera three-dimensional reconstruction combination mobile robot platform three-dimensional reconstruction
On the basis of obtain, according to monitoring requirements, the region monitored in real time the video camera that has been certainly fixed in position, so scene
Update also primarily directed to its main monitored area（The vision area of fixed video camera）, the region updated is needed using shifting for part
The mode that mobile robot platform three-dimensional scenic local updates, its main method is consistent with the three-dimensional renewal of fixed video camera.
For a wide range of three-dimensional reconstruction scene renewal be on global static map build local dynamic 3 D map.
The image information of different zones is got by live multiple cameras, for the image information of every camera collection, will be gathered
Image sequence in latter picture compared with preceding piece image, if, latter as in in preceding piece image
Scene things is had differences, then it represents that the things in scene is changed, and difference section is updated, without weighing again
Build whole three-dimensional scenic.Concrete methods of realizing is as follows：For the region having differences, difference section is extracted, and ask for difference
Partial minimum enclosed rectangle.For the difference of diverse location in image, the asking for of boundary rectangle can be roughly divided into two kinds of feelings
Condition：Information is had differences with having differences information at collection image boundary in the non-boundary of image of collection.For above-mentioned two
The situation of kind, does different subsequent treatments respectively.As shown in fig. 7, black shaded area represents difference section, blue rectangle represents poor
The minimum enclosed rectangle of different part, red rectangle represents the rectangular area after expansion.For diff area minimum enclosed rectangle
The four edges of minimum enclosed rectangle are expanded outwardly one fixed width by the situation of the non-boundary in image respectively, and for difference
Different region minimum enclosed rectangle is in the situation at image boundary, and only the non-boundary edge to minimum enclosed rectangle expands a fixed width
Degree, the side of boundary is in for it, then without expansion, the width of expansion is typically determined by empirical value.After expansion
Include the relevant position region of difference section and non-difference section, i.e. this rectangle and upper width image in rectangle, its rectangular extent
There is the public visual field, image mosaic can be carried out, so as to realize the scene information renewal to the area differentiation part.This programme is contrasted
The current state of each region scene and state before, for the region having differences, are extracted to difference section, and right
The scene information of this difference section is updated, and for the region in the absence of difference, then the state before keeping, without field
Scape updates.Operation time and the operand of system are greatly reduced, the operating rate of system is largely improved, makes system
More rapidly and efficiently.
The foregoing is only a preferred embodiment of the present invention, the application scope of application not limited to this of the present invention, appoints
What those familiar with the art is in the technical scope of present disclosure, the technical scheme that can be become apparent to
Simple change or equivalence replacement each fall within the present invention the application scope of application in.
Claims (4)
1. a kind of three-dimensional global visualization monitoring system construction method updated based on skeletal point local dynamic, this method is mainly wrapped
Include mobile robot visual angle scene three-dimensional reconstruction system, a wide range of view scene three-dimensional reconstruction system, skeletal point three-dimensional localization
And matching system, it is three-dimensional that implementation step mainly includes mobile robot visual angle scene three-dimensional reconstruction, a wide range of view scene
Reconstruction, global coordinate system lower skeleton point three-dimensional localization, the real-time update of the control of accumulated error and a wide range of three-dimensional scenic.
2. a kind of three-dimensional global visualization monitoring system structure updated based on skeletal point local dynamic according to claim 1
Construction method, it is characterised in that the multiple-camera that the array camera of comprehensive utilization mobile robot carrying and scene are placed is regarded
Frequency information, the global three-dimensional scenic that real-time local dynamic station updates is built based on static panorama three-dimensional map.
3. a kind of three-dimensional global visualization monitoring system structure updated based on skeletal point local dynamic according to claim 1
Construction method, it is characterised in that for a wide range of global three-dimensional scenic rebuild, in subsequent scenario renewal process, only updates
Region of variation in scene, and expanded by the minimum enclosed rectangle of region of variation, lift the local three-dimensional scenic updated effective
Property, and significant increase renewal efficiency, enhance the real-time of three-dimensional global visualization monitoring system.
4. a kind of three-dimensional global visualization monitoring system structure updated based on skeletal point local dynamic according to claim 1
Construction method, it is characterised in that this method is using the multiple cameras that there is the public visual field in scene, rather than reinstalls binocular camera
Or other image depth information measuring apparatus, compatible to a certain extent existing monitoring system, and greatly reduce system operation
Cost, makes this method application more extensive.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710347732.1A CN107194991B (en) | 2017-05-17 | 2017-05-17 | Three-dimensional global visual monitoring system construction method based on skeleton point local dynamic update |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710347732.1A CN107194991B (en) | 2017-05-17 | 2017-05-17 | Three-dimensional global visual monitoring system construction method based on skeleton point local dynamic update |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107194991A true CN107194991A (en) | 2017-09-22 |
| CN107194991B CN107194991B (en) | 2021-01-01 |
Family
ID=59873846
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710347732.1A Active CN107194991B (en) | 2017-05-17 | 2017-05-17 | Three-dimensional global visual monitoring system construction method based on skeleton point local dynamic update |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107194991B (en) |
Cited By (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108010084A (en) * | 2017-11-20 | 2018-05-08 | 深圳市云之梦科技有限公司 | A kind of depth camera is rebuild and method, system, the equipment of automatic Calibration |
| CN108510434A (en) * | 2018-02-12 | 2018-09-07 | 中德（珠海）人工智能研究院有限公司 | The method for carrying out three-dimensional modeling by ball curtain camera |
| CN109003325A (en) * | 2018-06-01 | 2018-12-14 | 网易（杭州）网络有限公司 | A kind of method of three-dimensional reconstruction, medium, device and calculate equipment |
| CN109589563A (en) * | 2018-12-29 | 2019-04-09 | 南京华捷艾米软件科技有限公司 | A kind of auxiliary method and system of dancing posture religion based on 3D body-sensing camera |
| WO2019089995A1 (en) * | 2016-11-01 | 2019-05-09 | Brachium Labs Llc | Vision guided robot path programming |
| CN109947242A (en) * | 2019-02-26 | 2019-06-28 | 贵州翰凯斯智能技术有限公司 | A kind of factory's virtual application system and application method based on information fusion |
| CN110136083A (en) * | 2019-05-14 | 2019-08-16 | 深圳大学 | A kind of the base map update method and device of combination interactive mode |
| WO2020073818A1 (en) * | 2018-10-12 | 2020-04-16 | 锥能机器人（上海）有限公司 | Visual localization method and system for visual editing |
| CN111046848A (en) * | 2019-12-30 | 2020-04-21 | 广东省实验动物监测所 | Gait monitoring method and system based on animal running platform |
| CN111651539A (en) * | 2020-05-22 | 2020-09-11 | 西北农林科技大学 | Method for realizing quick updating of plane map elements by using close-range remote sensing technology |
| CN111782739A (en) * | 2019-04-04 | 2020-10-16 | 西安四维图新信息技术有限公司 | Map updating method and device |
| CN113009590A (en) * | 2021-02-01 | 2021-06-22 | 西南科技大学 | Three-dimensional foreign matter detection system and method in vehicle bottom security inspection system |
| CN114494582A (en) * | 2021-12-30 | 2022-05-13 | 重庆交通大学 | Three-dimensional model dynamic updating method based on visual perception |
| CN112215887B (en) * | 2019-07-09 | 2023-09-08 | 深圳市优必选科技股份有限公司 | Pose determining method and device, storage medium and mobile robot |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101226640B (en) * | 2007-12-21 | 2010-08-18 | 西北工业大学 | Method for capturing movement based on multiple binocular stereovision |
| CN102364299B (en) * | 2011-08-30 | 2015-01-14 | 西南科技大学 | Calibration technology for multiple structured light projected three-dimensional profile measuring heads |
| CN103237192B (en) * | 2012-08-20 | 2016-03-30 | 苏州大学 | Intelligent video monitoring system based on multi-camera data fusion |
| CN105427293A (en) * | 2015-11-11 | 2016-03-23 | 中国科学院深圳先进技术研究院 | Indoor scene scanning reconstruction method and apparatus |
-
2017
- 2017-05-17 CN CN201710347732.1A patent/CN107194991B/en active Active
Cited By (19)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10556347B2 (en) | 2016-11-01 | 2020-02-11 | Brachium, Inc. | Vision guided robot path programming |
| WO2019089995A1 (en) * | 2016-11-01 | 2019-05-09 | Brachium Labs Llc | Vision guided robot path programming |
| CN108010084A (en) * | 2017-11-20 | 2018-05-08 | 深圳市云之梦科技有限公司 | A kind of depth camera is rebuild and method, system, the equipment of automatic Calibration |
| CN108510434B (en) * | 2018-02-12 | 2019-08-20 | 中德（珠海）人工智能研究院有限公司 | The method for carrying out three-dimensional modeling by ball curtain camera |
| CN108510434A (en) * | 2018-02-12 | 2018-09-07 | 中德（珠海）人工智能研究院有限公司 | The method for carrying out three-dimensional modeling by ball curtain camera |
| CN109003325A (en) * | 2018-06-01 | 2018-12-14 | 网易（杭州）网络有限公司 | A kind of method of three-dimensional reconstruction, medium, device and calculate equipment |
| CN109003325B (en) * | 2018-06-01 | 2023-08-04 | 杭州易现先进科技有限公司 | Three-dimensional reconstruction method, medium, device and computing equipment |
| WO2020073818A1 (en) * | 2018-10-12 | 2020-04-16 | 锥能机器人（上海）有限公司 | Visual localization method and system for visual editing |
| CN109589563A (en) * | 2018-12-29 | 2019-04-09 | 南京华捷艾米软件科技有限公司 | A kind of auxiliary method and system of dancing posture religion based on 3D body-sensing camera |
| CN109947242A (en) * | 2019-02-26 | 2019-06-28 | 贵州翰凯斯智能技术有限公司 | A kind of factory's virtual application system and application method based on information fusion |
| CN111782739A (en) * | 2019-04-04 | 2020-10-16 | 西安四维图新信息技术有限公司 | Map updating method and device |
| CN110136083A (en) * | 2019-05-14 | 2019-08-16 | 深圳大学 | A kind of the base map update method and device of combination interactive mode |
| CN112215887B (en) * | 2019-07-09 | 2023-09-08 | 深圳市优必选科技股份有限公司 | Pose determining method and device, storage medium and mobile robot |
| CN111046848A (en) * | 2019-12-30 | 2020-04-21 | 广东省实验动物监测所 | Gait monitoring method and system based on animal running platform |
| CN111651539B (en) * | 2020-05-22 | 2023-03-03 | 西北农林科技大学 | Method for realizing quick updating of plane map elements by using close-range remote sensing technology |
| CN111651539A (en) * | 2020-05-22 | 2020-09-11 | 西北农林科技大学 | Method for realizing quick updating of plane map elements by using close-range remote sensing technology |
| CN113009590A (en) * | 2021-02-01 | 2021-06-22 | 西南科技大学 | Three-dimensional foreign matter detection system and method in vehicle bottom security inspection system |
| CN113009590B (en) * | 2021-02-01 | 2022-04-08 | 西南科技大学 | Three-dimensional foreign matter detection system and method in vehicle bottom security inspection system |
| CN114494582A (en) * | 2021-12-30 | 2022-05-13 | 重庆交通大学 | Three-dimensional model dynamic updating method based on visual perception |
Also Published As
| Publication number | Publication date |
|---|---|
| CN107194991B (en) | 2021-01-01 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107194991A (en) | A kind of three-dimensional global visualization monitoring system construction method updated based on skeletal point local dynamic | |
| Huang et al. | Research on multi-camera calibration and point cloud correction method based on three-dimensional calibration object | |
| CN104778694B (en) | A kind of parametrization automatic geometric correction method shown towards multi-projection system | |
| CN102032878B (en) | Accurate on-line measurement method based on binocular stereo vision measurement system | |
| Teller et al. | Calibrated, registered images of an extended urban area | |
| CN108053476B (en) | Human body parameter measuring system and method based on segmented three-dimensional reconstruction | |
| CN110487216A (en) | A kind of fringe projection 3-D scanning method based on convolutional neural networks | |
| Hoppe et al. | Online Feedback for Structure-from-Motion Image Acquisition. | |
| CN103345736B (en) | A kind of virtual viewpoint rendering method | |
| Xie et al. | Study on construction of 3D building based on UAV images | |
| CN108711185A (en) | Joint rigid moves and the three-dimensional rebuilding method and device of non-rigid shape deformations | |
| CN110232655A (en) | A kind of double light image splicings of the Infrared-Visible for coal yard and fusion method | |
| CN110246186A (en) | A kind of automatized three-dimensional colour imaging and measurement method | |
| CN108665535A (en) | A kind of three-dimensional structure method for reconstructing and system based on coding grating structured light | |
| CN103278138A (en) | Method for measuring three-dimensional position and posture of thin component with complex structure | |
| CN111091076B (en) | Tunnel limit data measuring method based on stereoscopic vision | |
| Remondino | 3-D reconstruction of static human body shape from image sequence | |
| CN111060006A (en) | Viewpoint planning method based on three-dimensional model | |
| CN110049304A (en) | A kind of method and device thereof of the instantaneous three-dimensional imaging of sparse camera array | |
| CN113450416B (en) | TCSC method applied to three-dimensional calibration of three-dimensional camera | |
| CN107633537A (en) | A kind of camera calibration method based on projection | |
| CN107610215B (en) | High-precision multi-angle oral cavity three-dimensional digital imaging model construction method | |
| CN113345084A (en) | Three-dimensional modeling system and three-dimensional modeling method | |
| CN108830921A (en) | Laser point cloud reflected intensity correcting method based on incident angle | |
| CN107798731A (en) | A kind of method based on satellite image automatic modeling |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |