CN110443898A - A kind of AR intelligent terminal target identification system and method based on deep learning - Google Patents
A kind of AR intelligent terminal target identification system and method based on deep learning Download PDFInfo
- Publication number
- CN110443898A CN110443898A CN201910739784.2A CN201910739784A CN110443898A CN 110443898 A CN110443898 A CN 110443898A CN 201910739784 A CN201910739784 A CN 201910739784A CN 110443898 A CN110443898 A CN 110443898A
- Authority
- CN
- China
- Prior art keywords
- image
- deep learning
- intelligent terminal
- information
- camera
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000013135 deep learning Methods 0.000 title claims abstract description 113
- 238000000034 method Methods 0.000 title claims abstract description 42
- 238000012545 processing Methods 0.000 claims abstract description 92
- 230000003190 augmentative effect Effects 0.000 claims abstract description 37
- 238000006243 chemical reaction Methods 0.000 claims abstract description 16
- 230000003993 interaction Effects 0.000 claims abstract description 4
- 238000013136 deep learning model Methods 0.000 claims description 36
- 238000004891 communication Methods 0.000 claims description 18
- 230000008569 process Effects 0.000 claims description 17
- 238000004364 calculation method Methods 0.000 claims description 16
- 230000004927 fusion Effects 0.000 claims description 15
- 238000000605 extraction Methods 0.000 claims description 12
- 230000000007 visual effect Effects 0.000 claims description 10
- 230000003287 optical effect Effects 0.000 claims description 6
- 230000001133 acceleration Effects 0.000 claims description 5
- 230000002708 enhancing effect Effects 0.000 claims description 3
- 238000007619 statistical method Methods 0.000 claims description 3
- 238000013507 mapping Methods 0.000 claims description 2
- 230000001131 transforming effect Effects 0.000 claims description 2
- 235000013399 edible fruits Nutrition 0.000 claims 1
- 238000001514 detection method Methods 0.000 description 16
- 238000004422 calculation algorithm Methods 0.000 description 12
- 230000000694 effects Effects 0.000 description 9
- 230000000875 corresponding effect Effects 0.000 description 6
- 239000011521 glass Substances 0.000 description 6
- 238000005516 engineering process Methods 0.000 description 5
- 238000013527 convolutional neural network Methods 0.000 description 4
- 239000003550 marker Substances 0.000 description 4
- 230000009466 transformation Effects 0.000 description 4
- 230000008901 benefit Effects 0.000 description 3
- 230000015572 biosynthetic process Effects 0.000 description 2
- 230000007613 environmental effect Effects 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 238000005286 illumination Methods 0.000 description 2
- 238000005259 measurement Methods 0.000 description 2
- 238000003786 synthesis reaction Methods 0.000 description 2
- 241001465754 Metazoa Species 0.000 description 1
- 230000004888 barrier function Effects 0.000 description 1
- 230000002596 correlated effect Effects 0.000 description 1
- 230000007812 deficiency Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 230000010365 information processing Effects 0.000 description 1
- 239000000203 mixture Substances 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
- 238000010977 unit operation Methods 0.000 description 1
- 239000011800 void material Substances 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/22—Matching criteria, e.g. proximity measures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T19/00—Manipulating 3D models or images for computer graphics
- G06T19/006—Mixed reality
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Data Mining & Analysis (AREA)
- General Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Computer Hardware Design (AREA)
- Computer Graphics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Image Analysis (AREA)
- Image Processing (AREA)
Abstract
The AR intelligent terminal target identification system based on deep learning that the invention discloses a kind of, comprising: deep learning image processing unit exports recognition result image after capable of obtaining the shooting image recognition of AR intelligent terminal camera；Position and attitude computing unit can determine that current relative position, camera direction and posture obtain the point of interest in recognition result field of view according to recognition result image；The spatial information inquired is associated with attribute information to recognition result image by Spatial data query unit；Augmented reality information unit after the recognition result image is converted to 3-D image by coordinate conversion, is shown as dummy object information for human-computer interaction after adding corresponding informance to the 3-D image.The system and method identifies that accuracy is high, is arranged independent of preparatory label, and flexibility is more preferable.
Description
Technical field
The present invention relates to field of target recognition more particularly to a kind of AR intelligent terminal target identification systems based on deep learning
System and method.
Background technique
Augmented reality (Augmented Reality-AR) technology makes void by adding dummy object in real scene
Quasi- object combines together with true environment, and understanding and experience of the people to true environment can be enhanced.Augmented reality (i.e. AR---
Augmented Reality) it more preferably by the key point that subject fusion enters true environment is infused using which kind of identification technology and tracking
Volume algorithm.
The identification technology that existing augmented reality uses mostly is the tracking that target is realized based on mark (Marker) mode
And registration, the scene used are confined to controllable environment, while needing to prepare in advance to make correlated identities content.Therefore this base
In the identification method of mark, the indoor environment controllable in environmental factor is applied due to being, and large-scale geographical environment outdoors
In, it is limited by various external environmental factors, it is difficult to large-scale to use.With global-positioning technology (GPS), sensor technology
The development of (gyroscope, compass, Inertial Measurement Unit IMU) and computer vision technique and universal, using GPS positioning, in conjunction with
The multiple sensors such as Inertial Measurement Unit, magnetic sensor assist determining position and equipment posture to realize, are also able to achieve and are used in
Augmented reality carries out the target following and registration in Outdoor Scene.But it is limited by the precision and error of GPS, outdoor target positioning produces
The probability performance of raw offset is obvious, and effect is also unsatisfactory, and the function being able to achieve is also limited, and also there is no extensive at present
Application.Using at most in AR identification application field at present is to be based on mark (Marker) this form, main including sharp
With Marker, two dimensional code, natural picture as identification figure, usually square and all clear and legible four sides template clip
Piece, this kind of recognizer is relatively easy, is usually solved with template matching algorithm, and effect is also good.Identification process based on mark
Substantially are as follows: first pass through management backstage and upload identification figure to server, server can carry out gray proces to uploading pictures, and picture becomes
For black white image；Then the characteristic point of black white image is extracted；Characteristic point data is packaged again；Contrast characteristic when program is run
Point data packet.For augmented reality mainly using combined tracking registration algorithm, the algorithm of view-based access control model identification is its core again at present
The heart.But it is this using extract characteristic point as the visual identification algorithm of core it is not high there are computationally intensive and precision the problems such as, carrying out
When image recognition, the structural information of original image can be lost, is unable to reach the efficiency and essence for improving image recognition and target detection
Degree.
The above-mentioned prior art one at least has the disadvantage in that
(1) due to carrying out AR identification based on mark, it is necessary to identification figure (Marker) be arranged in advance or carried out to identification object
Required Image Acquisition, then carry out template matching；
(2) the instant recognition mode versatility of AR intelligent terminal based on mark is poor, inflexible for use；
(3) since the AR identification based on mark is all based on template greatly, AR application extension has been fettered to answering more on a large scale
Use scene；
(4) when terminal carries out AR identification based on the mode of mark, the data volume that can be handled simultaneously is very small；
(5) the AR identification method based on mark can not also be good to the discrimination problem of barrier and vertical plane in plane
It solves.
Summary of the invention
Based on the problems of prior art, the AR intelligence based on deep learning that the object of the present invention is to provide a kind of is eventually
Target identification system is held, the existing AR carried out based on mark is can solve and identifies poor existing flexibility, accuracy difference and using ring
The problems such as border is limited can improve the efficiency and precision of target identification and target detection under the premise of convenient use.
The purpose of the present invention is what is be achieved through the following technical solutions:
Embodiment of the present invention provides a kind of AR intelligent terminal target identification system based on deep learning, comprising:
Deep learning image processing unit, position and attitude computing unit, Spatial data query unit and augmented reality information
Unit；Wherein,
The deep learning image processing unit can obtain the shooting image of AR intelligent terminal camera, by the shooting figure
As being converted to identification image, after being identified with image with deep learning mode by deep learning model to the identification, output
Recognition result image；
The position and attitude computing unit is communicated to connect with the deep learning image processing unit, can be according to the depth
The recognition result image of degree study image processing unit output, the GPS information for obtaining the AR intelligent terminal determine the identification
The current relative position of geography target and the AR intelligent terminal in the result images visual field, and obtain the AR intelligent terminal
Sensor information determines camera direction and posture, and according to the determination of determining current relative position, camera direction and posture
Point of interest in recognition result field of view；
The Spatial data query unit is communicated to connect with the position and attitude computing unit, can be according to the position appearance
The point of interest in the recognition result field of view that state computing unit determines, and the spatial information and category that point of interest will be inquired
Property information association is to the recognition result image；
The augmented reality information unit, respectively with the deep learning image processing unit, position and attitude computing unit
It is connected with Spatial data query unit communication, the recognition result image that can be exported by the deep learning image processing unit,
It is determined with the GPS information of the AR intelligent terminal got and is identified target and AR intelligence in the recognition result image
The current relative position of terminal, while by the AR intelligent terminal calculated camera internal parameter in real time, it is incorporated in the world
External parameter in coordinate system sets virtual camera, and the AR intelligent terminal is obtained by the position and attitude computing unit
Posture information, and the posture of the virtual camera is demarcated, three-dimensional virtual scene is become by the virtual camera
It changes, the image of three-dimensional virtual scene after transformation is added in the image of actual scene and carries out virtual reality fusion, passes through optical projection
Image after showing the virtual reality fusion；And the space for the three-dimensional virtual scene for inquiring the Spatial data query unit
Information and attribute information (type includes: text information, three-D grain information, 3-D graphic information three classes), are rendered into true generation
On the point of interest on boundary, carries out augmented reality and show.
Embodiment of the present invention also provides a kind of AR intelligent terminal target identification method based on deep learning, using this hair
The bright AR intelligent terminal target identification system based on deep learning, comprising the following steps:
Step 1, deep learning image procossing: obtaining the shooting image of AR intelligent terminal camera, and the shooting image is turned
It is changed to identification image, after being identified with image with deep learning mode by deep learning model to the identification, output identification
Result images；
Step 2, AR intelligent terminal information is obtained: the recognition result figure exported according to the deep learning image processing unit
Picture, the GPS information for obtaining the AR intelligent terminal determine geography target and AR intelligence in the recognition result field of view
The current relative position of terminal, and the sensor information of the acquisition AR intelligent terminal determine camera direction and posture, and root
The point of interest in the recognition result field of view is determined according to determining current relative position, camera direction and posture；
Step 3, incident space data: the recognition result field of view determined according to the position and attitude computing unit
Interior point of interest, and the spatial information for inquiring point of interest is associated with attribute information to the recognition result image；
Step 4, information enhancement exports: the recognition result image exported by the deep learning image processing unit, with
The GPS information of the AR intelligent terminal got, which determines, is identified target and AR intelligence eventually in the recognition result image
The current relative position at end, while by the AR intelligent terminal calculated camera internal parameter in real time, it is incorporated in world's seat
External parameter in mark system sets virtual camera, and the appearance of the AR intelligent terminal is obtained by the position and attitude computing unit
State information, and the posture of the virtual camera is demarcated, three-dimensional virtual scene is converted by the virtual camera,
The image of three-dimensional virtual scene after transformation is added in the image of actual scene and carries out virtual reality fusion, is shown by optical projection
Image after the virtual reality fusion；And the spatial information for the three-dimensional virtual scene for inquiring the Spatial data query unit
With attribute information (type includes: text information, three-D grain information, 3-D graphic information three classes), it is rendered into real world
On point of interest, carries out augmented reality and show.
As seen from the above technical solution provided by the invention, the AR provided in an embodiment of the present invention based on deep learning
Intelligent terminal target identification system, it has the advantage that:
By using deep learning image processing unit and position and attitude computing unit, Spatial data query unit and enhancing
Real information unit cooperation, forms a kind of target identification system suitable for AR intelligent terminal, due to being carried out based on deep learning
Image recognition can use any object (such as: the cover of book) with enough characteristic points as datum plane, without prior
Certain moduli plate is made, the constraint that template applies AR is got rid of；Without using specific information, by being parsed to image,
Virtual coordinates are established on real world images, and are overlapped synthesis with real world images；The application of deep learning model is increased outdoors
In Tracing Registration identifying system in strong reality, the different image recognition and target detection carried out with traditional algorithm can reach
Higher identification and positioning accuracy；Performance of the AR intelligent terminal under external noise circumstance is improved, external rings are reduced
Border is such as: the influence of size, distance, illumination, angle, weather conditions to AR recognition effect improves the robustness of identifying system；
Deep learning SSD engine is integrated in the client of AR intelligent terminal, it, can be under the premise of not past mobile terminal device performance
Improve the real-time and accuracy of identifying system；The target identification system of the deep learning can be integrated in the client of AR intelligent terminal
System, reduces the dependence to network.
Detailed description of the invention
In order to illustrate the technical solution of the embodiments of the present invention more clearly, required use in being described below to embodiment
Attached drawing be briefly described, it should be apparent that, drawings in the following description are only some embodiments of the invention, for this
For the those of ordinary skill in field, without creative efforts, it can also be obtained according to these attached drawings other
Attached drawing.
Fig. 1 is that the composition of the AR intelligent terminal target identification system provided in an embodiment of the present invention based on deep learning is illustrated
Figure；
Fig. 2 is the flow chart of the AR intelligent terminal target identification method provided in an embodiment of the present invention based on deep learning；
Fig. 3 is the deep learning of the AR intelligent terminal target identification method provided in an embodiment of the present invention based on deep learning
The flow chart of image processing process；
Fig. 4 is the another way of the AR intelligent terminal target identification system provided in an embodiment of the present invention based on deep learning
Constitute schematic diagram.
Specific embodiment
Below with reference to particular content of the invention, technical solution in the embodiment of the present invention is clearly and completely retouched
It states, it is clear that described embodiments are only a part of the embodiments of the present invention, instead of all the embodiments.Based on the present invention
Embodiment, every other embodiment obtained by those of ordinary skill in the art without making creative efforts,
Belong to protection scope of the present invention.The content being not described in detail in the embodiment of the present invention belongs to professional and technical personnel in the field
The well known prior art.
As shown in Figure 1, the embodiment of the present invention provides a kind of AR intelligent terminal target identification system based on deep learning, packet
It includes:
Deep learning image processing unit, position and attitude computing unit, Spatial data query unit and augmented reality information
Unit；Wherein,
The deep learning image processing unit can obtain the shooting image of AR intelligent terminal camera, by the shooting figure
As being converted to identification image, after being identified with image with deep learning mode by deep learning model to the identification, output
Recognition result image；
The position and attitude computing unit is communicated to connect with the deep learning image processing unit, can be according to the depth
The recognition result image of degree study image processing unit output, the GPS information for obtaining the AR intelligent terminal determine the identification
The current relative position of geography target and the AR intelligent terminal in the result images visual field, and obtain the AR intelligent terminal
Sensor information determines camera direction and posture, and according to the determination of determining current relative position, camera direction and posture
Point of interest in recognition result field of view；
The Spatial data query unit is communicated to connect with the position and attitude computing unit, can be according to the position appearance
The point of interest in the recognition result field of view that state computing unit determines, and the spatial information and category that point of interest will be inquired
Property information association is to the recognition result image；
The augmented reality information unit, respectively with the deep learning image processing unit, position and attitude computing unit
It is connected with Spatial data query unit communication, the recognition result image exported by the deep learning image processing unit, with
The GPS information of the AR intelligent terminal got, which determines, is identified target and AR intelligence eventually in the recognition result image
The current relative position at end, while by the AR intelligent terminal calculated camera internal parameter in real time, it is incorporated in world's seat
External parameter in mark system sets virtual camera, and the appearance of the AR intelligent terminal is obtained by the position and attitude computing unit
State information, and the posture of the virtual camera is demarcated, three-dimensional virtual scene is converted by the virtual camera,
The image of three-dimensional virtual scene after transformation is added in the image of actual scene and carries out virtual reality fusion, is shown by optical projection
Image after the virtual reality fusion；And the spatial information for the three-dimensional virtual scene for inquiring the Spatial data query unit
With attribute information (type includes: text information, three-D grain information, 3-D graphic information three classes), it is rendered into real world
On point of interest, carries out augmented reality and show.The augmented reality information unit during mobile target identification constantly iteration this
Treatment process.
In above-mentioned target identification system, deep learning image processing unit includes:
Camera image obtains module, image conversion module, SSD deep learning model and output module；Wherein,
The camera image obtains module, can obtain the shooting image of AR intelligent terminal camera；
Described image conversion module obtains module communication connection with the camera image, can obtain the camera image
The shooting image that module obtains is converted to identification image；
The SSD deep learning model is communicated to connect with described image conversion module, can be by deep learning mode to institute
Identification is stated to be identified to obtain recognition result image with image；
The output module is connect with the SSD deep learning modeling communication, can export the SSD deep learning model
Identify obtained recognition result image.
Above-mentioned SSD deep learning model is the convolutional neural networks model for running SSD algorithm, utilizes convolutional neural networks mould
Type is that the image recognition algorithm of core can significantly improve the efficiency and precision of image recognition and target detection.It overcomes general
Deep learning model can lose the structural information of original image when carrying out image recognition, thus influence asking for recognition effect
Topic.
In above-mentioned target identification system, the position and attitude computing unit includes:
Relative position computing module, camera direction computing module, camera Attitude Calculation module and output processing module；
The relative position computing module is communicated to connect with the deep learning image processing unit, can be according to the depth
The recognition result image of degree study image processing unit output, the GPS information for obtaining the AR intelligent terminal determine relative position；
The camera direction computing module is communicated to connect with the deep learning image processing unit, can be according to the depth
The recognition result image of degree study image processing unit output, the magnetometric sensor information for obtaining the AR intelligent terminal determine phase
Machine direction；
The camera Attitude Calculation module is communicated to connect with the deep learning image processing unit, can be according to the depth
The recognition result image of degree study image processing unit output, the acceleration transducer information for obtaining the AR intelligent terminal determine
Camera posture；
The output processing module, respectively with the relative position computing module, camera direction computing module and camera appearance
The communication connection of state computing module the determining relative position of computing module, the camera direction can calculate depending on that relative position
The camera posture that the camera direction and the camera Attitude Calculation module that module determines determine determines the recognition result image view
Point of interest in open country.
In above-mentioned target identification system, the Spatial data query unit includes:
Attribute query module and perimeter query module；Wherein,
The attribute query module is communicated to connect with the position and attitude computing unit, can be according to the position and attitude meter
Calculate module output space querying condition, search meet the space querying condition spatial object distribution, orientation calculate and into
The corresponding statistical analysis of row, and support the SQL query language of standard；
The perimeter query module is communicated to connect with the position and attitude computing unit, can be according to the position and attitude meter
The periphery P OI querying condition of module output is calculated, the POI point of interest met in the present viewing field of periphery P OI querying condition is searched
Information, and it is used for augmented reality information unit Overlapping display.
In above-mentioned target identification system, augmented reality information unit includes:
Coordinate transferring, Text extraction module, three-D grain processing module, three-dimensional graph process module and virtual
Object information output module；Wherein,
The coordinate transferring, respectively with the deep learning image processing unit, position and attitude computing unit and sky
Between data query unit communication connect, the recognition result image, described that can be exported according to the deep learning image processing unit
Point of interest in the visual field that position and attitude computing unit determines is converted by coordinate the recognition result image being converted to three-dimensional figure
Picture；
The Text extraction module is communicated to connect with the coordinate transferring, can be to the coordinate transferring
3-D image after conversion adds text information；
The three-D grain processing module is communicated to connect with the Text extraction module, can be to the text information
Processing module adds the 3-D image addition three-D grain information after text information；
The three-dimensional graph process module is communicated to connect with the three-D grain processing module, can be by the three-D grain
3-D image after processing module addition three-D grain information is plotted as the 3-D graphic corresponding to dummy object；
The dummy object message output module is communicated to connect with the three-dimensional graph process module, can be by the three-dimensional
Treated corresponds to the 3-D graphic of dummy object and export to the display of the AR intelligent terminal and show for pattern process module
Human-computer interaction is used for for dummy object information.
In above-mentioned target identification system, in the augmented reality information unit, AR intelligent terminal calculated camera in real time
Inner parameter includes: the focal length and imaging screen size of video camera；
External parameter in world coordinate system are as follows: position and direction angle.
In above-mentioned target identification system, deep learning image processing unit, position and attitude computing unit, Spatial data query
Unit and augmented reality information unit are each provided in AR intelligent terminal；It is formed a kind of completely in the target knowledge of AR intelligent terminal operation
Other system；
Alternatively,
Referring to fig. 4, the position and attitude computing unit, Spatial data query unit and augmented reality information unit are each provided at
In AR intelligent terminal, the SSD deep learning model of the deep learning image processing unit is located at logical with the AR intelligent terminal
In the server for believing connection, other modules of the deep learning image processing unit are located in the AR intelligent terminal.It is formed
A kind of target identification system with the server in AR intelligent terminal and cloud, recognition capability are stronger.
As shown in Fig. 2, the embodiment of the present invention also provides a kind of AR intelligent terminal target identification method based on deep learning,
Using the above-mentioned AR intelligent terminal target identification system based on deep learning, comprising the following steps:
Step 1, deep learning image procossing: obtaining the shooting image of AR intelligent terminal camera, and the shooting image is turned
It is changed to identification image, after being identified with image with deep learning mode by deep learning model to the identification, output identification
Result images；
Step 2, AR intelligent terminal information is obtained: the recognition result figure exported according to the deep learning image processing unit
Picture, the GPS information for obtaining the AR intelligent terminal determine geography target and AR intelligence in the recognition result field of view
The current relative position of terminal, and the sensor information of the acquisition AR intelligent terminal determine camera direction and posture, and root
The point of interest in the recognition result field of view is determined according to determining current relative position, camera direction and posture；
Step 3, incident space data: the recognition result field of view determined according to the position and attitude computing unit
Interior point of interest, and the spatial information for inquiring point of interest is associated with attribute information to the recognition result image；
Step 4, information enhancement exports: the recognition result image exported by the deep learning image processing unit, with
The GPS information of the AR intelligent terminal got, which determines, is identified target and AR intelligence eventually in the recognition result image
The current relative position at end, while by the AR intelligent terminal calculated camera internal parameter in real time, it is incorporated in world's seat
External parameter in mark system sets virtual camera, and the appearance of the AR intelligent terminal is obtained by the position and attitude computing unit
State information, and the posture of the virtual camera is demarcated, three-dimensional virtual scene is converted by the virtual camera,
The image of three-dimensional virtual scene after transformation is added in the image of actual scene and carries out virtual reality fusion, is shown by optical projection
Image after the virtual reality fusion；And the spatial information for the three-dimensional virtual scene for inquiring the Spatial data query unit
With attribute information (type includes: text information, three-D grain information, 3-D graphic information three classes), it is rendered into real world
On point of interest, carries out augmented reality and show.
As shown in figure 3, the deep learning image procossing of above-mentioned target identification method step 1 includes:
Step 11, camera image obtaining step obtains the shooting image of AR intelligent terminal camera；
Step 12, the camera image is obtained the shooting image that module obtains and is converted to identification use by image conversion step
Image；
Step 13, SSD deep learning step identifies the identification with image by deep learning mode and is known
Other result images；
Step 14, step is exported, the recognition result image that the SSD deep learning model identifies is exported.
The acquisition AR intelligent terminal information of above-mentioned target identification method step 2 includes:
Step 21, relative position calculates step, the recognition result figure exported according to the deep learning image processing unit
Picture, the GPS information for obtaining the AR intelligent terminal determine geography target and AR intelligence in the recognition result field of view
The current relative position of terminal；
Step 22, camera direction calculates step, the recognition result figure exported according to the deep learning image processing step
Picture, the magnetometric sensor information for obtaining the AR intelligent terminal determine camera direction；
Step 23, camera Attitude Calculation step, the recognition result figure exported according to the deep learning image processing step
Picture, the acceleration transducer information for obtaining the AR intelligent terminal determine camera posture；
Step 24, processing step is exported, calculates current relative position, the phase that step determines depending on that relative position
The camera posture that the camera direction and the camera Attitude Calculation step that machine direction calculating step determines determine determines the identification
Point of interest in the result images visual field.
The information enhancement of above-mentioned target identification method step 4 exports
Step 41, three-dimensional registration step determines three-dimensional using inside and outside portion's parameter of the AR intelligent terminal
Corresponding parameter between the image of scene and acquisition environment, the image for calculating the three-dimensional virtual scene will be added to true ring
Three-dimensional coordinate information in border；The step is contacted actual situation scene using inside and outside portion's parameter of the AR intelligent terminal
Come, obtains the virtual reality fusion that three-dimensional coordinate information is next step and show and prepare；The external parameter is the AR intelligent terminal
Position and angle in collection process, the inner parameter be the camera group of the AR intelligent terminal inherently
Structure determines (including focal length, pixel aspect ratio etc.)；
Step 42, enhancement information shows step, and three-dimensional coordinate information is calculated by the three-dimensional registration step, determines
The image of the three-dimensional virtual scene is added to the mapping position in world coordinate system, and by the image of the three-dimensional virtual scene
(including: text, three-D grain, 3-D image, geometrical model etc.) shows output after correctly transforming to projection plane.After making fusion
Scene visually see and do not separate sense.
The present invention at least has the advantages that
(1) the AR intelligent identifying system based on deep learning can with it is any with enough characteristic points object (such as: book
Cover) as datum plane without making certain moduli plate in advance get rid of the constraint that template applies AR；
(2) specific information is not used, by parsing to image, establishes virtual coordinates on real world images, and with
Real world images are overlapped synthesis；
(3) deep learning model SSD is applied in the Tracing Registration identifying system in augmented reality outdoors, different and biography
The image recognition and target detection that algorithm of uniting carries out, can reach higher identification and positioning accuracy；
(4) performance of the AR intelligent terminal under external noise circumstance is improved, reduces external environment such as: size,
The influence to AR recognition effect such as distance, illumination, angle, weather conditions, improves the robustness of identifying system；
(5) deep learning SSD engine is integrated in AR intelligent terminal client, not past mobile terminal device performance
Under the premise of, the real-time and accuracy of identifying system can be improved；
(6) deep learning SSD engine is integrated in AR intelligent terminal client, reduces the dependence to network；
The embodiment of the present invention is specifically described in further detail below.
The embodiment of the present invention provides a kind of AR intelligent terminal identifying system based on SSD deep learning model, the system energy
Image is obtained as input source, when Camera is obtained using the Camera that AR intelligent terminal (such as by taking AR intelligent glasses as an example) carries
To after a frame image, the image that the image of YUV is switched to RGB by image conversion is first passed around, the image of RGB is then sent into target
Detection identification is carried out in detection unit (i.e. SSD deep learning model), object detection unit uses one and runs similar SSD calculation
The CNN model of method carries out object detection identification to input picture；SSD deep learning model is namely imported into AR intelligence eventually
End that pedestrian, vehicle, steamer, the various animals etc. in scene can be distinguished for carrying out target detection to the image of input
Information, and the area information of each detection object in the picture is calculated in real time, and calculates the frame of detection object in real time
And classification information, realize visual identity.It is drawn simultaneously on AR intelligent terminal using Canvas 2D to draw object to be detected
Frame；Textured rotatable 3D figure enhancement information is developed using OpenGL ES, then passes through virtual 3D figure and video
The superposition of stream achievees the effect that augmented reality is shown.
The system be based on AR intelligent terminal platform, the system architecture using pure client Technical Architecture mode
(referring to Fig. 1), the SSD deep learning model of operation SSD algorithm are stored in the storage of AR intelligent terminal local after being packaged, own
The content information that space, attribute information and the enhancing of identification target are shown is stored in the local data base of AR intelligent terminal
In SQLlite.
Specifically, the target identification system mainly includes four units, it may be assumed that deep learning image processing unit, position appearance
State computing unit, Spatial data query unit and augmented reality information unit.
Wherein, deep learning image processing unit includes: that camera image obtains module, image conversion module, SSD depth
Practise model and output module；The camera image obtains module, can obtain the shooting image of AR intelligent terminal camera；Described image
Conversion module obtains module communication connection with the camera image, the camera image can be obtained the shooting figure that module obtains
As being converted to identification image；The SSD deep learning model communicates to connect with described image conversion module, can pass through depth
Mode of learning identifies the identification with image to obtain recognition result image；The output module, with the SSD depth
Modeling communication connection is practised, the recognition result image that the SSD deep learning model identifies can be exported.
Position and attitude computing unit includes: relative position computing module, camera direction computing module, camera Attitude Calculation mould
Block and output processing module；
Relative position computing module is communicated to connect with the deep learning image processing unit, can be according to the depth
The recognition result image of image processing unit output is practised, the GPS information for obtaining the AR intelligent terminal determines current relative position；
The camera direction computing module is communicated to connect with the deep learning image processing unit, can be according to the deep learning figure
As the recognition result image that processing unit exports, the magnetometric sensor information for obtaining the AR intelligent terminal determines camera direction；
The camera Attitude Calculation module is communicated to connect with the deep learning image processing unit, can be according to the deep learning figure
As the recognition result image that processing unit exports, the acceleration transducer information for obtaining the AR intelligent terminal determines camera appearance
State；The output processing module, respectively with the relative position computing module, camera direction computing module and camera Attitude Calculation
Module communication connection the determining current relative position of computing module, the camera direction can calculate mould depending on that relative position
The camera posture that the camera direction and the camera Attitude Calculation module that block determines determine determines the recognition result field of view
Interior point of interest.
Spatial data query unit includes attribute query module and perimeter query module, for inquiring periphery and calculating side
Position.Wherein, the attribute query module is communicated to connect with the position and attitude computing unit, can be according to the position and attitude meter
Calculate module output space querying condition, search meet the space querying condition spatial object distribution, orientation calculate and into
The corresponding statistical analysis of row, and support the SQL query language of standard；The perimeter query module is calculated with the position and attitude
Unit communication connection, the periphery P OI querying condition that can be exported according to the position and attitude computing module, lookup meet the periphery
POI interest point information in the present viewing field of POI querying condition, and it is used for augmented reality information unit Overlapping display.
Augmented reality information unit include: coordinate transferring, Text extraction module, three-D grain processing module,
Three-dimensional graph process module and dummy object message output module；Wherein, the coordinate transferring, respectively with the depth
Habit image processing unit, position and attitude computing unit are connected with Spatial data query unit communication, can be according to the deep learning
Point of interest in the visual field that recognition result image, the position and attitude computing unit of image processing unit output determine, passes through seat
The recognition result image is converted to 3-D image by mark conversion；The Text extraction module, with the coordinate modulus of conversion
Block communication connection, the 3-D image after capable of converting to the coordinate transferring add text information；The three-D grain processing
Module is communicated to connect with the Text extraction module, after capable of adding text information to the Text extraction module
3-D image adds three-D grain information；The three-dimensional graph process module is communicated to connect with the three-D grain processing module,
3-D image after the three-D grain processing module can be added to three-D grain information is plotted as three corresponding to dummy object
Tie up figure；The dummy object message output module is communicated to connect with the three-dimensional graph process module, can be by the three-dimensional figure
Treated corresponds to the 3-D graphic of dummy object and export to the display of the AR intelligent terminal and be shown as shape processing module
Dummy object information is used for human-computer interaction.
Embodiment
It is stopped with running the AR intelligent glasses of the AR intelligent terminal target identification system of the invention based on deep learning
For the automobile of some user of region recognition, the image of vehicle target is obtained by AR intelligent glasses, by the image of corresponding object
It is sent into target identification system after interception, why first image is intercepted, it is contemplated that vapour can not only be clapped by taking pictures in reality
Vehicle, automobile only accounts for a region in scene, is surrounding enviroment content there are also most of region.So need first to entire image into
Row interception.Then object-recognition unit combination image identifies vehicle, takes the model data identified and acquisition later
To car owner provide vehicle picture compare, if the vehicle of car owner is consistent with the result identified, just by present frame with
Target tracking unit is sent into target area, and carries out continuing tracking to current vehicle.
The overall flow of target identification system includes: image procossing, obtain AR intelligent terminal information, incident space data,
Enhancement information shows several parts, and process flow is following (referring to fig. 2):
Wherein, deep learning image processing unit is the key that entire target identification system, specifically include that obtain image,
Run SSD deep learning model and output test result three zones.The Camera capturing scenes carried using AR intelligent glasses
Image, the image information got can be supplied to image output unit and SSD deep learning model to continue with.Work as scene
After image is transferred to SSD deep learning model as input information, SSD deep learning model is after a series of detections, having handled
2D image is drawn using Canvas to show the object information of detection.
During the registration of 3D model and display, by be pre-stored within AR intelligent glasses local data base, be based on
The textured rotatable 3D figure of OpenGLES exploitation is extracted as enhancement information, by the XY for establishing OpenGL ES
Transfer algorithm between coordinate system and the UV coordinate system of AR intelligent glasses display screen is registered to according to detection block coordinate
In OpenGL ES coordinate system, to draw out, the bandwagon effect of augmented reality is realized.Image processing process can be found in Fig. 3.
The target identification system of the present embodiment at least has the effect that
(1) present invention will filter out the candidate regions of vehicle target by carrying out outline identification to the video frame inputted in real time
Domain be input in convolutional neural networks carry out vehicle target identification classification, improve vehicle target identification accuracy and in real time
Property.
(2) present invention is extracted multilane vehicle by the difference in lane the trace information of vehicle respectively, can be effective
Ground improves the trajectory extraction speed of vehicle, while also having carried out effective classification processing to track of vehicle.
Above-mentioned target identification system may be arranged at AR intelligent terminal local, and advantage is the dependence reduced to network, can be improved
The flexibility of application, but power deficiency and the slow problem of arithmetic speed are calculated there is also AR intelligent terminal is limited to.
Further, the server by the deployment of deep learning image processing unit beyond the clouds forms cloud-AR intelligence eventually
The mode of the client cooperation at end, the mainly service by deep learning recognition unit SSD deep learning model running beyond the clouds
Device can solve and calculate power and arithmetic speed inadequate problem when data volume is more than AR intelligent terminal operation bearing capacity.Together
When also Spatial data query unit is run on the server in cloud, AR intelligent terminal only need to by interface submit inquiry in
Hold, then receives query feedback result, while the operational capability of AR intelligent terminal can also be saved；Simultaneously by augmented reality
The three-dimensional graph process module of the three-dimensional registration algorithm of information unit operation also moves to the server in cloud, and AR intelligent terminal is logical
Interface is crossed to obtain three-dimensional registration operation result, then cooperates the experience of augmented reality information unit better on AR intelligent terminal again
AR effect, the object-recognition unit of this structure constitute as shown in Figure 4.
Target identification system of the invention is realized and is carried out in client by integrating SSD deep learning model in client
Target detection, identification and the function of tracking are got rid of based on identification figure mode, it is necessary to first provide identification figure to identification server
Dependence Problem；Identifying system based on deep learning model and conventional identification method compare, have higher recognition efficiency and
Accurate positioning；The knowledge for having stronger robustness than conventional identification method is brought by the identifying system based on deep learning model
Other mode；Identification of the identifying system based on deep learning model more suitable for indoor and outdoor size physical object；It is based on
The identifying system of deep learning model can integrate can also be engine-operated in server end by deep learning in client no matter
It is integrated in AR intelligent terminal or remote server, has stronger reality than conventional recognition mode on recognition efficiency
When property and accuracy.
The foregoing is only a preferred embodiment of the present invention, but scope of protection of the present invention is not limited thereto,
Within the technical scope of the present disclosure, any changes or substitutions that can be easily thought of by anyone skilled in the art,
It should be covered by the protection scope of the present invention.Therefore, protection scope of the present invention should be with the protection model of claims
Subject to enclosing.
Claims (10)
1. a kind of AR intelligent terminal target identification system based on deep learning characterized by comprising
Deep learning image processing unit, position and attitude computing unit, Spatial data query unit and augmented reality information unit；
Wherein,
The deep learning image processing unit can obtain the shooting image of AR intelligent terminal camera, and the shooting image is turned
It is changed to identification image, after being identified with image with deep learning mode by deep learning model to the identification, output identification
Result images；
The position and attitude computing unit is communicated to connect with the deep learning image processing unit, can be according to the depth
The recognition result image of image processing unit output is practised, the GPS information for obtaining the AR intelligent terminal determines the recognition result
The current relative position of geography target and the AR intelligent terminal in field of view, and obtain the sensing of the AR intelligent terminal
Device information determines camera direction and posture, and determines the identification according to determining current relative position, camera direction and posture
Point of interest in the result images visual field；
The Spatial data query unit is communicated to connect with the position and attitude computing unit, can be according to the position and attitude meter
The point of interest in the recognition result field of view that unit determines is calculated, and the spatial information for inquiring point of interest and attribute are believed
Breath is associated with to the recognition result image；
The augmented reality information unit, respectively with the deep learning image processing unit, position and attitude computing unit and sky
Between data query unit communication connect, by the deep learning image processing unit export recognition result image, with acquisition
To the GPS information of the AR intelligent terminal determine and be identified target and the AR intelligent terminal in the recognition result image
Current relative position, while by the AR intelligent terminal calculated camera internal parameter in real time, it is incorporated in world coordinate system
In external parameter set virtual camera, pass through the posture letter that the position and attitude computing unit obtains the AR intelligent terminal
Breath, and the posture of the virtual camera is demarcated, three-dimensional virtual scene is converted by the virtual camera, will be become
Change rear three-dimensional virtual scene image be added to actual scene image in carry out virtual reality fusion, shown by optical projection described in
Image after virtual reality fusion；And the spatial information and category for the three-dimensional virtual scene for inquiring the Spatial data query unit
Property information, be rendered on the point of interest of real world, carry out augmented reality show.
2. the AR intelligent terminal target identification system according to claim 1 based on deep learning, which is characterized in that described
Deep learning image processing unit includes:
Camera image obtains module, image conversion module, SSD deep learning model and output module；Wherein,
The camera image obtains module, can obtain the shooting image of AR intelligent terminal camera；
Described image conversion module obtains module communication connection with the camera image, the camera image can be obtained module
The shooting image of acquisition is converted to identification image；
The SSD deep learning model is communicated to connect with described image conversion module, can be by deep learning mode to the knowledge
It is not identified to obtain recognition result image with image；
The output module is connect with the SSD deep learning modeling communication, can export the SSD deep learning model identification
Obtained recognition result image.
3. the AR intelligent terminal target identification system according to claim 1 based on deep learning, which is characterized in that described
Position and attitude computing unit includes:
Relative position computing module, camera direction computing module, camera Attitude Calculation module and output processing module；Wherein,
The relative position computing module is communicated to connect with the deep learning image processing unit, can be according to the depth
The recognition result image of image processing unit output is practised, the GPS information for obtaining the AR intelligent terminal determines recognition result image
The current relative position of geography target and AR intelligent terminal in the visual field；
The camera direction computing module is communicated to connect with the deep learning image processing unit, can be according to the depth
The recognition result image of image processing unit output is practised, the magnetometric sensor information for obtaining the AR intelligent terminal determines camera side
To；
The camera Attitude Calculation module is communicated to connect with the deep learning image processing unit, can be according to the depth
The recognition result image of image processing unit output is practised, the acceleration transducer information for obtaining the AR intelligent terminal determines camera
Posture；
The output processing module, respectively with the relative position computing module, camera direction computing module and camera posture meter
Module communication connection is calculated, the determining current relative position of computing module, the camera direction can be calculated depending on that relative position
The camera posture that the camera direction and the camera Attitude Calculation module that module determines determine determines the recognition result image view
Point of interest in open country.
4. the AR intelligent terminal target identification system according to claim 1 based on deep learning, which is characterized in that described
Spatial data query unit includes:
Attribute query module and perimeter query module；Wherein,
The attribute query module is communicated to connect with the position and attitude computing unit, can calculate mould according to the position and attitude
The space querying condition of block output searches the spatial object distribution for meeting the space querying condition, orientation calculates and carry out phase
The statistical analysis answered；
The perimeter query module is communicated to connect with the position and attitude computing unit, can calculate mould according to the position and attitude
The periphery P OI querying condition of block output, searches the POI interest point information met in the present viewing field of the condition, existing for enhancing
Real information unit Overlapping display.
5. the AR intelligent terminal target identification system according to claim 1 based on deep learning, which is characterized in that described
Augmented reality information unit includes:
Coordinate transferring, Text extraction module, three-D grain processing module, three-dimensional graph process module and dummy object
Message output module；Wherein,
The coordinate transferring, respectively with the deep learning image processing unit, position and attitude computing unit and space number
It is investigated that ask unit communication connection, can according to the deep learning image processing unit export recognition result image, the position
Point of interest in the visual field that attitude calculation unit determines is converted by coordinate the recognition result image being converted to 3-D image；
The Text extraction module is communicated to connect with the coordinate transferring, can be converted to the coordinate transferring
3-D image afterwards adds text information；
The three-D grain processing module is communicated to connect with the Text extraction module, can be to the Text extraction
Module adds the 3-D image addition three-D grain information after text information；
The three-dimensional graph process module communicates to connect with the three-D grain processing module, can handle the three-D grain
3-D image after module addition three-D grain information is plotted as the 3-D graphic corresponding to dummy object；
The dummy object message output module is communicated to connect with the three-dimensional graph process module, can be by the 3-D graphic
Treated corresponds to the 3-D graphic of dummy object and export to the display of the AR intelligent terminal and be shown as empty for processing module
Quasi- object information is used for human-computer interaction.
6. the AR intelligent terminal target identification system according to any one of claims 1 to 5 based on deep learning, feature
It is, the deep learning image processing unit, position and attitude computing unit, Spatial data query unit and augmented reality information
Unit is each provided in AR intelligent terminal；
Alternatively,
The position and attitude computing unit, Spatial data query unit and augmented reality information unit are each provided at AR intelligent terminal
Interior, the SSD deep learning model of the deep learning image processing unit is located at the clothes with AR intelligent terminal communication connection
It is engaged in device, other modules of the deep learning image processing unit are located in the AR intelligent terminal.
7. a kind of AR intelligent terminal target identification method based on deep learning, which is characterized in that use claim 1 to 6 times
AR intelligent terminal target identification system described in one based on deep learning, comprising the following steps:
Step 1, deep learning image procossing: the shooting image of AR intelligent terminal camera is obtained, the shooting image is converted to
Identification image after being identified with image with deep learning mode by deep learning model to the identification, exports recognition result
Image；
Step 2, AR intelligent terminal information is obtained: the recognition result image exported according to the deep learning image processing unit,
The GPS information for obtaining the AR intelligent terminal determines geography target and the AR intelligent terminal in the recognition result field of view
Current relative position, and obtain the sensor information of the AR intelligent terminal and determine camera direction and posture, and according to true
Fixed current relative position, camera direction and posture determines the point of interest in the recognition result field of view；
Step 3, incident space data: in the recognition result field of view determined according to the position and attitude computing unit
Point of interest, and the spatial information for inquiring point of interest is associated with attribute information to the recognition result image；
Step 4, information enhancement exports: the recognition result image exported by the deep learning image processing unit, with acquisition
To the GPS information of the AR intelligent terminal determine and be identified target and the AR intelligent terminal in the recognition result image
Current relative position, while by the AR intelligent terminal calculated camera internal parameter in real time, it is incorporated in world coordinate system
In external parameter set virtual camera, pass through the posture letter that the position and attitude computing unit obtains the AR intelligent terminal
Breath, and the posture of the virtual camera is demarcated, three-dimensional virtual scene is converted by the virtual camera, will be become
Change rear three-dimensional virtual scene image be added to actual scene image in carry out virtual reality fusion, shown by optical projection described in
Image after virtual reality fusion；And the spatial information and category for the three-dimensional virtual scene for inquiring the Spatial data query unit
Property information, be rendered on the point of interest of real world, carry out augmented reality show.
8. the AR intelligent terminal target identification method according to claim 7 based on deep learning, which is characterized in that described
The deep learning image procossing of method and step 1 includes:
Step 11, camera image obtaining step obtains the shooting image of AR intelligent terminal camera；
Step 12, the camera image is obtained the shooting image that module obtains and is converted to identification image by image conversion step；
Step 13, SSD deep learning step identifies the identification with image by deep learning mode to obtain identification knot
Fruit image；
Step 14, step is exported, the recognition result image that the SSD deep learning model identifies is exported.
9. the AR intelligent terminal target identification method according to claim 7 based on deep learning, which is characterized in that described
The acquisition AR intelligent terminal information of method and step 2 includes:
Step 21, relative position calculates step, according to the recognition result image that the deep learning image processing unit exports, obtains
The GPS information of the AR intelligent terminal is taken to determine geography target and the AR intelligent terminal in the recognition result field of view
Current relative position；
Step 22, camera direction calculates step, according to the recognition result image that the deep learning image processing step exports, obtains
The magnetometric sensor information of the AR intelligent terminal is taken to determine camera direction；
Step 23, camera Attitude Calculation step is obtained according to the recognition result image that the deep learning image processing step exports
The acceleration transducer information of the AR intelligent terminal is taken to determine camera posture；
Step 24, processing step is exported, calculates current relative position, the camera side that step determines depending on that relative position
The recognition result is determined to the camera posture that the determining camera direction of step and the camera Attitude Calculation step determine is calculated
Point of interest in field of view.
10. the AR intelligent terminal target identification method according to claim 7 based on deep learning, which is characterized in that institute
The information enhancement for stating method and step 4, which exports, includes:
Step 41, three-dimensional registration step determines three-dimensional virtual scene using inside and outside portion's parameter of the AR intelligent terminal
Image and acquisition environment between corresponding parameter, the image for calculating the three-dimensional virtual scene will be added in true environment
Three-dimensional coordinate information；
Step 42, enhancement information shows step, and three-dimensional coordinate information is calculated by the three-dimensional registration step, determine described in
The image of three-dimensional virtual scene is added to the mapping position in world coordinate system, and the image of the three-dimensional virtual scene is correct
Output is shown after transforming to projection plane.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910739784.2A CN110443898A (en) | 2019-08-12 | 2019-08-12 | A kind of AR intelligent terminal target identification system and method based on deep learning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910739784.2A CN110443898A (en) | 2019-08-12 | 2019-08-12 | A kind of AR intelligent terminal target identification system and method based on deep learning |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110443898A true CN110443898A (en) | 2019-11-12 |
Family
ID=68434650
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910739784.2A Pending CN110443898A (en) | 2019-08-12 | 2019-08-12 | A kind of AR intelligent terminal target identification system and method based on deep learning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110443898A (en) |
Cited By (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111429585A (en) * | 2020-03-30 | 2020-07-17 | 北京字节跳动网络技术有限公司 | Image generation method and device, electronic equipment and computer readable storage medium |
| CN111696216A (en) * | 2020-06-16 | 2020-09-22 | 浙江大华技术股份有限公司 | Three-dimensional augmented reality panorama fusion method and system |
| CN112037314A (en) * | 2020-08-31 | 2020-12-04 | 北京市商汤科技开发有限公司 | Image display method, image display device, display equipment and computer readable storage medium |
| CN112330753A (en) * | 2020-11-16 | 2021-02-05 | 北京理工大学 | Target detection method of augmented reality system |
| CN112330816A (en) * | 2020-10-19 | 2021-02-05 | 杭州易现先进科技有限公司 | AR identification processing method and device and electronic device |
| CN112580631A (en) * | 2020-12-24 | 2021-03-30 | 北京百度网讯科技有限公司 | Indoor positioning method and device, electronic equipment and storage medium |
| CN112633145A (en) * | 2020-12-21 | 2021-04-09 | 武汉虚世科技有限公司 | WebAR processing method based on 3D detection and identification and moving target tracking |
| US11043038B1 (en) | 2020-03-16 | 2021-06-22 | Hong Kong Applied Science and Technology Research Institute Company Limited | Apparatus and method of three-dimensional interaction for augmented reality remote assistance |
| CN113366489A (en) * | 2018-11-07 | 2021-09-07 | 脸谱公司 | Detecting augmented reality targets |
| CN113793389A (en) * | 2021-08-24 | 2021-12-14 | 国网甘肃省电力公司 | Virtual-real fusion calibration method and device for augmented reality system |
| CN115064023A (en) * | 2022-05-06 | 2022-09-16 | 中国人民解放军陆军防化学院 | Portable terminal teaching training system based on AR glasses |
| CN115497087A (en) * | 2022-11-18 | 2022-12-20 | 广州煌牌自动设备有限公司 | Tableware posture recognition system and method |
| CN116524160A (en) * | 2023-07-04 | 2023-08-01 | 应急管理部天津消防研究所 | Product consistency auxiliary verification system and method based on AR identification |
| CN117078975A (en) * | 2023-10-10 | 2023-11-17 | 四川易利数字城市科技有限公司 | AR space-time scene pattern matching method based on evolutionary algorithm |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102831401A (en) * | 2012-08-03 | 2012-12-19 | 樊晓东 | Method and system for tracking, three-dimensionally superposing and interacting target object without special mark |
| CN103489214A (en) * | 2013-09-10 | 2014-01-01 | 北京邮电大学 | Virtual reality occlusion handling method, based on virtual model pretreatment, in augmented reality system |
| CN110109535A (en) * | 2019-03-18 | 2019-08-09 | 国网浙江省电力有限公司信息通信分公司 | Augmented reality generation method and device |
-
2019
- 2019-08-12 CN CN201910739784.2A patent/CN110443898A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102831401A (en) * | 2012-08-03 | 2012-12-19 | 樊晓东 | Method and system for tracking, three-dimensionally superposing and interacting target object without special mark |
| CN103489214A (en) * | 2013-09-10 | 2014-01-01 | 北京邮电大学 | Virtual reality occlusion handling method, based on virtual model pretreatment, in augmented reality system |
| CN110109535A (en) * | 2019-03-18 | 2019-08-09 | 国网浙江省电力有限公司信息通信分公司 | Augmented reality generation method and device |
Non-Patent Citations (2)
| Title |
|---|
| 乔延军等: "面向户外增强现实的地理实体目标检测", 《地理信息世界》 * |
| 张少博: "基于SSD物体追踪算法的增强现实系统设计与实现", 《中国优秀硕士学位论文全文数据库 信息科技辑》 * |
Cited By (23)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113366489A (en) * | 2018-11-07 | 2021-09-07 | 脸谱公司 | Detecting augmented reality targets |
| WO2021184234A1 (en) * | 2020-03-16 | 2021-09-23 | Hong Kong Applied Science and Technology Research Institute Company Limited | Apparatus and method of three-dimensional interaction for augmented reality remote assistance |
| US11043038B1 (en) | 2020-03-16 | 2021-06-22 | Hong Kong Applied Science and Technology Research Institute Company Limited | Apparatus and method of three-dimensional interaction for augmented reality remote assistance |
| CN111429585A (en) * | 2020-03-30 | 2020-07-17 | 北京字节跳动网络技术有限公司 | Image generation method and device, electronic equipment and computer readable storage medium |
| CN111696216A (en) * | 2020-06-16 | 2020-09-22 | 浙江大华技术股份有限公司 | Three-dimensional augmented reality panorama fusion method and system |
| CN111696216B (en) * | 2020-06-16 | 2023-10-03 | 浙江大华技术股份有限公司 | Three-dimensional augmented reality panorama fusion method and system |
| CN112037314A (en) * | 2020-08-31 | 2020-12-04 | 北京市商汤科技开发有限公司 | Image display method, image display device, display equipment and computer readable storage medium |
| CN112330816A (en) * | 2020-10-19 | 2021-02-05 | 杭州易现先进科技有限公司 | AR identification processing method and device and electronic device |
| CN112330816B (en) * | 2020-10-19 | 2024-03-26 | 杭州易现先进科技有限公司 | AR identification processing method and device and electronic device |
| CN112330753A (en) * | 2020-11-16 | 2021-02-05 | 北京理工大学 | Target detection method of augmented reality system |
| CN112330753B (en) * | 2020-11-16 | 2023-05-09 | 北京理工大学 | Target detection method of augmented reality system |
| CN112633145A (en) * | 2020-12-21 | 2021-04-09 | 武汉虚世科技有限公司 | WebAR processing method based on 3D detection and identification and moving target tracking |
| CN112633145B (en) * | 2020-12-21 | 2024-04-26 | 武汉虚世科技有限公司 | WebAR processing method based on 3D detection recognition and moving target tracking |
| CN112580631A (en) * | 2020-12-24 | 2021-03-30 | 北京百度网讯科技有限公司 | Indoor positioning method and device, electronic equipment and storage medium |
| CN113793389A (en) * | 2021-08-24 | 2021-12-14 | 国网甘肃省电力公司 | Virtual-real fusion calibration method and device for augmented reality system |
| CN113793389B (en) * | 2021-08-24 | 2024-01-26 | 国网甘肃省电力公司 | Virtual-real fusion calibration method and device for augmented reality system |
| CN115064023A (en) * | 2022-05-06 | 2022-09-16 | 中国人民解放军陆军防化学院 | Portable terminal teaching training system based on AR glasses |
| CN115497087A (en) * | 2022-11-18 | 2022-12-20 | 广州煌牌自动设备有限公司 | Tableware posture recognition system and method |
| CN115497087B (en) * | 2022-11-18 | 2024-04-19 | 广州煌牌自动设备有限公司 | Tableware gesture recognition system and method |
| CN116524160B (en) * | 2023-07-04 | 2023-09-01 | 应急管理部天津消防研究所 | Product consistency auxiliary verification system and method based on AR identification |
| CN116524160A (en) * | 2023-07-04 | 2023-08-01 | 应急管理部天津消防研究所 | Product consistency auxiliary verification system and method based on AR identification |
| CN117078975A (en) * | 2023-10-10 | 2023-11-17 | 四川易利数字城市科技有限公司 | AR space-time scene pattern matching method based on evolutionary algorithm |
| CN117078975B (en) * | 2023-10-10 | 2024-01-02 | 四川易利数字城市科技有限公司 | AR space-time scene pattern matching method based on evolutionary algorithm |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110443898A (en) | A kind of AR intelligent terminal target identification system and method based on deep learning | |
| CN112894832B (en) | Three-dimensional modeling method, three-dimensional modeling device, electronic equipment and storage medium | |
| US10580162B2 (en) | Method for determining the pose of a camera and for recognizing an object of a real environment | |
| CN107742311B (en) | Visual positioning method and device | |
| CN108564647B (en) | A method of establishing virtual three-dimensional map | |
| CN105930819B (en) | Real-time city traffic lamp identifying system based on monocular vision and GPS integrated navigation system | |
| CN103093191B (en) | A kind of three dimensional point cloud is in conjunction with the object identification method of digital image data | |
| CN103530881B (en) | Be applicable to the Outdoor Augmented Reality no marks point Tracing Registration method of mobile terminal | |
| CN110017841A (en) | Vision positioning method and its air navigation aid | |
| CN103839277B (en) | A kind of mobile augmented reality register method of outdoor largescale natural scene | |
| CN108596974A (en) | Dynamic scene robot localization builds drawing system and method | |
| CN109631855A (en) | High-precision vehicle positioning method based on ORB-SLAM | |
| CN103514446B (en) | A kind of outdoor scene recognition methods merging sensor information | |
| US10043097B2 (en) | Image abstraction system | |
| CN106447585A (en) | Urban area and indoor high-precision visual positioning system and method | |
| KR102200299B1 (en) | A system implementing management solution of road facility based on 3D-VR multi-sensor system and a method thereof | |
| CN110073362A (en) | System and method for lane markings detection | |
| CN109308722A (en) | A kind of spatial pose measuring system and method based on active vision | |
| CN103632626A (en) | Intelligent tour guide realizing method and intelligent tour guide device based on mobile network and mobile client | |
| CN111028358B (en) | Indoor environment augmented reality display method and device and terminal equipment | |
| CN110675453B (en) | Self-positioning method for moving target in known scene | |
| CN106155299B (en) | A kind of pair of smart machine carries out the method and device of gesture control | |
| CN103716399B (en) | Remote interaction picking fruit based on wireless network works in coordination with asynchronous control system and method | |
| CN111060924A (en) | SLAM and target tracking method | |
| CN110119190A (en) | Localization method, device, recognition and tracking system and computer-readable medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20191112 |