CN103049729A - Method, system and terminal for augmenting reality based on two-dimension code - Google Patents
Method, system and terminal for augmenting reality based on two-dimension code Download PDFInfo
- Publication number
- CN103049729A CN103049729A CN2012105871531A CN201210587153A CN103049729A CN 103049729 A CN103049729 A CN 103049729A CN 2012105871531 A CN2012105871531 A CN 2012105871531A CN 201210587153 A CN201210587153 A CN 201210587153A CN 103049729 A CN103049729 A CN 103049729A
- Authority
- CN
- China
- Prior art keywords
- dimension code
- image
- code
- resource
- scene image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/45—Management operations performed by the client for facilitating the reception of or the interaction with the content or administrating data related to the end-user or to the client device itself, e.g. learning user preferences for recommending movies, resolving scheduling conflicts
- H04N21/462—Content or additional data management, e.g. creating a master electronic program guide from data received from the Internet and a Head-end, controlling the complexity of a video stream by scaling the resolution or bit-rate based on the client capabilities
- H04N21/4622—Retrieving content or additional data from different sources, e.g. from a broadcast channel and the Internet
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/41—Structure of client; Structure of client peripherals
- H04N21/414—Specialised client platforms, e.g. receiver in car or embedded in a mobile appliance
- H04N21/41407—Specialised client platforms, e.g. receiver in car or embedded in a mobile appliance embedded in a portable device, e.g. video client on a mobile phone, PDA, laptop
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/41—Structure of client; Structure of client peripherals
- H04N21/422—Input-only peripherals, i.e. input devices connected to specially adapted client devices, e.g. global positioning system [GPS]
- H04N21/4223—Cameras
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/435—Processing of additional data, e.g. decrypting of additional data, reconstructing software from modules extracted from the transport stream
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs
- H04N21/44008—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/47—End-user applications
- H04N21/472—End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
- H04N21/81—Monomedia components thereof
- H04N21/8146—Monomedia components thereof involving graphical data, e.g. 3D object, 2D graphics
- H04N21/8153—Monomedia components thereof involving graphical data, e.g. 3D object, 2D graphics comprising still images, e.g. texture, background image
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Databases & Information Systems (AREA)
- Computer Graphics (AREA)
- General Engineering & Computer Science (AREA)
- Human Computer Interaction (AREA)
- Processing Or Creating Images (AREA)
- Information Transfer Between Computers (AREA)
- User Interface Of Digital Computer (AREA)
Abstract
The invention discloses a method for augmenting reality based on a two-dimension code. The method comprises the following steps of taking the two-dimension code as an identifying and positioning identifier in a reality-augmenting system; decoding and re-encoding the two-dimension code in a real scene image to directly generate a two-dimension code elevation view which is the same as the two-dimension code in the real scene image; and tracing and matching the generated two-dimension code elevation view and a two-dimension code image in the scene image, and calculating a homography matrix. Correspondingly, the invention also discloses a system and mobile terminal for augmenting reality based on the two-dimension code. The limitation that tracing match can be performed only by pre-storing a corresponding sample image in a database in the traditional reality augmenting application is broken through, the steps of searching and matching to a template of a remote server when a traditional identifier is used are also avoided, system response delay caused by network transmission can be reduced, and the network communication flow rate of a user is saved.
Description
Technical field
The present invention relates to the mobile augmented reality field, relate in particular to a kind of augmented reality method, system and portable terminal based on two-dimension code.
Background technology
Two-dimension code claims again two-dimensional bar code, that the chequered with black and white figure of (on the two-dimensional directional) distribution comes the record data symbolic information on the plane according to certain rules with certain specific geometric figure, its ingenious utilization on coding consists of " 0 ", " 1 " bit stream concept of computer-internal logical foundations, represents the literal numerical information with several geometrical bodies corresponding with scale-of-two.
In recent years, two-dimension code is widely applied, the mobile application that is specifically designed on the market the scanning recognition two-dimension code emerged in an endless stream, but behind these application scanning two-dimension codes, all directly present two-dimension code and resolve rear resulting Word message or the resource link network address such as video, webpage, use not and dazzle.
When the two-dimension code use is more and more general, augmented reality (AR, Augumented Reality) beginning slowly enters public view, its core is that virtual information is added in the scene that true environment presents in real time, utilize virtual information that real scene is replenished, strengthens, allow virtual information in real world, show synchronously.In the existing augmented reality, realize virtual information is added on the real scene, must calculate the relative position relation between video camera and real scene, namely carry out registration by real scene image and sample image, obtain homography matrix.Therefore for existing augmented reality, if portable terminal or augmented reality server end are not stored the sample image of certain real scene or the characteristic point information of this sample image, then can't realize the fusion of virtual information and this real scene.
Summary of the invention
The purpose of this invention is to provide a kind of augmented reality method based on two-dimension code, system and portable terminal, do not having in the situation of sample image, generate the two-dimension code front elevation consistent with the two-dimension code in the scene image by two-dimension code and recompile in the decoding scene image, and the image in 2 D code in the scene image that two-dimension code front elevation and photographing module are caught carries out feature detection and coupling, calculate homography matrix, be implemented in the certain deviation position of real scene two-dimension code position or two-dimension code, present the relevant video of two-dimension code, image, text, the multimedia messages of 3D model.
In order to realize the foregoing invention purpose, the invention provides a kind of augmented reality method based on two-dimension code, comprising:
Photographing module is caught the real scene image that contains two-dimension code;
Scan the two-dimension code in the scene image, and two-dimension code is decoded, obtain the coded message of two-dimension code, described coded message comprises: code system, version and resource information;
Coded message to the two-dimension code that obtains is encoded again, generate with scene image in two-dimension code code system and the identical two-dimension code front elevation of version; Resolve simultaneously described resource information, to obtain virtual information corresponding to two-dimension code;
Image in 2 D code in the scene image that two-dimension code front elevation and photographing module are caught carries out feature detection, and the feature that obtains respectively the two is described; And describe according to the feature of the two and to carry out image registration, calculate the attitude of video camera, obtain homography matrix;
According to homography matrix, play up and the described virtual information corresponding with two-dimension code of output display the certain deviation position of two-dimension code position or two-dimension code in real scene.
Preferably, when resolve that described resource information obtains be content of text the time, content of text is played up as texture; Be resource URI when what resolve that described resource information obtains, then access this URI and obtain virtual information, and load with predetermined manner according to the virtual information type.
Preferably, described virtual information type comprises: one or more in video, image, text, the 3D model.
Wherein, the two-dimension code in the described real scene image is conventional two-dimension code or customization of two-dimension code; Resource information in the customization of two-dimension code comprises that resource identifier, resource type, resource load interface dimensions, play up in the drift rate of position one or more.
Preferably, the image in 2 D code in the described scene image that two-dimension code front elevation and photographing module are caught carries out feature detection, and the feature that obtains respectively the two is described, and is one of following dual mode:
Mode one: the image in 2 D code in the scene image that two-dimension code front elevation and photographing module are caught carries out the full figure feature detection, and the feature that obtains respectively the two is described;
Mode two: feature detection is carried out in the stabilized zone of the image in 2 D code in the scene image of only two-dimension code front elevation and photographing module being caught, and the feature that obtains respectively the two is described.
Accordingly, the invention allows for a kind of augmented reality system based on two-dimension code, comprising:
Photographing module is used for catching the real scene image that contains two-dimension code;
The two-dimension code decoder module is used for the two-dimension code of scanning scene image, and two-dimension code is decoded, and obtains the coded message of two-dimension code, and described coded message comprises: code system, version and resource information;
The two-dimension code coding module is encoded to the coded message that described two-dimension code decoder module parses again, generate with scene image in two-dimension code code system and the identical two-dimension code front elevation of version;
The resource acquisition module is for the resource information of resolving described coded message, to obtain virtual information corresponding to two-dimension code;
The image characteristics extraction module, the image in 2 D code that is used for scene image that two-dimension code front elevation and photographing module are caught carries out feature detection, and the feature that obtains respectively the two is described;
Image is followed the tracks of registration module, is used for carrying out image registration according to the image in 2 D code feature description of two-dimension code front elevation and scene image, calculates the attitude of video camera, obtains homography matrix;
Play up display module, be used for according to homography matrix, play up and the described virtual information corresponding with two-dimension code of output display the certain deviation position of two-dimension code position or two-dimension code in real scene.
Preferably, described image characteristics extraction module, the image in 2 D code in the scene image that two-dimension code front elevation and photographing module are caught carries out feature detection, is one of following dual mode:
Mode one: the image in 2 D code in the scene image that two-dimension code front elevation and photographing module are caught carries out the full figure feature detection, and the feature that obtains respectively the two is described;
Mode two: feature detection is carried out in the stabilized zone of the image in 2 D code in the scene image of only two-dimension code front elevation and photographing module being caught, and the feature that obtains respectively the two is described.
Preferably, the two-dimension code in the described real scene image is conventional two-dimension code or customization of two-dimension code; Resource information in the customization of two-dimension code comprises that resource identifier, resource type, resource load interface dimensions, play up in the drift rate of position one or more.
Preferably, when the described resource information of described resource acquisition module parses obtain be content of text the time, content of text is played up as texture; And be resource URI when what resolve that described resource information obtains, then to access this URI and obtain virtual information, and load with predetermined manner according to the virtual information type, described virtual information type comprises: one or more in video, image, text, the 3D model.
Accordingly, the invention allows for a kind of portable terminal, described portable terminal comprises the above-mentioned augmented reality system based on two-dimension code.
Compared with prior art, the present invention has following beneficial effect:
1, invention is directly according to the two-dimension code in the scene image, regenerate the two-dimension code front elevation consistent with two-dimension code in the scene image, carry out tracking and matching with the two-dimension code front elevation that generates and the image in 2 D code in the scene image, calculate the attitude of video camera, obtain homography matrix, do not need storage two-dimension code sample image in the database, all applicable to arbitrary two-dimension code, broken during traditional augmented reality uses, must be in database pre-stored corresponding sample image just can carry out the limitation of tracking and matching.
2, therefore the present invention has avoided inquiry and coupling step from remote server owing to do not need sample image, can reduce the system response delay that causes because of the Internet Transmission problem, saves user's network traffic.
3, the degree of depth of the present invention is excavated two-dimension code as the application potential of information entry, makes the relevant information of two-dimension code, resource present to the user with more lively form.
Description of drawings
In order to be illustrated more clearly in the embodiment of the invention or technical scheme of the prior art, the below will do to introduce simply to the accompanying drawing of required use in embodiment or the description of the Prior Art, apparently, accompanying drawing in the following describes only is some embodiments of the present invention, for those of ordinary skills, under the prerequisite of not paying creative work, can also obtain according to these accompanying drawings other accompanying drawing:
Fig. 1 is that the embodiment of the invention is based on the augmented reality method flow synoptic diagram of two-dimension code;
Fig. 2 is for carrying out process and the effect synoptic diagram of virtual information stack according to Fig. 1 method;
Fig. 3 is the image in 2 D code synoptic diagram of several code systems commonly used;
But Fig. 4 uses synoptic diagram for the two-dimension code that occlusion area is blocked by small icon.
Embodiment
Below in conjunction with the accompanying drawing in the embodiment of the invention, the technical scheme in the embodiment of the invention is clearly and completely described, obviously, described embodiment only is the present invention's part embodiment, rather than whole embodiment.Based on the embodiment among the present invention, those of ordinary skills belong to the scope of protection of the invention not making the every other embodiment that obtains under the creative work prerequisite.
Those skilled in the art should know: in the augmented reality process, virtual information is superimposed upon on the target object accurately, need to calculates the attitude of video camera, determine to be tied to from the coordinate of target object the homography matrix of image coordinate system.
The present inventor finds that existing augmented reality generally all is to carry out registration by real scene image and sample image, obtains homography matrix.This mode requires carrying out the image of augmented reality, must store the sample image that it is used for registration at portable terminal or server end, if portable terminal or augmented reality server end are not stored the sample image of certain real scene or the characteristic point information of this sample image, then can't realize the fusion of virtual information and this real scene, the popularization of augmented reality is limited by sample image.In addition sample image be stored in portable terminal can occupied terminal storage space, and storage that can not the satisfying magnanimity marker; And if sample image is stored in remote server, the retrieval of sample image and the network traffic of downloading the response of meeting delay system and waste user.
The present inventor finds, two-dimension code has ease for use and popularization, and two-dimension code is as information entry, can be by any Information generation, it can carry the URI of any one section text or any cost, the most important is: because two-dimension code decoding can obtain the information such as code system, version and error-correction level of this two-dimension code, can be again after the two-dimension code decoding generate and the identical two-dimension code front elevation of pattern in 2 D code before according to identical code system and version recompile, or generate and the identical two-dimension code front elevation of two-dimension code stabilized zone pattern before.Therefore propose with two-dimension code as the identification in the augmented reality system and positioning mark, decoding is encoded again to the two-dimension code in the real scene image, generate the two-dimension code front elevation identical with image in 2 D code in the real scene image, it is carried out the feature point tracking coupling as the two-dimension code in sample image and the scene, find the unique point that is complementary in the two, carry out the calculating of homography matrix.
But described two-dimension code stabilized zone is for the occlusion area of two-dimension code, because two-dimension code has certain error correcting capability, it can be correctly decoded in the situation that certain area is blocked (but such as Fig. 4 for to have placed the two-dimension code of a small icon at the occlusion area of two-dimension code, this two-dimension code can be correctly decoded), therefore but we claim to be blocked and do not affect the zone that two-dimension code is correctly decoded and be occlusion area, but the zone except occlusion area is the stabilized zone.
Introduce in detail the present invention below in conjunction with accompanying drawing and introduce scheme.
Referring to Fig. 1, Fig. 2, be the augmented reality method flow synoptic diagram of the embodiment of the invention based on two-dimension code, comprise following S101 ~ S105 step:
S101: photographing module is caught the real scene image that contains two-dimension code, described two-dimension code can be conventional two-dimension code, also can be the customization of two-dimension code, conventional two-dimension code refers to comprise in the resource information network two-dimension code of a text field or URI link, and the customization of two-dimension code refers to that its resource information comprises that resource identifier, resource type, resource load interface dimensions, play up in some other configuration information such as position drift rate one or more;
S102: scan the two-dimension code in the scene image, and two-dimension code is decoded, obtain the coded message of two-dimension code, described coded message comprises: code system, version and resource information etc., and described resource information refers to scan the relevant information that two-dimension code obtains;
S103: the coded message to the two-dimension code that obtains is encoded again, generate with scene image in two-dimension code code system and the identical two-dimension code front elevation of version; Resolve simultaneously described resource information, to obtain virtual information corresponding to two-dimension code, the resource information that two-dimension code is corresponding may be text message, also may be resource URI, when being resource URI, according to UIR address access remote server, obtain virtual information content corresponding to URI;
In this step, coded message to the two-dimension code that obtains is encoded again, generate with scene image in two-dimension code code system and the identical two-dimension code front elevation of version, this two-dimension code front elevation may for scene image in the identical two-dimension code front elevation of pattern in 2 D code (but the occlusion area of scene two-dimension code without any blocking time), also may for scene image in pattern identical two-dimension code front elevation in two-dimension code stabilized zone (but when the occlusion area of scene two-dimension code is partly or entirely blocked, because two-dimension code has error correcting capability, can be correctly decoded having small icon and block in the situation of certain area, yet decoded information is encoded again, can generate the two-dimension code front elevation that comprises data consistent with former two-dimension code, but can not have the small icon of blocking in the former two-dimension code in the front elevation that generates).
S104: the image in 2 D code in the scene image that two-dimension code front elevation and photographing module are caught carries out feature detection, and the feature that obtains respectively the two is described; And describe according to the feature of the two and to carry out image registration (be Feature Points Matching, find the unique point that is complementary in the two), calculate the attitude of video camera, obtain homography matrix;
In this step, image in 2 D code in the scene image that two-dimension code front elevation and photographing module are caught carries out feature detection, can be image in 2 D code in the scene image that two-dimension code front elevation and photographing module are caught carry out the full figure feature detection (but this kind mode the occlusion area of scene two-dimension code during without any blocking effect better, but it may occur certain matching error to the shield portions of scene two-dimension code by the zone or when all blocking), also can be the stabilized zone of the image in 2 D code in the scene image of only two-dimension code front elevation and photographing module being caught carry out feature detection (but this kind mode to the no requirement (NR) that whether is blocked of the occlusion area of scene two-dimension code, but equal matched well images).
Wherein, the method for feature detection can be FAST, Harris, Shi-Thomas etc., and feature is described can use SIFT, SURF, ORB, BRIEF, FREAK etc., and these technology all belong to prior art, are not repeated herein.Two-dimension code in the real scene is planar object, it has determined a world coordinate system, the two-dimension code front elevation that generates through the coding of decoding again belongs to image coordinate system, match 4 groups of character pair points as example on the image in 2 D code in the scene image of catching take two-dimension code front elevation and photographing module, coordinate and the image coordinate coordinate fastened of these four unique points on world coordinate system can be set up following corresponding relation:
Wherein, the image coordinate of (u, v) representation feature point, (x
m, y
m, z
m) coordinate of representation feature point on world coordinate system, being known parameters, H represents homography matrix to be calculated, and the unique point that matches is more, and result of calculation is more accurate.
S105: according to homography matrix, play up and the described virtual information corresponding with two-dimension code of output display the certain deviation position of two-dimension code position or two-dimension code in real scene.
In this step, if two-dimension code is conventional two-dimension code, the two-dimension code position directly is added to virtual information; And if two-dimension code is the customization of two-dimension code, set resource in its resource information and loaded interface dimensions, play up position drift rate etc., then according to the size of setting and drift rate folded being presented at of virtual information set the deviation post place apart from the target location.In addition, in this step, when resolve that described resource information obtains be content of text the time, content of text is played up as texture; Resource URI when what resolve that described resource information obtains, then access this URI and obtain virtual information, and load with predetermined manner according to the virtual information type, described virtual information type comprises: one or more in video, image, text, the 3D model, for example: when virtual information is text message, content of text is directly played up as texture; When virtual information is the 3D model, need first the 3D model to be resolved; When virtual information is video information, need first video to be decoded, each two field picture of video as texture, is mapped on the described 3D model frame by frame by sequence, carry out graph rendering.
Embodiment of the invention method can be take two-dimension codes such as PDF417, QR Code, Data Matrix, Grid Matrix and Aztec as application, and the two-dimension code synoptic diagram of several code systems commonly used can be referring to Fig. 3.
Two-dimension code can be conventional two-dimension code in the embodiment of the invention, also can be the customization of two-dimension code, so-called conventional two-dimension code refers to that two-dimension code derives from the internet, and the resource information that comprises in it is generally a character string, for example is the URI of one section text or two-dimension code related resource; So-called customization of two-dimension code refers to, the resource identification that forms according to certain consolidation form, namely the resource information in the two-dimension code comprises that resource identifier, resource type, resource load interface dimensions, play up in some other configuration information such as position drift rate one or more.For example, ID:xxx_URI:xxx_TYPE:xxx_WIDTH:xxx_HEIGHT:xxx_OFFSET:xxx, (this address may be at remote server to determine resource address by ID and URI, also may be at client terminal local), can set in advance the load mode (load mode of picture, text, audio frequency, video and 3D model is different) of related resource by resource type, other configuration informations have comprised that resource loads the size at interface and plays up the position with respect to skew of two-dimension code etc.
The invention allows for a kind of augmented reality system based on two-dimension code corresponding with said method, comprise: photographing module, two-dimension code decoder module, two-dimension code coding module, resource acquisition module, image characteristics extraction module, image are followed the tracks of registration module and are played up display module, wherein:
Described photographing module, be used for catching the real scene image that contains two-dimension code, two-dimension code in the described real scene image is conventional two-dimension code or customization of two-dimension code, and the resource information in the customization of two-dimension code comprises that resource identifier, resource type, resource load interface dimensions, play up in the drift rate of position one or more;
Described two-dimension code decoder module is used for the two-dimension code of scanning scene image, and two-dimension code is decoded, and obtains the coded message of two-dimension code, and described coded message comprises: code system, version and resource information;
Described two-dimension code coding module, for the coded message that parses according to described two-dimension code decoder module, encode with identical with it code system and version, the identical two-dimension code front elevation of pattern in 2 D code in generation and the scene image, or with scene image in the identical two-dimension code front elevation of two-dimension code stabilized zone pattern, described resource acquisition module is for the resource information of resolving described coded message, to obtain virtual information corresponding to two-dimension code;
Described image characteristics extraction module, the image in 2 D code that is used for scene image that two-dimension code front elevation and photographing module are caught carries out feature detection, and the feature that obtains respectively the two is described;
Described image is followed the tracks of registration module, is used for carrying out image registration according to the image in 2 D code feature description of two-dimension code front elevation and scene image, calculates the attitude of video camera, obtains homography matrix;
The described display module of playing up, be used for according to homography matrix, the certain deviation position of two-dimension code position or two-dimension code in real scene, play up and the described virtual information corresponding with two-dimension code of output display, when the described resource information of described resource acquisition module parses obtain be content of text the time, content of text is played up as texture; And be resource URI when what resolve that described resource information obtains, then to access this URI and obtain virtual information, and load with predetermined manner according to the virtual information type, described virtual information type comprises: one or more in video, image, text, the 3D model.
The invention allows for a kind of portable terminal, described portable terminal comprises the above-mentioned augmented reality system based on two-dimension code.
The present invention with two-dimension code as the identification in the augmented reality system and positioning mark, by the decoding of two-dimension code and the directly generation two-dimension code front elevation of encoding again, be used for the calculating of homography matrix, both broken in traditional augmented reality application, must be in database pre-stored corresponding sample image just can carry out the limitation of tracking and matching, when having avoided again using traditional marker to the template query of remote server and coupling step, can reduce the system response delay that causes because of the Internet Transmission problem, save user's network traffic.
Disclosed all features in this instructions, or the step in disclosed all methods or the process except mutually exclusive feature and/or step, all can make up by any way.
Disclosed arbitrary feature in this instructions (comprising any accessory claim, summary and accompanying drawing) is unless special narration all can be replaced by other equivalences or the alternative features with similar purpose.That is, unless special narration, each feature is an example in a series of equivalences or the similar characteristics.
The present invention is not limited to aforesaid embodiment.The present invention expands to any new feature or any new combination that discloses in this manual, and the arbitrary new method that discloses or step or any new combination of process.
Claims (10)
1. the augmented reality method based on two-dimension code is characterized in that, comprising:
Photographing module is caught the real scene image that contains two-dimension code;
Scan the two-dimension code in the scene image, and two-dimension code is decoded, obtain the coded message of two-dimension code, described coded message comprises: code system, version and resource information;
Coded message to the two-dimension code that obtains is encoded again, generate with scene image in two-dimension code code system and the identical two-dimension code front elevation of version; Resolve simultaneously described resource information, to obtain virtual information corresponding to two-dimension code;
Image in 2 D code in the scene image that two-dimension code front elevation and photographing module are caught carries out feature detection, and the feature that obtains respectively the two is described; And describe according to the feature of the two and to carry out image registration, calculate the attitude of video camera, obtain homography matrix;
According to homography matrix, play up and the described virtual information corresponding with two-dimension code of output display the certain deviation position of two-dimension code position or two-dimension code in real scene.
2. the method for claim 1 is characterized in that:
When resolve that described resource information obtains be content of text the time, content of text is played up as texture;
Be resource URI when what resolve that described resource information obtains, then access this URI and obtain virtual information, and load with predetermined manner according to the virtual information type.
3. method as claimed in claim 2, it is characterized in that: described virtual information type comprises: one or more in video, image, text, the 3D model.
4. such as each described method in the claims 1 to 3, it is characterized in that:
Two-dimension code in the described real scene image is conventional two-dimension code or customization of two-dimension code;
Resource information in the customization of two-dimension code comprises that resource identifier, resource type, resource load interface dimensions, play up in the drift rate of position one or more.
5. such as each described method in the claims 1 to 3, it is characterized in that, image in 2 D code in the described scene image that two-dimension code front elevation and photographing module are caught carries out feature detection, and the feature that obtains respectively the two is described, and is one of following dual mode:
Mode one: the image in 2 D code in the scene image that two-dimension code front elevation and photographing module are caught carries out the full figure feature detection, and the feature that obtains respectively the two is described;
Mode two: feature detection is carried out in the stabilized zone of the image in 2 D code in the scene image of only two-dimension code front elevation and photographing module being caught, and the feature that obtains respectively the two is described.
6. the augmented reality system based on two-dimension code is characterized in that, comprising:
Photographing module is used for catching the real scene image that contains two-dimension code;
The two-dimension code decoder module is used for the two-dimension code of scanning scene image, and two-dimension code is decoded, and obtains the coded message of two-dimension code, and described coded message comprises: code system, version and resource information;
The two-dimension code coding module is encoded to the coded message that described two-dimension code decoder module parses again, generate with scene image in two-dimension code code system and the identical two-dimension code front elevation of version;
The resource acquisition module is for the resource information of resolving described coded message, to obtain virtual information corresponding to two-dimension code;
The image characteristics extraction module, the image in 2 D code that is used for scene image that two-dimension code front elevation and photographing module are caught carries out feature detection, and the feature that obtains respectively the two is described;
Image is followed the tracks of registration module, is used for carrying out image registration according to the image in 2 D code feature description of two-dimension code front elevation and scene image, calculates the attitude of video camera, obtains homography matrix;
Play up display module, be used for according to homography matrix, play up and the described virtual information corresponding with two-dimension code of output display the certain deviation position of two-dimension code position or two-dimension code in real scene.
7. system as claimed in claim 6 is characterized in that, described image characteristics extraction module, and the image in 2 D code in the scene image that two-dimension code front elevation and photographing module are caught carries out feature detection, is one of following dual mode:
Mode one: the image in 2 D code in the scene image that two-dimension code front elevation and photographing module are caught carries out the full figure feature detection, and the feature that obtains respectively the two is described;
Mode two: feature detection is carried out in the stabilized zone of the image in 2 D code in the scene image of only two-dimension code front elevation and photographing module being caught, and the feature that obtains respectively the two is described.
8. such as claim 6 or 7 described systems, it is characterized in that:
Two-dimension code in the described real scene image is conventional two-dimension code or customization of two-dimension code;
Resource information in the customization of two-dimension code comprises that resource identifier, resource type, resource load interface dimensions, play up in the drift rate of position one or more.
9. such as claim 6 or 7 described systems, it is characterized in that,
When the described resource information of described resource acquisition module parses obtain be content of text the time, content of text is played up as texture; And be resource URI when what resolve that described resource information obtains, then to access this URI and obtain virtual information, and load with predetermined manner according to the virtual information type, described virtual information type comprises: one or more in video, image, text, the 3D model.
10. a portable terminal is characterized in that, described portable terminal comprises each described augmented reality system based on two-dimension code in the claim 6 to 9.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201210587153.1A CN103049729B (en) | 2012-12-30 | 2012-12-30 | Based on the augmented reality method of Quick Response Code, system and terminal |
| PCT/CN2013/081876 WO2014101435A1 (en) | 2012-12-30 | 2013-08-20 | Augmented reality method, system, and terminal based on two-dimensional code |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201210587153.1A CN103049729B (en) | 2012-12-30 | 2012-12-30 | Based on the augmented reality method of Quick Response Code, system and terminal |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN103049729A true CN103049729A (en) | 2013-04-17 |
| CN103049729B CN103049729B (en) | 2015-12-23 |
Family
ID=48062362
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201210587153.1A Active CN103049729B (en) | 2012-12-30 | 2012-12-30 | Based on the augmented reality method of Quick Response Code, system and terminal |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN103049729B (en) |
| WO (1) | WO2014101435A1 (en) |
Cited By (19)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2014101435A1 (en) * | 2012-12-30 | 2014-07-03 | 成都理想境界科技有限公司 | Augmented reality method, system, and terminal based on two-dimensional code |
| CN103996063A (en) * | 2014-06-12 | 2014-08-20 | 北京金山网络科技有限公司 | Method and device for processing data |
| CN104504402A (en) * | 2015-01-15 | 2015-04-08 | 刘畅 | Data processing method and system based on image search |
| CN104504155A (en) * | 2015-01-15 | 2015-04-08 | 刘畅 | Data acquisition method and system based on image search |
| CN104834680A (en) * | 2015-04-13 | 2015-08-12 | 西安教育文化数码有限责任公司 | Index-type reality enhancing method |
| CN104850582A (en) * | 2015-04-13 | 2015-08-19 | 西安教育文化数码有限责任公司 | Indexed augmented reality system |
| CN105446626A (en) * | 2015-12-04 | 2016-03-30 | 上海斐讯数据通信技术有限公司 | Augmented reality technology based commodity information acquisition method and system and mobile terminal |
| CN105787534A (en) * | 2016-02-29 | 2016-07-20 | 上海导伦达信息科技有限公司 | Realization method of content identification and learning and augmented reality, fused with two-dimensional code and AR code |
| CN105989390A (en) * | 2015-02-11 | 2016-10-05 | 北京鼎九信息工程研究院有限公司 | Generation method of two-dimensional code and apparatus thereof |
| CN106372144A (en) * | 2016-08-26 | 2017-02-01 | 江西科骏实业有限公司 | Two-dimensional code processing apparatus and method |
| CN106408667A (en) * | 2016-08-30 | 2017-02-15 | 西安小光子网络科技有限公司 | Optical label-based customized reality method |
| CN107250802A (en) * | 2015-02-24 | 2017-10-13 | 株式会社日立高新技术 | Automatic analysing apparatus |
| CN107464288A (en) * | 2017-07-24 | 2017-12-12 | 腾讯科技（深圳）有限公司 | Model display method and device |
| WO2018014601A1 (en) * | 2016-07-19 | 2018-01-25 | 央数文化(上海)股份有限公司 | Method and relevant apparatus for orientational tracking, method and device for realizing augmented reality |
| CN108665035A (en) * | 2017-03-31 | 2018-10-16 | 清华大学 | Code target generation method and device |
| CN109840951A (en) * | 2018-12-28 | 2019-06-04 | 北京信息科技大学 | The method and device of augmented reality is carried out for plane map |
| CN111291584A (en) * | 2016-07-22 | 2020-06-16 | 阿里巴巴集团控股有限公司 | Method and system for identifying position of two-dimensional code |
| CN111859199A (en) * | 2019-04-30 | 2020-10-30 | 苹果公司 | Locating content in an environment |
| TWI785332B (en) * | 2020-05-14 | 2022-12-01 | 光時代科技有限公司 | Three-dimensional reconstruction system based on optical label |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108986163A (en) * | 2018-06-29 | 2018-12-11 | 南京睿悦信息技术有限公司 | Augmented reality location algorithm based on more mark identifications |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101944187A (en) * | 2010-09-07 | 2011-01-12 | 龚湘明 | Two-dimensional micro code and processing method and device thereof |
| CN102323880A (en) * | 2011-06-30 | 2012-01-18 | 中兴通讯股份有限公司 | Mobile phone application interface development method and terminal based on browser parsing mode |
| CN102800065A (en) * | 2012-07-13 | 2012-11-28 | 苏州梦想人软件科技有限公司 | Augmented reality equipment and method based on two-dimensional code identification and tracking |
| CN102821323A (en) * | 2012-08-01 | 2012-12-12 | 成都理想境界科技有限公司 | Video playing method, video playing system and mobile terminal based on augmented reality technique |
| US20120327117A1 (en) * | 2011-06-23 | 2012-12-27 | Limitless Computing, Inc. | Digitally encoded marker-based augmented reality (ar) |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8350871B2 (en) * | 2009-02-04 | 2013-01-08 | Motorola Mobility Llc | Method and apparatus for creating virtual graffiti in a mobile virtual and augmented reality system |
| CN103049729B (en) * | 2012-12-30 | 2015-12-23 | 成都理想境界科技有限公司 | Based on the augmented reality method of Quick Response Code, system and terminal |
-
2012
- 2012-12-30 CN CN201210587153.1A patent/CN103049729B/en active Active
-
2013
- 2013-08-20 WO PCT/CN2013/081876 patent/WO2014101435A1/en active Application Filing
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101944187A (en) * | 2010-09-07 | 2011-01-12 | 龚湘明 | Two-dimensional micro code and processing method and device thereof |
| US20120327117A1 (en) * | 2011-06-23 | 2012-12-27 | Limitless Computing, Inc. | Digitally encoded marker-based augmented reality (ar) |
| CN102323880A (en) * | 2011-06-30 | 2012-01-18 | 中兴通讯股份有限公司 | Mobile phone application interface development method and terminal based on browser parsing mode |
| CN102800065A (en) * | 2012-07-13 | 2012-11-28 | 苏州梦想人软件科技有限公司 | Augmented reality equipment and method based on two-dimensional code identification and tracking |
| CN102821323A (en) * | 2012-08-01 | 2012-12-12 | 成都理想境界科技有限公司 | Video playing method, video playing system and mobile terminal based on augmented reality technique |
Cited By (26)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2014101435A1 (en) * | 2012-12-30 | 2014-07-03 | 成都理想境界科技有限公司 | Augmented reality method, system, and terminal based on two-dimensional code |
| CN103996063A (en) * | 2014-06-12 | 2014-08-20 | 北京金山网络科技有限公司 | Method and device for processing data |
| CN104504402A (en) * | 2015-01-15 | 2015-04-08 | 刘畅 | Data processing method and system based on image search |
| CN104504155A (en) * | 2015-01-15 | 2015-04-08 | 刘畅 | Data acquisition method and system based on image search |
| CN104504155B (en) * | 2015-01-15 | 2018-06-08 | 刘畅 | A kind of data capture method and system based on picture search |
| CN105989390A (en) * | 2015-02-11 | 2016-10-05 | 北京鼎九信息工程研究院有限公司 | Generation method of two-dimensional code and apparatus thereof |
| CN107250802B (en) * | 2015-02-24 | 2018-11-16 | 株式会社日立高新技术 | Automatic analysing apparatus |
| CN107250802A (en) * | 2015-02-24 | 2017-10-13 | 株式会社日立高新技术 | Automatic analysing apparatus |
| CN104850582B (en) * | 2015-04-13 | 2017-11-07 | 西安教育文化数码有限责任公司 | A kind of index type augmented reality system |
| CN104850582A (en) * | 2015-04-13 | 2015-08-19 | 西安教育文化数码有限责任公司 | Indexed augmented reality system |
| CN104834680A (en) * | 2015-04-13 | 2015-08-12 | 西安教育文化数码有限责任公司 | Index-type reality enhancing method |
| CN105446626A (en) * | 2015-12-04 | 2016-03-30 | 上海斐讯数据通信技术有限公司 | Augmented reality technology based commodity information acquisition method and system and mobile terminal |
| CN105787534A (en) * | 2016-02-29 | 2016-07-20 | 上海导伦达信息科技有限公司 | Realization method of content identification and learning and augmented reality, fused with two-dimensional code and AR code |
| CN105787534B (en) * | 2016-02-29 | 2018-07-10 | 上海导伦达信息科技有限公司 | Quick Response Code and the identification of AR digital contents are merged with study and with augmented reality implementation method |
| WO2018014601A1 (en) * | 2016-07-19 | 2018-01-25 | 央数文化(上海)股份有限公司 | Method and relevant apparatus for orientational tracking, method and device for realizing augmented reality |
| CN111291584A (en) * | 2016-07-22 | 2020-06-16 | 阿里巴巴集团控股有限公司 | Method and system for identifying position of two-dimensional code |
| CN111291584B (en) * | 2016-07-22 | 2023-05-02 | 创新先进技术有限公司 | Method and system for identifying two-dimensional code position |
| CN106372144A (en) * | 2016-08-26 | 2017-02-01 | 江西科骏实业有限公司 | Two-dimensional code processing apparatus and method |
| CN106408667A (en) * | 2016-08-30 | 2017-02-15 | 西安小光子网络科技有限公司 | Optical label-based customized reality method |
| CN106408667B (en) * | 2016-08-30 | 2019-03-05 | 西安小光子网络科技有限公司 | Customization practical method based on optical label |
| CN108665035A (en) * | 2017-03-31 | 2018-10-16 | 清华大学 | Code target generation method and device |
| CN108665035B (en) * | 2017-03-31 | 2020-12-22 | 清华大学 | Code mark generation method and device |
| CN107464288A (en) * | 2017-07-24 | 2017-12-12 | 腾讯科技（深圳）有限公司 | Model display method and device |
| CN109840951A (en) * | 2018-12-28 | 2019-06-04 | 北京信息科技大学 | The method and device of augmented reality is carried out for plane map |
| CN111859199A (en) * | 2019-04-30 | 2020-10-30 | 苹果公司 | Locating content in an environment |
| TWI785332B (en) * | 2020-05-14 | 2022-12-01 | 光時代科技有限公司 | Three-dimensional reconstruction system based on optical label |
Also Published As
| Publication number | Publication date |
|---|---|
| CN103049729B (en) | 2015-12-23 |
| WO2014101435A1 (en) | 2014-07-03 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN103049728B (en) | Based on the augmented reality method of Quick Response Code, system and terminal | |
| CN103049729A (en) | Method, system and terminal for augmenting reality based on two-dimension code | |
| CN106982387B (en) | Bullet screen display and push method and device and bullet screen application system | |
| CN105243715A (en) | Dynamic two-dimensional bar code based automatic registration method and system for district visiting users | |
| CN102800065A (en) | Augmented reality equipment and method based on two-dimensional code identification and tracking | |
| CN103415849A (en) | Computerized method and device for annotating at least one feature of an image of a view | |
| CN108491897B (en) | Information identification method, server, client and system | |
| CN105654471A (en) | Augmented reality AR system applied to internet video live broadcast and method thereof | |
| KR20190032501A (en) | Method and system for recognizing location information in a two-dimensional code | |
| CN105243142A (en) | Intelligent information push method based on augmented reality and visual computation | |
| CN103561065A (en) | System and method for achieving 3D virtual advertisement with mobile terminal | |
| CN109636868B (en) | High-precision image map online construction method and device based on WebGIS and deep learning | |
| CN202918341U (en) | Geography position sharing system based on intelligent hand-held terminal and two-dimension code | |
| CN104361075A (en) | Image website system and realizing method | |
| CN103150392A (en) | Method for comparing prices of similar commodities | |
| CN103929669A (en) | Interactive video generator, player, generating method and playing method | |
| CN113542777A (en) | Live video editing method and device and computer equipment | |
| CN105447534A (en) | Imaged-based information presenting method and device | |
| CN105959814A (en) | Scene-recognition-based video bullet screen display method and display apparatus thereof | |
| KR101427715B1 (en) | Apparatus and method for recognizing quick response code | |
| Zhang et al. | Aesthetic QR codes based on two-stage image blending | |
| CN104571527B (en) | A kind of 3D molecules interaction docking system and implementation method based on augmented reality | |
| CN109034214B (en) | Method and apparatus for generating a mark | |
| CN110345950B (en) | Road coding method and road matching method | |
| CN116762090A (en) | Method, system, and non-transitory computer-readable recording medium for supporting experience sharing between users |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant |