CN101510913A - System and method for implementing intelligent mobile phone enhancement based on three-dimensional electronic compass - Google Patents

System and method for implementing intelligent mobile phone enhancement based on three-dimensional electronic compass Download PDF

Info

Publication number
CN101510913A
CN101510913A CNA2009100145974A CN200910014597A CN101510913A CN 101510913 A CN101510913 A CN 101510913A CN A2009100145974 A CNA2009100145974 A CN A2009100145974A CN 200910014597 A CN200910014597 A CN 200910014597A CN 101510913 A CN101510913 A CN 101510913A
Authority
CN
China
Prior art keywords
mobile phone
electronic compass
smart mobile
dimensional electronic
real time
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CNA2009100145974A
Other languages
Chinese (zh)
Inventor
常勇
孙瑛
薛立明
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shandong Normal University
Original Assignee
Shandong Normal University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shandong Normal University filed Critical Shandong Normal University
Priority to CNA2009100145974A priority Critical patent/CN101510913A/en
Publication of CN101510913A publication Critical patent/CN101510913A/en
Pending legal-status Critical Current

Links

Images

Landscapes

  • Processing Or Creating Images (AREA)

Abstract

The invention discloses an intelligent cellphone reality reinforcing system based on a three-dimensional electronic compass and a realizing method, which solves a problem occurring when a mobile termination reality reinforcing system is established; through an integrated micro electronic compass, the invention has the advantages of low cost, handy application, and long service time of power. The method is: a. real-time location is carried out through GPS built-in the intelligent cellphone; b. data from the electronic compass are received in real time by the intelligent cellphone through three-dimensional electronic compass software and hardware interfaces in order to determine information of a visual line direction; c. video information is collected through a CCD video camera of the intelligent cellphone; d. software is drawn by the intelligent cellphone by means of a three-dimensional virtual pattern; virtual patterns are drawn by means of information obtained by the procedure a and procedure b; e. a reality reinforcing image is obtained by the intelligent cellphone by means of a software for overlaying a virtual image and a video image in real time in a pattern builder installed in the intelligent cellphone, as well as the video image in the procedure c and the virtual image in the procedure d.

Description

Intelligent mobile phone enhancement system and implementation method based on three-dimensional electronic compass
Technical field
The present invention relates to a kind of intelligent mobile phone enhancement system and implementation method based on three-dimensional electronic compass.
Background technology
Augmented reality (AR, Augmented Reality), usually being also referred to as and expanding reality, enhancing vision or augmented reality vision, is the new research direction that development is come out on the basis of virtual reality research in the world in recent years, also is a difficult point hot issue of virtual reality research field.The augmented reality technology be exactly virtual image that computer is generated or other information organically, in real time, dynamically stack (or fusion) is in the middle of the actual environment that the observer saw, and the actual environment around these virtual informations and the user combines together, make the user be sure of that from sensory effects virtual environment is its organic component of true environment on every side, when the user is mobile in real scene, dummy object also changes thereupon, just look like these dummy objects be really be present in the real scene the same.This enhancing information can be the dummy object that coexists with real-world object in true environment, also can be the non-geological information about the real-world object that exists.From the present situation that develops both at home and abroad at present, the augmented reality system still is in laboratory stage, and the tracking registration technology of having succeeded in developing at present generally need just can be obtained accurate result under the condition of ambient controlled.With respect to the development of indoor augmented reality system, the development of outdoor augmented reality system relatively lags behind.
Virtual image to be superimposed upon on the scene of objective world exactly, just must determine user's position and direction of visual lines accurately, in real time.For outdoor augmented reality system, the method for tracking and positioning that adopts usually in the world mainly divides three classes at present: based on the registration location of computer vision, based on the registration location of tracking equipment, based on the mixing registration location of vision and tracking equipment.Based on the three-dimensional register method of computer vision based on theory on computer vision, the actual environment image that ccd video camera photographs is handled and discerned, obtain trace information, the equipment that is characterized in is simple, with low cost, at present the research based on the registration technology of computer vision is in leading position in the AR field.But, adopt the logon mode of computer vision also improper on a large scale for unknown outdoor environment.Based on the logon mode of tracking equipment, its location tracking equipment comprises GPS (Global Position System) (GPS), inertial navigation system (INS), three-dimensional electronic compass, accelerometer, inclinometer or the like.Because each tracer technique respectively has length, for outdoor AR system, also can satisfy its harsh requirement fully at present without any a kind of performance of tracking technique.Therefore, the final method that Welch (2002) and Azuma people such as (1998) have proposed to address this problem is to adopt to mix (hybrid) tracking technique, promptly by merging multiple tracer technique, learn from other's strong points to offset one's weaknesses, obtain comparatively accurate, reliable tracking results, but meanwhile also increased complexity, operand and the cost of system.At present, mainly there is following problem in the outdoor augmented reality system based on wearable computer or portable computer:
1, equipment heaviness, the user bears a heavy burden greatly, carries inconvenience;
2, system is owing to the restriction that is subjected to carrying power supply, and system's duration is short;
3, complete set of equipments cost height.
Though can realize the miniaturization, portable of outdoor augmented reality system being difficult to adopt the combined tracking technology based on the augmented reality system of portable terminal, mainly have following problem at present owing to be subjected to the restriction of hardware condition:
Though 1, smart mobile phone or PDA have built-in GPS and can position, owing to there is not integrated three-dimensional electronic compass, can't determine the direction of sight line, therefore just can't set up the intelligent mobile phone enhancement system;
2, some mobile phone the is integrated chip of electronic compass, but these mobile phones are not smart mobile phones generally, are difficult to carry out software development again and realize the augmented reality function, and what these mobile phones were integrated in addition generally all is two-dimentional electronic compass, the azimuth can only be provided, can't satisfy the requirement of augmented reality.
Progressively open develop rapidly with the mobile communication cause along with China's 3G network, people are to the 4A (Anytime of geospatial information, Anywhere, Anybody, Anything) Fu Wu demand manifests day by day, portable terminal combines with spatial Information Service, spatial orientation information service (LBS) occurred, thus the information service relevant that provides the user to need with the position.Augmented reality technology and LBS combine the information that can realize in real time, mutual, dynamically, three-dimensional display, for example can smart mobile phone real time inspection underground pipe network, the literal annotation information can be superimposed upon in real time and build on the thing or the like.Use the user of smart mobile phone also increasing year by year among the huge cellphone subscriber group of China, some is not necessarily very accurate for the positional information requirement in these high-end users, the precision of meter level or sub-meter grade just can meet the demands, and therefore adopts the augmented reality system based on smart mobile phone just to have very big application potential.
Summary of the invention
Purpose of the present invention is exactly in order to solve the problem of implementation based on the intelligent mobile phone enhancement system of three-dimensional electronic compass, provide a kind of integrated miniature three-dimensional electronic compass, GPS, ccd video camera, the intelligent mobile phone enhancement system and the implementation method based on three-dimensional electronic compass of advantage such as with low cost, easy to carry, power using time is long.
For achieving the above object, the present invention adopts following technical scheme:
A kind of implementation method of the intelligent mobile phone enhancement system based on three-dimensional electronic compass, concrete steps are:
A. locate in real time by the built-in GPS of smart mobile phone;
B. smart mobile phone, the hardware interface real-time data that receive from electronic compass soft by three-dimensional electronic compass are determined direction of visual lines information;
C. gather video information by the CCD video camera of smart mobile phone;
D. smart mobile phone utilizes three-dimensional graphic plotting software, and the information of utilizing step a and step b to obtain is drawn virtual pattern;
E. smart mobile phone utilizes virtual image in the graphic generator of installing in the smart mobile phone and the video image software that superposes in real time with the virtual pattern of the video image of step c and steps d, and real world images is enhanced.Utilize the built-in GPS of smart mobile phone to locate in real time among the described step a, system obtains the locator data of smart mobile phone GPS in real time, therefrom extracts positional information.
Smart mobile phone utilizes that three-dimensional electronic compass is soft among the described step b, hardware interface receives the data from electronic compass in real time, and extract real-time went out direction of visual lines information after system received the data of electronic compass, and handled accordingly; Wherein the hardware interface type is a serial ports, and three-dimensional electronic compass can obtain power supply by the serial ports of smart mobile phone, and gives the system transmissions data.
Three-dimensional electronic compass is external miniature three-dimensional electronic compass or electronic compass built-in or that be integrated into smart mobile phone chips among the described step b.
Among the described step c, the CCD video camera of smart mobile phone is gathered video information, each two field picture in the system acquisition video sequence, and the figure that is input to system as texture is produced device.
A kind of intelligent mobile phone enhancement system based on three-dimensional electronic compass, it comprises smart mobile phone, and smart mobile phone is provided with GPS module and ccd video camera, and described smart mobile phone is connected with three-dimensional electronic compass by serial ports.
The present invention is on the smart mobile phone of built-in GPS, utilize the interface between software and hardware of three-dimensional electronic compass, make it can be used as the data of main frame acceptance, and, realize the augmented reality function by the software that independently developed virtual image and video image superpose in real time from electronic compass.
The invention has the beneficial effects as follows: this smart mobile phone with augmented reality function, integrated miniature three-dimensional electronic compass, GPS, ccd video camera, with low cost, use light, the power supply longer duration of system.
Description of drawings
The whole principle schematic of Fig. 1 patent of the present invention;
Fig. 2 system receives and handles the gps data theory diagram;
Fig. 3 system receives and handles electronic compass data theory diagram;
Fig. 4 realizes virtual image and the video image augmented reality principle of work and power schematic diagram that superposes in real time;
Fig. 5 is a structural representation of the present invention.
Embodiment
Below in conjunction with accompanying drawing and example patent of the present invention is described further.
The present invention relates to a kind of implementation method with smart mobile phone of augmented reality function based on three-dimensional electronic compass, this method utilizes the CCD video camera of smart mobile phone 1 to gather video information, utilize built-in GPS to position, utilize that independently developed three-dimensional electronic compass is soft, hardware interface receives and carries out orientation from the data of compass, utilizes the software of independently developed virtual image and video image stack in real time to realize the augmented reality function.
Whole principle schematic of the present invention as shown in Figure 1, three-dimensional electronic compass 101 by and the special purpose interface 102 (serial ports) of smart mobile phone 1 obtain power supply from smart mobile phone 1, simultaneously with transfer of data to smart mobile phone 1.After the augmented reality system of smart mobile phone 1 receives the data of three-dimensional electronic compass transmission,, extract needed direction of visual lines information 103 through handling.Simultaneously, smart mobile phone 1 augmented reality system obtains position data in real time by built-in GPS104, through data processing 105, obtains positional information 106 in real time.The intelligent mobile phone enhancement system superposes virtual three-dimensional figure 109 and video captured sequence 108 in graphic generator 110 in real time by the ccd video camera 107 capture video images sequences 108 of smart mobile phone, and real world images 111 is enhanced.
The principle schematic that intelligent mobile phone enhancement of the present invention system obtains built-in gps data as shown in Figure 2, system initialization GPS port 201 at first, open this port then and open timer 202, open thread 203 then, with the character is that unit receives gps data 204, by the data that receive are handled 205, obtain positional information 206.
The interface of smart mobile phone 1 of the present invention and three-dimensional electronic compass as shown in Figure 3, system is initialization three-dimensional electronic compass port 301 at first, open this port then and open timer 302, begin reading of data 303 then, with the character is that unit regularly receives the data 304 that read, character 305 by processing satisfies condition obtains bearing data 306.
The present invention realizes that virtual image and video image superpose augmented reality principle of work and power schematic diagram as shown in Figure 4 in real time, at first system enters circulation 401, catch a frame video image 402 then, in the back buffering area 403 of smart mobile phone, handle texture display image and virtual pattern simultaneously, and piece image before buffering area 404 shows before exchanging to, finally realized the stack in real time of virtual image and video image ground by exchange buffering by dynamic demonstration 405.

Claims (6)

1. implementation method based on the intelligent mobile phone enhancement system of three-dimensional electronic compass is characterized in that concrete steps are:
A. locate in real time by the built-in GPS of smart mobile phone;
B. smart mobile phone, the hardware interface real-time data that receive from electronic compass soft by three-dimensional electronic compass are determined direction of visual lines information;
C. gather video information by the CCD video camera of smart mobile phone;
D. smart mobile phone utilizes three-dimensional graphic plotting software, and the information of utilizing step a and step b to obtain is drawn virtual pattern;
E. smart mobile phone utilizes virtual image in the graphic generator of installing in the smart mobile phone and the video image software that superposes in real time with the virtual pattern of the video image of step c and steps d, and real world images is enhanced.
2. the implementation method of the intelligent mobile phone enhancement system based on three-dimensional electronic compass according to claim 1, it is characterized in that, utilize the built-in GPS of smart mobile phone to locate in real time among the described step a, system obtains the locator data of smart mobile phone GPS in real time, therefrom extracts positional information.
3. the implementation method of the intelligent mobile phone enhancement system based on three-dimensional electronic compass according to claim 1, it is characterized in that, smart mobile phone utilizes that three-dimensional electronic compass is soft among the described step b, hardware interface receives the data from electronic compass in real time, the later extract real-time of data that system receives electronic compass goes out direction of visual lines information, and handles accordingly; Its hardware interface type is a serial ports, and three-dimensional electronic compass can obtain power supply by the serial ports of smart mobile phone, and gives the system transmissions data.
4. according to the implementation method of claim 1 or 3 described intelligent mobile phone enhancement systems based on three-dimensional electronic compass, it is characterized in that three-dimensional electronic compass is external miniature three-dimensional electronic compass or electronic compass built-in or that be integrated into smart mobile phone chips among the described step b.
5. the implementation method of the intelligent mobile phone enhancement system based on three-dimensional electronic compass according to claim 1, it is characterized in that, among the described step c, the CCD video camera of smart mobile phone is gathered video information, each two field picture in the system acquisition video sequence, the figure that is input to system as texture is produced device, utilizes the virtual image of graphic generator and the video image software that superposes in real time to realize the augmented reality function.
6. intelligent mobile phone enhancement system based on three-dimensional electronic compass, it comprises smart mobile phone, and smart mobile phone is provided with GPS module and ccd video camera, it is characterized in that, and described smart mobile phone is connected with three-dimensional electronic compass by serial ports.
CNA2009100145974A 2009-03-17 2009-03-17 System and method for implementing intelligent mobile phone enhancement based on three-dimensional electronic compass Pending CN101510913A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CNA2009100145974A CN101510913A (en) 2009-03-17 2009-03-17 System and method for implementing intelligent mobile phone enhancement based on three-dimensional electronic compass

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CNA2009100145974A CN101510913A (en) 2009-03-17 2009-03-17 System and method for implementing intelligent mobile phone enhancement based on three-dimensional electronic compass

Publications (1)

Publication Number Publication Date
CN101510913A true CN101510913A (en) 2009-08-19

Family

ID=41003174

Family Applications (1)

Application Number Title Priority Date Filing Date
CNA2009100145974A Pending CN101510913A (en) 2009-03-17 2009-03-17 System and method for implementing intelligent mobile phone enhancement based on three-dimensional electronic compass

Country Status (1)

Country Link
CN (1) CN101510913A (en)

Cited By (20)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101969488A (en) * 2010-09-10 2011-02-09 中兴通讯股份有限公司 Terminal and method for realizing compass function on terminal
CN102123194A (en) * 2010-10-15 2011-07-13 张哲颖 Method for optimizing mobile navigation and man-machine interaction functions by using augmented reality technology
CN102346660A (en) * 2011-10-13 2012-02-08 苏州梦想人软件科技有限公司 System and method for mixing real and virtual objects
CN102348029A (en) * 2010-07-26 2012-02-08 柯尼卡美能达商用科技株式会社 Hand-held device and apparatus management method
CN102402778A (en) * 2010-07-23 2012-04-04 株式会社泛泰 Apparatus and method for providing augmented reality using additional data
CN102592255A (en) * 2010-12-07 2012-07-18 卡西欧计算机株式会社 Information display system, information display apparatus, and storage medium
CN102696057A (en) * 2010-03-25 2012-09-26 比兹摩德莱恩有限公司 Augmented reality systems
CN103186528A (en) * 2011-12-27 2013-07-03 中国电信股份有限公司 Method, device and mobile terminal for screening geographical information
CN103384881A (en) * 2010-12-23 2013-11-06 谷歌公司 View dependent techniques to determine user interest in a feature in a 3D application
CN103632626A (en) * 2013-12-03 2014-03-12 四川省计算机研究院 Intelligent tour guide realizing method and intelligent tour guide device based on mobile network and mobile client
CN104123743A (en) * 2014-06-23 2014-10-29 联想(北京)有限公司 Image shadow adding method and device
CN104765801A (en) * 2011-03-07 2015-07-08 科宝2股份有限公司 Systems and methods for analytic data gathering from image providers at event or geographic location
CN104899920A (en) * 2015-05-25 2015-09-09 联想(北京)有限公司 Image processing method, image processing device and electronic device
CN104982090A (en) * 2013-03-15 2015-10-14 英特尔公司 Personal information communicator
CN106302444A (en) * 2016-08-16 2017-01-04 深圳市巴古科技有限公司 Intelligent cloud recognition methods
WO2017041731A1 (en) * 2015-09-11 2017-03-16 Huawei Technologies Co., Ltd. Markerless multi-user multi-object augmented reality on mobile devices
WO2017152600A1 (en) * 2016-03-11 2017-09-14 Effire Universal Limited Smartphone with a vr content capturing assembly
CN109246195A (en) * 2018-08-13 2019-01-18 孙琤 A kind of pipe network intelligence management-control method and system merging augmented reality, virtual reality
CN109874045A (en) * 2019-01-30 2019-06-11 深圳岚锋创视网络科技有限公司 A kind of portable terminal and its obtain the method that gyro data carries out video processing
CN110036635A (en) * 2016-12-28 2019-07-19 微软技术许可有限责任公司 Alleviate the system, method and computer-readable medium of motion sickness via the display of the enhancing for passenger for using video capture device

Cited By (27)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102696057A (en) * 2010-03-25 2012-09-26 比兹摩德莱恩有限公司 Augmented reality systems
CN102402778A (en) * 2010-07-23 2012-04-04 株式会社泛泰 Apparatus and method for providing augmented reality using additional data
CN102348029A (en) * 2010-07-26 2012-02-08 柯尼卡美能达商用科技株式会社 Hand-held device and apparatus management method
CN101969488B (en) * 2010-09-10 2014-06-11 中兴通讯股份有限公司 Terminal and method for realizing compass function on terminal
CN101969488A (en) * 2010-09-10 2011-02-09 中兴通讯股份有限公司 Terminal and method for realizing compass function on terminal
CN102123194A (en) * 2010-10-15 2011-07-13 张哲颖 Method for optimizing mobile navigation and man-machine interaction functions by using augmented reality technology
CN102123194B (en) * 2010-10-15 2013-12-18 张哲颖 Method for optimizing mobile navigation and man-machine interaction functions by using augmented reality technology
CN102592255A (en) * 2010-12-07 2012-07-18 卡西欧计算机株式会社 Information display system, information display apparatus, and storage medium
CN103384881A (en) * 2010-12-23 2013-11-06 谷歌公司 View dependent techniques to determine user interest in a feature in a 3D application
CN104765801A (en) * 2011-03-07 2015-07-08 科宝2股份有限公司 Systems and methods for analytic data gathering from image providers at event or geographic location
CN102346660A (en) * 2011-10-13 2012-02-08 苏州梦想人软件科技有限公司 System and method for mixing real and virtual objects
CN103186528A (en) * 2011-12-27 2013-07-03 中国电信股份有限公司 Method, device and mobile terminal for screening geographical information
CN104982090A (en) * 2013-03-15 2015-10-14 英特尔公司 Personal information communicator
CN104982090B (en) * 2013-03-15 2019-07-02 英特尔公司 Personal information communication device
CN103632626A (en) * 2013-12-03 2014-03-12 四川省计算机研究院 Intelligent tour guide realizing method and intelligent tour guide device based on mobile network and mobile client
CN104123743A (en) * 2014-06-23 2014-10-29 联想(北京)有限公司 Image shadow adding method and device
CN104899920A (en) * 2015-05-25 2015-09-09 联想(北京)有限公司 Image processing method, image processing device and electronic device
WO2017041731A1 (en) * 2015-09-11 2017-03-16 Huawei Technologies Co., Ltd. Markerless multi-user multi-object augmented reality on mobile devices
US9928656B2 (en) 2015-09-11 2018-03-27 Futurewei Technologies, Inc. Markerless multi-user, multi-object augmented reality on mobile devices
WO2017152600A1 (en) * 2016-03-11 2017-09-14 Effire Universal Limited Smartphone with a vr content capturing assembly
CN106302444A (en) * 2016-08-16 2017-01-04 深圳市巴古科技有限公司 Intelligent cloud recognition methods
CN110036635A (en) * 2016-12-28 2019-07-19 微软技术许可有限责任公司 Alleviate the system, method and computer-readable medium of motion sickness via the display of the enhancing for passenger for using video capture device
US11057574B2 (en) 2016-12-28 2021-07-06 Microsoft Technology Licensing, Llc Systems, methods, and computer-readable media for using a video capture device to alleviate motion sickness via an augmented display for a passenger
CN109246195A (en) * 2018-08-13 2019-01-18 孙琤 A kind of pipe network intelligence management-control method and system merging augmented reality, virtual reality
CN109246195B (en) * 2018-08-13 2023-11-24 孙琤 Intelligent management and control method and system for pipe network integrating augmented reality and virtual reality
CN109874045A (en) * 2019-01-30 2019-06-11 深圳岚锋创视网络科技有限公司 A kind of portable terminal and its obtain the method that gyro data carries out video processing
WO2020155596A1 (en) * 2019-01-30 2020-08-06 影石创新科技股份有限公司 Portable terminal and method thereof for acquiring gyroscope data to perform video processing

Similar Documents

Publication Publication Date Title
CN101510913A (en) System and method for implementing intelligent mobile phone enhancement based on three-dimensional electronic compass
CN103424113B (en) Indoor positioning and navigating method of mobile terminal based on image recognition technology
US9728007B2 (en) Mobile device, server arrangement and method for augmented reality applications
WO2011118282A1 (en) Server using world coordinate system database and terminal
WO2011118903A1 (en) Augmented reality systems
CN110858414A (en) Image processing method and device, readable storage medium and augmented reality system
CN101753807B (en) Image pick-up device
CN103335657A (en) Method and system for strengthening navigation performance based on image capture and recognition technology
KR101600456B1 (en) Method, system and recording medium for providing augmented reality service and file distribution system
WO2021093679A1 (en) Visual positioning method and device
KR100968837B1 (en) Portable camera system provides information about captured objects
CN102647512A (en) All-round display method of spatial information
CN201374736Y (en) Intelligent mobile phone augmented reality system based on three-dimensional electronic compass
CN102788577A (en) Positioning device and positioning method using augmented reality technology
CN104159036A (en) Display method and shooting equipment of image direction information
CN104748739A (en) Intelligent machine augmented reality implementation method
CN202171801U (en) Generating device including position information pictures and processing device including position information pictures
CN107607110A (en) A kind of localization method and system based on image and inertial navigation technique
KR20110070210A (en) Mobile terminal and method for providing augmented reality service using position-detecting sensor and direction-detecting sensor
CN103491631A (en) Indoor positioning system and method based on two-dimension code and wifi signals
CN116858215B (en) AR navigation map generation method and device
CN107958491B (en) Matching method of mobile augmented reality virtual coordinates and construction site coordinates
CN204069178U (en) A kind of hi-Fix video surveillance acquisition device based on CORS system
CN102156866A (en) Road sign recognition system and method
CN116701554A (en) Three-dimensional space data quality inspection method and system based on GIS, BIM and panoramic image technology

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
C02 Deemed withdrawal of patent application after publication (patent law 2001)
WD01 Invention patent application deemed withdrawn after publication

Open date: 20090819