CN110442813B

CN110442813B - Travel commemorative information processing system and method based on AR

Info

Publication number: CN110442813B
Application number: CN201910439257.XA
Authority: CN
Inventors: 黄国权; 张锋文; 吴迪斌; 莫秀聪
Original assignee: Guangdong Pharmaceutical University
Current assignee: Guangdong Pharmaceutical University
Priority date: 2019-05-24
Filing date: 2019-05-24
Publication date: 2023-01-24
Anticipated expiration: 2039-05-24
Also published as: CN110442813A

Abstract

The invention belongs to the technical field of AR (augmented reality) tourism, and discloses a tourist commemorative information processing system and a method based on AR, wherein the tourist commemorative information processing system based on AR comprises a user management module, a user information management module, a data processing module and a data processing module, wherein the user management module is used for user information management, including registration, modification and the like; the AR tourism function module is used for AR recognition, AR commemorative sticker self-definition, sending and storing shared AR pictures, focusing marked sites and checking AR stickers of corresponding scenery spots; the sticker unit has the function of generating and issuing a preset or self-defined AR commemorative sticker for binding the real scene of the real object; the map function module is used for positioning, displaying marked base points within 2 kilometers around, and automatically generating marked base points- 'recall points' when the user uses the AR tourism module to generate AR stickers to bind scenery spots; the memory module is used for realizing sharing and interaction between users in a picture (AR screenshot) and short text mode. The invention is based on AR technology, incorporates a commemorative sticker, and optimizes and upgrades the commemorative and recall modes of tourists.

Description

AR-based travel commemorative information processing system and method

Technical Field

The invention belongs to the technical field of AR (augmented reality) tourism, and particularly relates to a system and a method for processing tourist commemorative information based on an AR (augmented reality).

Background

Currently, the current state of the art commonly used in the industry is such that:

at present, the world economic situation brings certain challenges to the tourism market in China, particularly to the inbound tourism, but the basic surface of the good development of the tourism industry in China is not changed, the advantages and opportunities are still many, and the tourism industry in China is still in the gold development period. According to the prediction of the world tourism association, the income of the global tourism industry is increased to 16 trillion dollars by 2020, which is equivalent to 10% of the global GDP; can provide 3 hundred million work posts, which account for 9.20 percent of the total employment of the whole world.

However, the sharing link of sightseeing and traveling thoughts still has obvious defects, the mode of most passengers commemorating and commenting in the journey still remains in the past concept at the pain point, the interaction among the passengers is less, the commemoration mode is limited to traditional shooting and letters, the memory is only brought back, and the memory cannot be kept at the sight point, for example, some local traditional gourmets or artware are bought as the commemoration, the shooting commenting is more common, and after the journey is finished, the passengers are biased to leave points instead of bringing things, for example, trees studded with coins, walls full of messages and even lettering behaviors for destroying cultural relics.

In summary, the problems of the prior art are as follows:

(1) The traditional concept is kept for the mode of commemoration and memory of the journey, the expression mode is single, and the environmental cost is high;

(2) The interaction between passengers is few, the commemorative mode is limited to traditional shooting, hand letter and burning, the information processing aspect is limited by the equipment, the networking degree is low, and dynamic networking processing and real-time information updating cannot be realized.

(3) The memorial information can not be stored in the scenic spot and uploaded to the network, the ornamental value is lacked, and the tourist information and the memorial information are difficult to store and manage in a network way.

The difficulty of solving the technical problems is as follows:

(1) The method is not suitable for upgrading the information processing system by adding or upgrading the existing hardware equipment, because the method must increase the cost and reduce the portability, the daily use requirements of mass tourism users cannot be met;

(2) Traditional memorial information recording media such as camera shooting and hand letter have limited capacity and single backup, and are easy to lose and damage, thereby causing information data source loss;

(3) The traditional memorial information modes such as camera shooting and hand letter have low networking degree and low information processing speed, cannot realize real-time dynamic processing, and no effective scheme can advance the information processing performance at present.

The significance of solving the technical problems is as follows:

(1) The conventional commemorative information processing system is improved, the travel commemorative information processing speed is greatly increased, and dynamic updating, networked storage and management are realized;

(2) The method can reduce the environmental cost, improve the interaction behavior of the passengers and improve the user experience of the passengers.

Disclosure of Invention

Aiming at the problems in the prior art, the invention provides a tourist commemorative information processing system and method based on AR.

The invention is realized in this way, a travel commemorative information processing system based on AR, the travel commemorative information processing system based on AR includes:

a user management module: the system comprises an information management unit, a story management unit, a scenic spot collection unit and a private letter unit; the system is used for user information management, including registration, modification, story management and collection management;

AR tourism functional module: the system comprises an object recognition and tracking unit, an AR screenshot unit and a sticker unit; the AR commemorative sticker is used for AR identification and self-definition, and sending and storing the shared AR pictures, focusing the marked sites and checking the AR stickers of the corresponding scenery spots; the sticker unit has the function of generating and issuing a preset or self-defined AR commemorative sticker for binding the real scene of the real object;

the map function module: the system comprises a user positioning unit and a mark generating and viewing unit; the system is used for realizing positioning by using an LBS technology, displaying marked data points within 2 kilometers around, and automatically generating marked data points- 'recall points' when an AR sticker is generated by a user by using an AR tourism module and bound with a scenery spot;

a memory module: the system comprises a feeling recommendation unit, a feeling release unit and a feeling watching unit; the method is used for realizing sharing and interaction among users in a picture and short text mode.

Further, the user management module specifically includes:

an information management unit: the system is used for managing user information, and comprises user login, registration and exit, and user information such as user names, passwords, mobile phone numbers, mailboxes and the like can be checked and modified;

a story management unit: the system is used for managing the story article information published by the user, modifying and deleting the story article published by the user, and managing comments of other users on the article;

a scenic spot collection unit: the system is used for managing the collection of the scenic spots of the user, and comprises the steps of adding collection, deleting collection and sharing collection, and setting the collection open or non-open viewing permission;

a private letter unit: the method is used for information exchange among users, and private information content comprises, but is not limited to, pictures and texts, videos and voice.

Further, the AR travel function module specifically comprises:

an object recognition and tracking unit: realizing the moving and scaling of the object and the model by realizing the recognition and the tracking; the AR tourism function is turned on, the marked scenery can be identified or the current scenery can be tracked and marked after the automatic focusing is complete, and the separation of virtual data and an application platform can be realized through a cloud identification technology;

paster unit: the commemorative sticker is divided into preset and self-defined AR doodle, a cloud database is constructed by adopting a mode that the commemorative sticker is bound with a real scene of a real object and combining a cloud recognition technology, and AR commemorative sticker data can be stored in a server end for a long time;

an AR screenshot unit: and the AR recognition effect is stored, and the shared platform is served for travel recall and sharing and is used for network sharing and transmission.

Further, the recall module specifically includes:

the feeling recommending unit: the system has a Tag keyword classification system and a user preference setting function, and can push journey thoughts which are interesting to users or have higher social attention according to users with different personalities;

the idea issuing unit: the user can click a home page release button, enter a release manuscript page and narrate and share the journey story and the feeling of the user;

a perception viewing unit: the system is used for showing the journey story or feeling issued by the passenger spontaneously, entering a journey story detail page, browsing the details of the journey story of the issuer, and allowing the user to like, leave a message and share the details.

Further, the map function module specifically includes:

a user positioning unit: LBS is taken as a basic technology, and a user positioning and map is provided based on Baidu map SDK development;

a tag generation and viewing unit: the marker points automatically generated when the AR tourism module is used for generating the commemorative mapping indicate that the skill sticker added by the passenger is stored in the marker places, and the memory points within 2 kilometers are displayed by matching with the map LBS positioning technology and the geographic fence to search whether the memory points exist around.

Another object of the present invention is to provide an AR-based tourist souvenir information processing method operating the AR-based tourist souvenir information processing system, the AR-based tourist souvenir information processing method including:

step one, a user inputs a registered telephone number through a front-end system, and the system automatically sends a request to a background database to check the service condition of the registered telephone number. If the telephone number is not registered, the system replies a request to a front-end system, and the registration state of the telephone number is set as registerable, otherwise, the registration state is set as unregisterable; if the registration state is registrable, the user inputs name, age and login password through a front-end form, and selects information such as native place, gender, occupation and the like to register; if the registration is successful, generating a unique ID for the user, and using the unique ID for verifying the user and subsequent information processing operation by the background system; after the user logs in through the front-end system, the user can modify the personal basic information and use all functions of the system;

step two, the user opens the AR tourism function module through the front-end system, focuses on scenes or things worth commenting on, waits for the system to automatically focus, clicks and adds preset patterns or self-defined doodle AR stickers, the system sends a request to the background system, operates an AR program to automatically associate the patterns with the actual scenes, and synchronizes to the Internet for commemorative information storage and dynamic processing; by focusing the marked base points, the AR stickers of the corresponding scenery spots can be checked; clicking a camera icon at the top navigation bar to carry out screenshot commenting or sharing;

step three, the user opens the map function module through the front-end system, after the user allows the position use authority, the front-end system sends a request to a Baidu map development platform and returns the position information of the user, the front-end system processes the position information in real time and synchronizes the map view of the front end, the automatic positioning for the user is realized, and the user can click a left lower positioning button to perform positioning; the map function module acquires the position information of the marked data points from the background system by using the geo-fencing technology, then the self-defined program runs, the position information is checked with the positions within 2 kilometers around, processed data are returned to the front-end system, and the marked data points within 2 kilometers around are displayed for a user;

step four, the system pushes user trip story sharing with higher attention degree for different users individually according to data analysis; the user can view the travel memories of other people and can also release the travel dynamics by matching the short story sharing platform in the system with the AR travel function; the pushed share can be clicked by the system, the publisher profile is entered, and the publisher and the release story profile are checked; the short story detail can be entered through the system, and the details of the travel story of the publisher can be browsed, praise and leave a message.

Another object of the present invention is to provide a computer program for implementing the AR-based travel commemorative information processing method.

Another object of the present invention is to provide an information data processing terminal for implementing the AR-based travel commemoration information processing method.

It is another object of the present invention to provide a computer-readable storage medium including instructions which, when run on a computer, cause the computer to perform the AR-based travel commemorative information processing method.

In summary, the advantages and positive effects of the invention are:

the invention integrates a 'memorial sticker' based on AR technology, optimizes and upgrades the mode of commemoration and recall of tourists, realizes that recognizable and traceable scenery generates the AR memorial sticker, the memorial sticker can be kept in a scenic place for a long time due to the virtualization and the AR augmented reality technology and does not pass along with the end of a journey, and a user can watch the memorial stickers left by other people in the journey by matching the journey recall sharing platform with an AR tourism module, thereby enhancing the interaction between the tourists in the aspect of commemoration, achieving the effect of sharing the 'recall' with other users, and bringing a brand new and more advanced mode of commemoration to the journey to the user. The improvement of the invention to ORB-SLAM makes the device have robustness far beyond other schemes: even for a poor scene and a poor calibration internal reference, the ORB-SLAM can work smoothly, better feature points are provided for the AR module, and meanwhile, marking coordinates are provided for the map module. The main advantages of LBS are, firstly, no hardware investment: such powerful management functions are realized based on the most basic mobile technology, and business personnel can report data by means of wap by using a mobile phone only with little training. Secondly, cannot be destroyed: compared with other positioning modes, the positioning technology adopted by the invention does not depend on independent hardware equipment, and the position of the personnel can be obtained when the mobile phone is started, so that the personnel cannot be damaged, and the positioning method is more reliable and stable. The function pair ratio of the prior traditional travel commemorative information processing method is shown in the table 1:

TABLE 1 comparison of functions of AR-based travel commemorative information processing method and traditional method

Because the AR-based tourist commemorative information processing system has virtual components and real environment of the real world, the system has extremely high immersion environment. Augmented reality systems will become a new type of medium, where tracking registration is one of the most critical technologies of augmented reality systems, and registration is actually the process of "aligning" a computer-generated virtual object with a scene in a real environment. Registration must first determine the relationship between the virtual object and the observer and then project the virtual object out of the observer's field of view by the correct geometric projection-registration is achieved by affine transformation. The travel commemorative information processing system based on AR carries out 'alignment' to the scene in travel commemorative information and the real environment based on tracking registration technology to realize that travel commemorative information and geographical position track the registration, solved and do not support the problem that information and real geographical position bind in traditional commemorative information processing system, wherein registration technology includes but is not limited to following mode:

(1) Registration by scaling a camera

Camera calibration is the acquisition of camera internal parameters and then the calculation of the camera position and orientation from these parameters and the acquired image. This is in effect a transformation process from a three-dimensional scene to a two-dimensional imaging plane. Camera parameters of general interest include the focal length of the lens and the height, width, aspect ratio, etc. of the sensor pixels.

(2) Registration by affine transformation

Registration via affine transformation is in fact an application of computer graphics and computer vision techniques in augmented reality system design. Konenderink et al have proposed that given any at least four non-coplanar points in three-dimensional space, the projective transformation of any one point in space can be represented by a linear combination of the transformation results for these four points. By introducing the definition of the global affine coordinate system, the virtual object and the real world are placed under the same unified coordinate system, thereby conveniently realizing registration. Moreover, the registration method also enables an estimation of depth. Similar systems have also been discussed by Kiriakos et al abroad and specific augmented reality systems have been designed by this approach. The method for realizing registration through affine transformation is a breakthrough of an augmented reality registration technology, and the method bypasses the traditional registration methods such as tracking, calibration and the like which are tedious and easy to generate larger errors, and realizes registration through computer vision analysis.

Meanwhile, the invention supports novel interaction technologies such as voice interaction and gesture interaction in commemorative information interaction, and in the interaction technologies, the voice, the gesture, the action and the like of a user are captured by a computer, a mobile device and the like to be used as input, so that the method is more visual and natural compared with the traditional information interaction mode. The user can realize the interaction with the virtual information in the real world in a more accurate mode, the human-computer interaction mode is a more natural man-machine interaction mode with virtual-real fusion, and the human-computer interaction mode is more suitable for being applied to an augmented reality system. How to effectively express information on an AR display is difficult to interact with virtual information, and the research of AR interaction means has two trends, namely, firstly, different devices are used to gain the best of each family; for display information in an AR display, the following problems are to be solved:

(1) Visualization of the erroneous estimates. Secondly, when the virtual object shielded by the real object is drawn, the hidden virtual object is gradually faded out along the edge of the shielded area, so that the positioning error is reduced;

(2) The data density. If the real world is augmented with a large amount of virtual information, the display becomes confusing and difficult to read using a space-based interaction model, the amount of information displayed is reduced to a minimum, and only important information is retained in the view;

(3) And (5) drawing the reality sense. In AR applications, the key to improving the rendering quality of virtual objects is to be able to automatically obtain the illumination and reflection information of the environment. There are three methods at present, using a model to estimate the illumination parameters, based on the rendering of the image and the illumination acquisition of the dynamic range;

(4) Reality of regulation (MediatedReality). To remove the real object the system must be able to segment the individual objects Lepetit in the scene discusses a semi-automatic method of distinguishing the objects and their positions in the scene by contours, in some cases enabling the insertion of virtual objects and the removal of real objects without the need for 3D reconstruction of the environment.

In recent years, the contribution of the tourism industry of China to GDP is continuously increasing. In 2017, the comprehensive contribution of the tourism industry in China to the GDP is 9.13 trillion yuan, which accounts for 11.04% of the total amount of the GDP, compared with 2014, the comprehensive contribution of the tourism industry to the GDP is increased by 2.52 trillion yuan, the proportion of the tourism industry to the total amount of the GDP is also improved, and the pulling effect of the tourism industry to the economic growth is enhanced, as shown in a 132014-2017 comparison graph of the comprehensive contribution of the tourism industry in China to the GDP. In the spring festival holiday of 2019, the total number of tourism receptions in China is 4.15 hundred million people, the tourism income is 5139 hundred million yuan, and the increase is 8.2 percent on a year-by-year basis. The culture and tourism department publishes the information in 19 years, 1 month and 12 days, the number of domestic tourists in 2018 is 55.39 hundred million people, and the number of outbound tourists approaches 1.5 hundred million people. The data show that with the improvement of the living standard of people, the travel consumption is greatly increased, and the market is huge.

With the increase of the market of the tourism industry and the popularization and development of the internet technology, the rise of the technologies such as the internet of things, big data, cloud computing and the like, the development of the smart tourism is greatly promoted, the smart tourism enters the practical application level from the concept, the smart tourism brings the numerical change for the tourism industry, and the traditional service industry is changed to the modern service industry at an accelerated speed. However, the modernization of the commemoration mode still does not appear in the aspect of travel commemoration, the traditional travel commemoration shooting, letter and information burning are mainly used, the traditional travel commemoration shooting, letter and information burning are limited by the equipment, the networking degree is low, and the dynamic networking processing, the real-time information updating, voice, gesture and other interactions cannot be realized. The tourism information processing system based on AR can improve current commemorative information processing system, realizes that the information interface is visual and the gesture interaction, promotes the tourism commemorative information processing speed greatly, realizes dynamic update, and networking is preserved and is managed, can reduce the environmental cost, can improve passenger's interactive behavior again, improves passenger user experience, has filled the blank that domestic and foreign commemorative information processing market needs modernization.

Drawings

FIG. 1 is a schematic structural diagram of an AR-based tourist commemorative information processing system according to an embodiment of the present invention.

In the figure: 1. a user management module; 2. an AR travel function module; 3. a map function module; 4. a recall module; 5. an information management unit; 6. a story management unit; 7. a scenic spot collection unit; 8. a private letter unit; 9. an object recognition and tracking unit; 10. an AR screenshot unit; 11. a sticker unit; 12. a user positioning unit; 13. a tag generation and viewing unit; 14. a feeling recommendation unit; 15. a feeling release unit; 16. a unit is perceived to be viewed.

FIG. 2 is a functional diagram of an AR-based tourist commemorative information processing system according to an embodiment of the present invention.

Fig. 3 is a flowchart of ORB feature detection according to an embodiment of the present invention.

Fig. 4 is a flowchart of the SLAM algorithm according to an embodiment of the present invention.

Fig. 5 is a diagram of applying SLAM technology according to an embodiment of the present invention.

Fig. 6 is a diagram of a jitter optimization algorithm provided by an embodiment of the present invention.

Fig. 7 is a flowchart of a two-hand scaling algorithm provided by an embodiment of the present invention.

Fig. 8 is a flowchart of a model dragging algorithm provided in an embodiment of the present invention.

Fig. 9 is a flow chart of LBS location provided by the embodiment of the present invention.

Fig. 10 is a flowchart of a geo-fencing algorithm provided by an embodiment of the present invention.

FIG. 11 is a flow chart of the AR-based travel commemorative information processing provided by an embodiment of the present invention.

Fig. 12 is a network topology diagram of a test environment according to an embodiment of the present invention.

Fig. 13 is a comparison graph of the combined contribution of 2014-2017 to GDP in the tourism industry in china according to the embodiment of the present invention.

Detailed Description

In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is further described in detail with reference to the following embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and do not limit the invention.

The application of the principles of the present invention will now be described in further detail with reference to the accompanying drawings.

As shown in fig. 1-2, the AR-based tourist souvenir information processing system according to the embodiment of the present invention includes:

the system comprises a user management module 1, an AR tourism function module 2, a map function module 3 and a recall module 4;

the user management module 1: the system comprises an information management unit 5, a story management unit 6, a scenic spot collection unit 7 and a private letter unit 8; the method is used for user information management, including registration, modification and the like;

AR tourism functional module 2: the system comprises an object recognition and tracking unit 9, an AR screenshot unit 10 and a sticker unit 11; the AR commemorative sticker is used for AR identification and self-definition, and sending and storing the shared AR pictures, focusing the marked sites, and checking the AR stickers of the corresponding scenery spots; the sticker unit has the function of generating and issuing a preset or self-defined AR commemorative sticker for binding the real scene of the real object;

the map function module 3: comprises a user positioning unit 12 and a mark generating and viewing unit 13; the system is used for realizing positioning by using an LBS technology, displaying marked data points within 2 kilometers around, and automatically generating marked data points- 'recall points' when an AR sticker is generated by a user by using an AR tourism module and bound with a scenery spot;

memory module 4: comprises a feeling recommending unit 14, a feeling publishing unit 15 and a feeling watching unit 16; the method is used for realizing sharing and interaction between users in a picture (AR screenshot) and short text mode.

Further, the AR tour function module 2 specifically includes:

the object recognition and tracking unit 9: based on an ORB algorithm and an SLAM technology, model jitter problem and interaction are optimized based on easy ARSDK development, and object and model moving scaling is realized by recognition and tracking; the AR tourism function is turned on, the marked scenery can be identified or the current scenery can be tracked and marked after the automatic focusing is complete, and the separation of virtual data and an application platform can be realized through a cloud identification technology;

AR screenshot cell 10: the method mainly saves AR recognition effects, serves a travel memory sharing platform and is used for network sharing and transmission;

sticker unit 11: the memorial sticker is divided into preset and self-defined AR graffiti, for example, picture stickers, character stickers, dynamic stickers and the like have memorial patterns, the mode that the memorial sticker is bound with the real scene of the real object is adopted, the cloud identification technology is combined, a cloud database is built, and AR memorial sticker data can be stored at the server end for a long time.

The ORB algorithm provided by the embodiment of the present invention specifically includes:

ORB (organized FAST and organized BRIEF) is an algorithm for FAST feature point extraction and description; the ORB algorithm is divided into two parts, namely feature point extraction and feature point description; the feature extraction is developed by FAST (Features from accessed Segment Test) algorithm, and the feature point description is improved according to BRIEF (Binary Robust independent element Features) feature description algorithm; mainly provides more excellent characteristic point identification technology for the SLAM technology.

The SLAM technique provided by the embodiment of the present invention specifically includes:

SLAM (simultaneous Localization and Mapping), also known as CML (current Mapping and Localization), performs instant positioning and Mapping, or performs simultaneous Mapping and positioning; the method comprises the steps that a three-dimensional map is built by collecting base points through an SLAM technology, three-dimensional geometric information of a scene is provided for an AR tourism module, so that an AR tourism model can enable a virtual object to interact with the scene, and positioning (motion tracking) and map building (sparse or dense map) form the core function of the SLAM; the SLAM technology is mainly applied to the AR tourism module in the following 5 scenes:

(1) Surrounding landmarks such as buildings, big trees, flower beds and the like are observed through the visual field of the camera, the characteristics of the landmarks are recorded, and the extraction of the most basic functional characteristics is provided for image recognition;

(2) Reconstructing the characteristic landmark in a three-dimensional map according to the characteristic data of the internal environment of the system (three-dimensional reconstruction);

(3) Continuously acquiring new characteristic landmarks when the environment changes, and correcting a map model (bundle adjustment or EKF) in the system;

(4) From the characteristic landmarks in the system, the analysis determines its own position (objectory).

The development based on the easy AR SDK provided by the embodiment of the invention specifically comprises the following steps:

easy AR SDK is a powerful AR (Augmented Reality) engine in China; the application of easy AR SDK in the invention comprises:

(1) Combined with SLAM techniques: by means of the SLAM technology, more accurate object feature point information is obtained, monocular real-time 6-degree-of-freedom camera attitude tracking is achieved, and single-frame initialization, rapid relocation, motion blur robustness, mobile terminal optimization and low-texture scene optimization are achieved;

(2) Object recognition and tracking: through feature point matching, the planar image and the 3D object are identified and tracked in real time, and the spatial position attribute of the object is obtained, so that the AR commemorative sticker model can accurately track real scenes; optimizing the model jitter, if the jitter range is larger than a preset value, keeping the last position, wherein the core jitter optimization algorithm has the following points:

1) Firstly, acquiring the angles of the lens moving in the x axis, the y axis and the z axis, judging the angles, if the change angle is larger than 270, using algorithm optimization to forcibly convert all the change angles which are not within 1-270 into-270;

2) If the lens shaking range is larger than a preset value, when the model shaking exceeds a certain range, the coordinate angle of the model is not corrected, and only the current coordinate and the angle are recorded as the last coordinate; and if the model shaking range is too small, correcting the model coordinates to be the last correct coordinates.

(3) Self-defining a doodle model: the method comprises the steps of supporting a custom model and an AR custom doodling function, acquiring a user custom memorial pattern through a doodling panel, transmitting the user custom memorial pattern to an AR module, and binding a real object;

(4) Screen gesture interaction is supported, and the model realizes multi-point touch, zooming in and out, displacement and sharing of screenshot and video;

1) Multi-touch scaling algorithm: the algorithm compares the distance between the new touch points 2 on the screen with the distance between the touch points 2 at the last time, the distance difference is positive to represent the zooming-in gesture, and the distance difference is negative to represent the zooming-out gesture, and the core algorithm has the following points:

1. if no touch exists, namely the touch point is 0 or less than 0, no operation is performed;

2. if multi-point touch comprises zooming in and zooming out, recording the coordinates of a touch point, and just starting to contact the screen from the 2 nd point, only recording and not processing;

3. calculating the distance between the old two points and the new distance between the two points for the recorded touch points, wherein if the distance between the two points is positive, the zooming-in gesture is represented, and if the distance between the two points is negative, the zooming-out gesture is represented;

4. with the yield of the magnification factor, one pixel is calculated by 0.01 times (100 is adjustable), and if the scale factor of the x, y and z axes of magnification or reduction is more than or equal to 0.05f, the magnification or reduction is carried out;

5. remember the latest touch point for the next alignment.

2) The algorithm converts the space coordinate of the mapping screen and the space coordinate of the world, calculates the deviation value of the space coordinate of the screen after dragging and the space coordinate of the world, and updates the space coordinate of the mapping world, wherein the core algorithm is as follows:

1. converting the world space coordinate of the target object into the screen space coordinate of the target object, and storing the screen space coordinate of the mouse (the Z value uses the screen space coordinate of the target object);

2. calculating the offset of the target object and the mouse object in the world space;

3. when the left mouse button is pressed, storing the screen space coordinates of the mouse (the Z value uses the screen space coordinates of the target object);

4. converting the screen space coordinate of the mouse into a world space coordinate (the Z value uses the screen space coordinate of the target object), and adding an offset to the world space coordinate to obtain the world space coordinate of the target object;

5. the world space coordinates of the target object are updated.

(5) Cloud identification platform: cloud identification service is provided by easy AR, and image identification retrieval service based on the cloud comprises an image target cloud retrieval function and an image target management function:

1) Cloud identification localization: after the cloud identification is successful once, the cloud identification image can be cached in a local identification library, and the target can be quickly and accurately identified repeatedly even if no network exists;

2) Constructing a cloud database: the cloud database is constructed, the limitation is reduced by combining with the mobile internet, and the ordered management and construction of data are realized by combining with big data.

The map function module 3 provided by the embodiment of the present invention specifically includes:

the user positioning unit 12: the LBS is used as a basic technology, and the positioning and the map of the user are provided based on the development of the Baidu map SDK, and the LBS also comprises an optimal tour route, accommodation recommendation and a traffic route;

the mark generation and viewing unit 13: the marker points automatically generated when the AR tourism module is used for generating the commemorative mapping indicate that the skill sticker added by the passenger is stored in the marker places, and the memory points within 2 kilometers are displayed by matching with the map LBS positioning technology and the geographic fence to search whether the memory points exist around.

The LBS technology provided by the embodiment of the present invention specifically includes:

LBS (Location-Based Services), also known as Location Services, is a value-added service provided by combining a mobile communication network and a satellite positioning system, obtains Location information (such as longitude and latitude coordinate data) of a mobile terminal through a group of positioning technologies, provides the Location information to a mobile user or others and a communication system, and realizes various Location-related Services.

The geo-fence provided by the embodiment of the invention specifically comprises:

geo-fencing (Geo-fencing) is an application of LBS, i.e. a virtual fence enclosing a virtual geographical boundary, which can sense the mark of the surrounding boundary; by using the technology, the marked base point is retrieved in the map module, the AR recalls the automatic marking of the base point, and then the marked base point is displayed to the user on the map module through the icon.

The SDK development platform based on the Baidu map provided by the embodiment of the invention specifically comprises the following steps:

the Baidu map android SDK is a set of application program interfaces based on android4.0 and above version equipment, can realize functions such as navigation and site marking, and is specifically applied to the following steps:

(1) Map and navigation: the map comprises a common map (2D, 3D), a satellite map and a real-time traffic map, and gesture control is supported to realize operations of clicking, double clicking, long pressing, zooming, rotating, changing a visual angle and the like of the map; the navigation can provide the bus information inquiry, the walking route planning, the driving route planning and other route planning functions;

(2) Site display and automatic marking: by combining a geo-fencing algorithm, recall data points in an annular area with the radius of 2 kilometers away from the center are retrieved by taking a user as the center, and data point information details can be displayed; when the AR tourism module generates a memorial map, automatically generating a recall base point on the map by LBS positioning technology;

(3) Geofence retrieval mechanism:

peripheral retrieval: searching by taking a user as a center and taking the appointed distance as a radius of 2 kilometers;

area retrieval: searching in the specified rectangular area according to the keywords;

and (3) retrieval in cities: in a certain city, searching according to keywords input by a user;

detail retrieval: the site details are retrieved based on information such as the recall point ID.

The recall module 4 provided by the embodiment of the present invention specifically includes:

the feeling recommending unit 14: the system has a Tag keyword classification system and a user preference setting function, and can push journey thoughts which are interesting to users or have higher social attention according to users with different personalities;

the feeling issuing unit 15: the user can click a home page release button, enter a release manuscript page and narrate and share the journey story and the feeling of the user;

the sensory viewing unit 16: the system is used for showing the journey story or feeling issued by the passenger spontaneously, entering a journey story detail page, browsing the details of the journey story of the issuer, and allowing the user to like, leave a message and share the details.

The travel commemorative information processing system based on AR provided by the embodiment of the invention also comprises:

(1) System time characteristics:

1) Response time requirement: the software starting and closing response is less than 0.5 second, the map positioning response is less than 1s, and the first starting requirement of the AR module is less than 3s;

2) Update processing time requirement: recommending update for less than 1.5 seconds, updating the map for less than 2s, and updating the AR model for less than 2s;

3) Conversion and transfer time requirements of the data: the interaction between the module and the server is less than 2 seconds;

(2) System flexibility:

1) Change of operating environment: the method is suitable for a mainstream Android platform;

2) Variation in precision and effective time limit: the influence of the precision change on the APP is reduced to the minimum;

1) Planned changes or improvements: is easier to be improved.

(3) System confidentiality:

the user information is kept secret from other users so as to ensure the personal privacy of the users; when a user logs in, the user name and the password need to be verified, and for a user with a wrong password input for three times, the user needs to wait for 3 minutes before the user can input the password again.

The method for processing the travel commemorative information based on the AR comprises the following steps:

s101, a user inputs a registered telephone number, checks the registered telephone number, and sets the registration state as registrable if the telephone number is not registered yet, or sets the registration state as unregisterable if not; if the registration state is registerable, the user inputs name, age and login password, selects native place, gender and occupation to register; if the registration is successful, generating a unique ID for the user; after logging in, the personal basic information can be modified;

s102, opening an AR (augmented reality) tourism function, focusing scenes or things worth commenting, waiting for full automatic focusing, adding preset patterns or custom doodle AR stickers, and clicking camera icons at a top navigation bar to carry out screenshot commenting or sharing; focusing the marked base points, and checking the AR stickers of the corresponding scenery spots;

s103, a map function is turned on, the map automatically positions the user, a left lower positioning button can be clicked for positioning, and marked data points within 2 kilometers around are displayed by means of a geo-fence technology;

s104, the system pushes different trip story shares of other users with higher attention according to individuation of different users, and the users can view the trip memories of other people through the story short text sharing platform in cooperation with the AR travel function and can also release the trip dynamics; clicking the pushed share, entering the publisher profile, and checking the publisher and the published story profile; and entering the details of the short story text, browsing the details of the travel story of the publisher, and commenting and leaving a message.

The application of the principles of the present invention will now be described in further detail with reference to specific embodiments.

The specific embodiment is as follows:

1. system testing

(1) Test type and test method

(2) Test environment

1) Test environment and configuration

2) Network topology, as shown in fig. 12.

(3) Tester

Job title	Name(s)	E-mail	Telephone set
				Development engineer	Zhang Feng Wen	2441784417@qq.com	17875595409
Tester 1	Mongolian hearing-improving device	737575467@qq.com	15989217825
				Tester 2	Wu Dibin	781721794@qq.com	13642833738

(4) Problem of testing

In the process of starting testing and putting into use of the app, problems which affect the use experience of users to a greater or lesser extent exist, and a small part of the problems are not solved completely.

2. Summary of the tests

(1) Test case execution results

(2) Compatibility testing

(3) Installation, uninstallation test

1) Mounting of

2) Unloading

(4) UI testing

1) Navigation testing

Content of test	Test results
		Requiring navigation between different connected pages	Is that
Whether navigation is easy and whether navigation is intuitive	Is that
		Whether or not there is a search engine	Is that
Whether navigation assistance is accurate and intuitive	Is that
		Whether the navigation is consistent with the style of the page structure, the menu and the connection page	Is that

2) Pattern testing

3) Content testing

(5) Functional testing

1) Operation of

Content of test	Test results
		Software can be normally opened in test running after App installation	Is that
App opens test, whether there is a progress prompt of loading status	Is that
		App opening speed test whether the speed is considerable	Whether or not
Whether switching among App pages is smooth or not and whether logic is correct or not	Correct logic and not smooth enough switching

2) Registration

Content of test	Test results
		Background availability to add users	Is that
Whether the username and password length is limited	Is that
		Whether a prompt page exists after registration	Whether or not
Whether foreground registered page data are consistent with background management page data	Is that
		After registration, page hints are provided in background management	Whether or not

3) Logging in

4) Logging off

5) Foreground-background switching of applications

6) Exempt from to login

7) Data update

8) Offline browsing

Content of test	Test results
		Browsing local data without network	Whether or not
Can browse normally when exiting the app and then starting the app	Whether or not
		Switching to background and switching back to foreground for normal browsing	Whether or not
After the screen is locked, the screen is unlocked and the application foreground is returned to be capable of browsing normally	Whether or not
		When the data of the server side is updated, corresponding prompt of offline is given	Whether or not

9) APP update

10 Time testing

11 Performance test

a. Response testing

Content of test	Test results
		If the network is disconnected when the App is operated and other uses of the mobile phone are influenced	Whether or not
Response time of installation/uninstallation, whether it is acceptable	Is that
		Response time of various functional operations of App, whether it can be accepted	NO (the subscription function response of the recommended item is a bit slow)
Repeatedly installing and uninstalling to check whether the system resources are normal	Is that

b. Cross-sexual event testing

(6) Security testing

1) Software permissions

a. Risk of deduction

Content of test	Whether or not to deduct fee	Remarks to note
			Receiving and sending short messages	Whether or not	Receiving short message without charge (such as short message identifying code)
Dialing telephone	Is that	The customer service telephone provided in the APP is dialed with a fee (charged by the operator)
			Connecting network	Is that	Connection 2G, 3G, 4G and WIFI networks has a fee (charged by the operator)

b. Risk of privacy disclosure

c. Restrictive testing

Content of test	Whether or not to limit
		Function for limiting/allowing use of mobile phone to send and receive information	Allow for
Restricting/allowing applications to register for auto-launch applications	Allow for
		Restricting/allowing user data to be read using a mobile phone	Allow for
Limiting/permitting use of mobile phone writer user data	Allow for

2) Installation and uninstallation security

3) Data security

4) Communication security

5) Human-machine interface security

6) User experience testing

3. Criteria for evaluation

(1) Range(s) of

The selected test case can be subjected to basic correct and wrong judgment, whether the user name and the password are legal or not can be checked, so that the user authority is given, the authority can be checked and judged only when an administrator modifies information, and the local test can be completely finished only through the final integral test

In the whole testing process, the test case can check various errors of the system, the functions required by the system are basically realized, and the processing speed of the system can be accepted by a user. However, the test has its limitations, and cannot completely include various situations, such as the bearing capacity of the system when the user has a large number of accesses at the same time.

(2) Data collation

In order to put the data into an appropriate form for evaluation, the conversion processing technique to be applied needs to be manual, and a part of the testers needs to process the recorded input/output data.

(3) Dimension(s)

The types of reasonable output results are: various conditions occurring in the use process of the system are clearly and clearly shown, and the error processing path is provided when an error occurs; a little deviation is allowed between the test output result and the expected output, namely the output result sometimes cannot correctly judge the problem, and then a tester is required to perform further test to help a developer to perform error checking; the maximum number of allowed interruptions or outages is temporarily not a definite limit.

It should be noted that the embodiments of the present invention can be realized by hardware, software, or a combination of software and hardware. The hardware portions may be implemented using dedicated logic; the software portions may be stored in a memory and executed by a suitable instruction execution system, such as a microprocessor or specially designed hardware. It will be appreciated by those skilled in the art that the apparatus and methods described above may be implemented using computer executable instructions and/or embodied in processor control code, for example such code provided on a carrier medium such as a diskette, CD-or DVD-ROM, a programmable memory such as read-only memory (firmware) or a data carrier such as an optical or electronic signal carrier. The apparatus of the present invention and its modules may be implemented by hardware circuits such as very large scale integrated circuits or gate arrays, semiconductors such as logic chips, transistors, or programmable hardware devices such as field programmable gate arrays, programmable logic devices, or software executed by various types of processors, or a combination of hardware circuits and software, e.g., firmware.

The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents and improvements made within the spirit and principle of the present invention are intended to be included within the scope of the present invention.

Claims

1. An AR-based travel commemorative information processing system, characterized in that the AR-based travel commemorative information processing system comprises:

the user management module is used for managing, registering and modifying user information;

the AR tourism function module is used for AR recognition, AR commemorative sticker self-definition, sending and storing shared AR pictures, focusing marked sites and checking AR stickers of corresponding scenery spots;

the map function module is used for realizing positioning by using LBS technology, displaying marked base points within 2 kilometers around, and automatically generating marked base points-recall points when the user uses the AR tourism module to generate AR stickers to bind scenery spots;

the memory module is used for realizing sharing and interaction between users in a picture and short text mode;

the user management module comprises an information management unit, a story management unit, a scenic spot collection unit and a private letter unit;

the recall module comprises a feeling recommendation unit, a feeling release unit and a feeling watching unit;

the recall module specifically comprises:

the feeling recommending unit: the system has a Tag keyword classification system and a user preference setting function, and can push the travel thoughts which are interesting to the user or have higher social attention according to users with different personalities;

a perception viewing unit: the system is used for displaying the journey story or the feeling spontaneously issued by the passenger, entering a journey story detail page, browsing the journey story detail of the issuer, and commenting, leaving a message and sharing;

the AR tourism function module specifically comprises:

an object recognition and tracking unit: realizing the moving and scaling of the object and the model by realizing the recognition and the tracking; the AR tourism function is started, the marked scenery can be identified or the current scenery can be tracked and marked after the automatic focusing is complete, and the separation of virtual data and an application platform can be realized through a cloud identification technology;

paster unit: the method comprises the steps that the commemorative sticker is divided into preset and custom AR graffiti, a cloud database is constructed by adopting a mode that the commemorative sticker is bound with a real object scene and combining a cloud recognition technology, and AR commemorative sticker data can be stored in a server side for a long time;

an AR screenshot unit: the AR recognition effect is stored, and the server serves a journey recall sharing platform and is used for network sharing and transmission;

the map function module specifically includes:

a user positioning unit: LBS is used as a basic technology, and the method is developed based on a Baidu map SDK to provide user positioning and a map;

2. An AR-based travel commemorative information processing method of operating the AR-based travel commemorative information processing system of claim 1, wherein the AR-based travel commemorative information processing method comprises:

step one, a user inputs a registered telephone number, checks the registered telephone number, sets the registration state as registrable if the telephone number is not registered yet, otherwise sets the registration state as unregisterable; if the registration state is registerable, the user inputs name, age and login password, selects native place, gender and occupation to register; if the registration is successful, generating a unique ID for the user;

after logging in, the personal basic information can be modified;

opening an AR (augmented reality) tourism function, focusing scenes or objects worth commenting, waiting for full automatic focusing, adding preset patterns or custom doodle AR stickers, and clicking a camera icon at the top navigation bar to perform screenshot commenting or sharing; focusing the marked base points, and checking the AR stickers of the corresponding scenery spots;

step three, turning on the map function, automatically positioning the map for the user, and also clicking a left lower positioning button to position, and displaying marked data points within 2 kilometers around by using the geo-fencing technology;

step four, the system pushes different trip story shares of other users with higher attention according to different users in an individualized way, and the users can watch the trip memories of other people through the story short-article sharing platform and cooperate with the AR travel function and can also release the trip dynamics;

clicking the pushed share, entering the publisher profile, and checking the publisher and the published story profile; and entering the details of the short story text, browsing the details of the travel story of the publisher, and commenting and leaving a message.

3. A computer program for implementing the AR-based travel commemorative information processing method of claim 2.

4. An information data processing terminal implementing the AR-based travel commemorative information processing method of claim 2.

5. A computer-readable storage medium comprising instructions which, when executed on a computer, cause the computer to perform the AR-based travel commemorative information processing method of claim 2.