CN109783465B - Mass three-dimensional model integration system under cloud computing framework - Google Patents
Mass three-dimensional model integration system under cloud computing framework Download PDFInfo
- Publication number
- CN109783465B CN109783465B CN201811590873.7A CN201811590873A CN109783465B CN 109783465 B CN109783465 B CN 109783465B CN 201811590873 A CN201811590873 A CN 201811590873A CN 109783465 B CN109783465 B CN 109783465B
- Authority
- CN
- China
- Prior art keywords
- dimensional model
- data
- cloud computing
- model
- computing framework
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Abstract
The invention relates to a mass three-dimensional model integration system under a cloud computing framework, which comprises: the three-dimensional model storage module is used for realizing the storage of a three-dimensional model based on a NoSQL database; the three-dimensional model access module is used for carrying out collaborative concurrent access on the stored three-dimensional model; and the three-dimensional model recommendation module is used for realizing implicit recommendation of the three-dimensional model based on the Spark cloud computing framework. Compared with the prior art, the invention has the advantages of high storage efficiency, strong concurrency, high-efficiency storage and the like.
Description
Technical Field
The invention relates to a storage and application technology of a mass three-dimensional model, in particular to a mass three-dimensional model integration system under a cloud computing framework.
Background
With the rapid development of virtual reality technology and the Internet, scientific researchers have proposed a large number of applications of virtual reality oriented to the Internet, including smart cities and the like, and the construction of a smart city platform oriented to multi-user interaction is more and more urgent. However, some key links are severely limited by application of virtual reality at the browser end, including coordination problems between limited memory at the browser end and limited bandwidth of network transmission, concurrent demands of multiple people and real application of the internet lacking concurrency, intelligent demands of virtual reality and intelligent mining bottleneck of big data of a three-dimensional model, and the like. Meanwhile, smart city applications facing the mobile internet still face a number of problems, mainly expressed in: the scale of the three-dimensional model is striking and the actual storage mechanism is single and inefficient. The method has the advantages that the loading speed of the browser is low, interaction response of the model in the scene is not timely, and the like, so that the model in the virtual city scene is simple in overall structure, low in scene complexity and single in interaction means in the scene. In reality, the city structure is complex and the interaction means are various. These deficiencies severely restrict the application of virtual reality technology in large-scale virtual cities.
In addition, with the rapid development of artificial intelligence technology, the demand of people for intelligent application is increasing, related research on large data mining is being touted more and more, the connection between virtual reality and artificial intelligence is becoming more and more tight, and in reality, the technology and artificial intelligence for virtual reality are still in the respective independent development stages. Increasingly, it is urgent to build intelligent virtual reality applications, and efficient computation of mass data is required for both virtual reality and artificial intelligence technologies.
Disclosure of Invention
The invention aims to overcome the defects of the prior art and provide a mass three-dimensional model integration system under a cloud computing framework.
The aim of the invention can be achieved by the following technical scheme:
a mass three-dimensional model integration system under a cloud computing framework, comprising:
the three-dimensional model storage module is used for realizing the storage of a three-dimensional model based on a NoSQL database;
the three-dimensional model access module is used for carrying out collaborative concurrent access on the stored three-dimensional model;
and the three-dimensional model recommendation module is used for realizing implicit recommendation of the three-dimensional model based on the Spark cloud computing framework.
Further, the data stored by the three-dimensional model storage module comprises three-dimensional model data, three-dimensional model auxiliary information, three-dimensional model keyword information and machine learning result information.
Further, the three-dimensional model auxiliary information comprises a model uploader, coordinates of a center point of a model surrounding a sphere and names and categories of the model.
Further, the three-dimensional model recommendation module includes:
the data acquisition unit reads log file data of the three-dimensional model library through Spark and groups the data;
the data analysis unit analyzes the read data to form statistical data;
the training unit is used for training by taking the statistical data as a learning sample;
and the prediction unit is used for realizing implicit recommendation of the three-dimensional model based on the training result and outputting a prediction result.
Further, in the data acquisition unit, the read data is saved as an elastic distributed file.
Further, the parsing in the data parsing unit includes a persistence operation on the elastic distributed file, a data Map operation and a Reduce operation.
Further, the training unit trains the ALS function.
Compared with the prior art, the invention has the following beneficial effects:
(1) Efficient storage mechanism: the storage management efficiency of the three-dimensional model can be greatly improved by the corresponding Hbase database under the Hadoop cloud framework, the Hbase database is used as a NoSQL database, the diversity characteristics of three-dimensional model data can be met, if the model exists texture mapping, some models do not exist (such as white mode), some models are formed by a plurality of components (such as models under BIM scene), the complex and various model data are stored by the NoSQL database, and compared with the direct file management mode, the database management has the characteristics of efficient management mechanism, quick access and the like, and the characteristics ensure that the platform-based application has strong interaction capability. The invention adopts a completely high-efficiency storage model, solves the problem that the current mainstream method only processes a certain model, and realizes the compatibility storage of various different three-dimensional models. The HBase database storage mechanism based on NoSQL can realize efficient and concurrent execution of the model in an online visual environment, and simultaneously meets the requirements of some special attributes of three-dimensional data.
(2) The concurrency is strong: cloud platforms are essentially a highly efficient distributed system, with co-processing across multiple servers, often far exceeding stand-alone systems. In fact, a large number of practical applications need to process a plurality of users to send requests online at the same time, but the sequential response mechanism of the traditional method can not meet the concurrent requests, especially for applications facing the mobile internet, and the browser has poor concurrent capability and low execution capability and can not provide efficient response service. The invention is based on the concurrent response request of the three-dimensional model, reduces the response time of the request model when a plurality of persons request the model data, improves the data interaction efficiency, can greatly meet the requirements of users, and improves the high-quality collaborative service request.
(3) The intelligence is strong: at present, software systems increasingly enter an intelligent era, virtual reality applications are not exceptional, especially applications facing mobile internet, the demand for intelligent is increasingly urgent, such as applications in smart cities, and it is very important to provide intelligent services. The platform is based on an implicit recommendation mechanism, and related rules are obtained by performing implicit analysis on user access logs. Meanwhile, the Spark completely places all data processing processes in the memory, so that the speed is greatly improved, and the processing efficiency is enhanced. The invention utilizes the efficient computing capability of the cloud platform, the platform can greatly improve the computing efficiency, and a plurality of users are supported to operate online. The new generation of cloud computing framework can completely provide concurrent processing capacity and a cooperative processing mechanism, so that cloud computing platform service with higher efficiency and stronger concurrency is realized.
Drawings
FIG. 1 is a schematic diagram of the present invention;
FIG. 2 is a schematic diagram of the structure information of the HBase-based TObject table and the TProperty table of the present invention;
FIG. 3 is a schematic diagram of the structural information of the HBase-based TParameter table and TResult table;
fig. 4 is a schematic diagram of an implicit recommendation process according to the present invention.
Detailed Description
The invention will now be described in detail with reference to the drawings and specific examples. The present embodiment is implemented on the premise of the technical scheme of the present invention, and a detailed implementation manner and a specific operation process are given, but the protection scope of the present invention is not limited to the following examples.
In virtual reality, management of a three-dimensional model is an important environment, but the existing management platform faces a plurality of challenges including poor concurrency, low efficiency of cooperative interactivity and the like, which seriously affect the construction of a virtual reality platform, such as a smart city and the like. The cloud platform has extremely high concurrency capability and a reliable storage mechanism, has achieved great success in environments such as big data storage, and a three-dimensional model is used as special big data, and has a plurality of special attributes including complex storage structure, rich geometric information, various auxiliary elements and the like.
As shown in fig. 1, the invention provides a mass three-dimensional model integration system under a cloud computing framework, which comprises a three-dimensional model storage module, a three-dimensional model access module and a three-dimensional model recommendation module, wherein the three-dimensional model storage module is used for storing a three-dimensional model based on a NoSQL database, the three-dimensional model access module is used for carrying out collaborative concurrent access on the stored three-dimensional model, and the three-dimensional model recommendation module is used for realizing implicit recommendation of the three-dimensional model based on a Spark cloud computing framework. The integrated platform has the following two functions: 1. the storage and access mechanism of the three-dimensional model based on the cloud platform can realize the concurrent and collaborative access mechanism of the mass model at the browser end, thereby realizing the efficient management of the mass model; 2. the model implicit recommendation service based on the Spark cloud computing framework is realized, so that an intelligent management mechanism is realized.
1. Massive three-dimensional model storage mechanism based on Hadoop and Hbase databases
Hadoop taking an HDFS file system as a core is used as a novel distributed storage mechanism, so that the Hadoop storage method is very suitable for storing big data, three-dimensional model big data have sparse attribute information and huge data scale, and the Hadoop storage method is very suitable for cloud architecture storage, and particularly suitable for the HDFS file system. HBase is a NoSQL database that does not rely on primary keys, where the entire row of data must be read per read, and the number of columns can be defined dynamically.
The data stored in the three-dimensional model storage module of the invention comprises three-dimensional model data, three-dimensional model auxiliary information, three-dimensional model keyword information and machine learning result information, as shown in fig. 2 and 3. The TObject table is used for storing three-dimensional model data, and in fact, since the three-dimensional model contains many different information including a map of the model, model materials, model thumbnails, simplified models, etc., the whole data can be acquired at one time each time when the data is read, so that all three-dimensional data information can be acquired. In practice, the information for each three-dimensional model is not exactly the same, as in the BIM model, the entire BIM model is made up of many components, and the model information for that row will contain more columns. The TProperty table contains three-dimensional model attachment information, including information such as the uploader of the model, the coordinates of the center point of the model surrounding the sphere, the name and class of the model, etc., which are all stored in the form of key-value pairs in the associated columns. The TParameter table is used for solving the conventional keyword-based retrieval service, and keywords of each different module are predefined in the table, so that related retrieval service can be conveniently provided. TResult is a table that stores machine learning results.
2. Implicit recommendation mechanism based on new generation cloud computing framework
The Spark framework is used as a new generation cloud computing framework after MapReduce, and has wide application in the fields of electronic commerce and the like. In fact, the recommendation of the three-dimensional model library is similar to the recommendation process of electronic commerce, the model itself can be regarded as a specific commodity, but since the user has no relevant preference choice for the model, namely, the user selects a certain model to view and can not indicate that the user has preference for the model, the collaborative filtering algorithm is displayed to be incapable of making relevant recommendation for the three-dimensional model. As shown in fig. 4, the three-dimensional model recommendation module of the present invention includes a data acquisition unit, a data analysis unit, a training unit and a training unit, wherein the data acquisition unit reads log file data of a three-dimensional model library through Spark and groups the data; the data analysis unit analyzes the read data to form statistical data; the training unit takes the statistical data as a learning sample for training; the prediction unit realizes implicit recommendation of the three-dimensional model based on the training result and outputs a prediction result. The recommended flow is specifically as follows:
first, the log file of the three-dimensional model library is read by Spark, and when a user accesses a model, such a record is stored in the log file (user a, model i), and Spark stores data as RDD (Resilient Distributed Datasets elastic distributed file), and at this time, the data is stored as a data set of individual key-value pairs ((user a, model i), …).
Second, the RDD is persisted as Data frames, which are more abstract than RDD and have specialized Data structures. And secondly, carrying out relevant analysis on the data, wherein the data Map operation and the Reduce operation are mainly carried out to form statistical data. The final statistics of key-value pairs are generated as (((user a, model i), 19), ((user a, model j), 52), …).
And thirdly, calling an ALS function in a spark.MLib module to train, wherein the key value pair formed in the last step is a sample to be learned, and the function expression is as follows:
（1）
wherein the parameters areIs a decision function, when +.>When (I)>Otherwise, let(s)>. Parameter->The confidence level is represented, and in the implicit collaborative recommendation method, the recommendation level is measured by the confidence level, not the score.Here->。/>Respectively represent matrix->Is the first of (2). Parameter->Is the regularization rate, which is a parameter to prevent overfitting of the optimization function。
Finally, the prediction function of the Spark ALS function is called, so that relevant prediction can be carried out on the user, and a hidden recommendation mechanism based on the Spark model is completed.
The mass three-dimensional model integrated system under the cloud computing framework is high in compatibility, can meet different model storage requirements, and provides various different model storage mechanisms; secondly, the platform is collaborative and concurrent, and meets the response service of the online requests of multiple people; finally, the platform is intelligent, namely, based on relevant log data, corresponding analysis is carried out, an intelligent recommendation service is provided, and the intelligent requirement of the platform can be met.
The foregoing describes in detail preferred embodiments of the present invention. It should be understood that numerous modifications and variations can be made in accordance with the concepts of the invention by one of ordinary skill in the art without undue burden. Therefore, all technical solutions which can be obtained by logic analysis, reasoning or limited experiments based on the prior art by the person skilled in the art according to the inventive concept shall be within the scope of protection defined by the claims.
Claims (6)
1. The mass three-dimensional model integration system under the cloud computing framework is characterized by comprising:
the three-dimensional model storage module is used for realizing the storage of a three-dimensional model based on a NoSQL database;
the three-dimensional model access module is used for carrying out collaborative concurrent access on the stored three-dimensional model, and reducing response time of a request model when a plurality of persons request model data;
the three-dimensional model recommendation module is used for realizing implicit recommendation of the three-dimensional model based on the Spark cloud computing framework;
the three-dimensional model recommendation module comprises:
the data acquisition unit reads log file data of the three-dimensional model library through Spark and groups the data;
the data analysis unit analyzes the read data to form statistical data;
the training unit is used for training by taking the statistical data as a learning sample;
and the prediction unit is used for realizing implicit recommendation of the three-dimensional model based on the training result and outputting a prediction result.
2. The mass three-dimensional model integration system under the cloud computing framework of claim 1, wherein the data stored by the three-dimensional model storage module comprises three-dimensional model data, three-dimensional model auxiliary information, three-dimensional model keyword information and machine learning result information.
3. The mass three-dimensional model integration system under a cloud computing framework of claim 2, wherein the three-dimensional model attachment information includes model uploaders, model bounding sphere center coordinates, and names and categories of models.
4. The mass three-dimensional model integration system under the cloud computing framework of claim 1, wherein the data acquisition unit stores the read data as an elastic distributed file.
5. The system of claim 4, wherein the parsing in the data parsing unit includes a persistence operation on an elastic distributed file and a data Map operation and a Reduce operation.
6. The mass three-dimensional model integration system under a cloud computing framework of claim 1, wherein the training unit trains ALS functions.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811590873.7A CN109783465B (en) | 2018-12-25 | 2018-12-25 | Mass three-dimensional model integration system under cloud computing framework |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811590873.7A CN109783465B (en) | 2018-12-25 | 2018-12-25 | Mass three-dimensional model integration system under cloud computing framework |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109783465A CN109783465A (en) | 2019-05-21 |
| CN109783465B true CN109783465B (en) | 2023-09-08 |
Family
ID=66498258
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811590873.7A Active CN109783465B (en) | 2018-12-25 | 2018-12-25 | Mass three-dimensional model integration system under cloud computing framework |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109783465B (en) |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103106680A (en) * | 2013-02-16 | 2013-05-15 | 赞奇科技发展有限公司 | Implementation method for three-dimensional figure render based on cloud computing framework and cloud service system |
| CN106056427A (en) * | 2016-05-25 | 2016-10-26 | 中南大学 | Spark-based big data hybrid model mobile recommending method |
| CN106126641A (en) * | 2016-06-24 | 2016-11-16 | 中国科学技术大学 | A kind of real-time recommendation system and method based on Spark |
| CN106296305A (en) * | 2016-08-23 | 2017-01-04 | 上海海事大学 | Electric business website real-time recommendation System and method under big data environment |
| CN107402958A (en) * | 2017-06-12 | 2017-11-28 | 重庆市勘测院 | A kind of establishment in GKF three-dimensional space datas storehouse and access method |
| CN107515952A (en) * | 2017-09-21 | 2017-12-26 | 北京星闪世图科技有限公司 | The method and its system of cloud data storage, parallel computation and real-time retrieval |
| CN108122153A (en) * | 2016-11-28 | 2018-06-05 | 宁波有哒云商务服务有限公司 | Personalized recommendation method based on cloud computing tupe under e-commerce environment |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP3032442B1 (en) * | 2014-12-08 | 2018-12-26 | Tata Consultancy Services Limited | Modeling and simulation of infrastructure architecture for big data |
| IN2015CH01424A (en) * | 2015-03-20 | 2015-04-10 | Wipro Ltd |
-
2018
- 2018-12-25 CN CN201811590873.7A patent/CN109783465B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103106680A (en) * | 2013-02-16 | 2013-05-15 | 赞奇科技发展有限公司 | Implementation method for three-dimensional figure render based on cloud computing framework and cloud service system |
| CN106056427A (en) * | 2016-05-25 | 2016-10-26 | 中南大学 | Spark-based big data hybrid model mobile recommending method |
| CN106126641A (en) * | 2016-06-24 | 2016-11-16 | 中国科学技术大学 | A kind of real-time recommendation system and method based on Spark |
| CN106296305A (en) * | 2016-08-23 | 2017-01-04 | 上海海事大学 | Electric business website real-time recommendation System and method under big data environment |
| CN108122153A (en) * | 2016-11-28 | 2018-06-05 | 宁波有哒云商务服务有限公司 | Personalized recommendation method based on cloud computing tupe under e-commerce environment |
| CN107402958A (en) * | 2017-06-12 | 2017-11-28 | 重庆市勘测院 | A kind of establishment in GKF three-dimensional space datas storehouse and access method |
| CN107515952A (en) * | 2017-09-21 | 2017-12-26 | 北京星闪世图科技有限公司 | The method and its system of cloud data storage, parallel computation and real-time retrieval |
Non-Patent Citations (1)
| Title |
|---|
| "城市三维建模数据库建库方案研究";王震等;《北京测绘》;20180125(第01期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109783465A (en) | 2019-05-21 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110990638B (en) | Large-scale data query acceleration device and method based on FPGA-CPU heterogeneous environment | |
| Han et al. | Survey on NoSQL database | |
| Kraska | Finding the needle in the big data systems haystack | |
| CN104239501B (en) | Mass video semantic annotation method based on Spark | |
| CN106897322B (en) | A kind of access method and device of database and file system | |
| CN103246749B (en) | The matrix database system and its querying method that Based on Distributed calculates | |
| CN106909644A (en) | A kind of multistage tissue and indexing means towards mass remote sensing image | |
| US20220222796A1 (en) | Image processing method and apparatus, server, and storage medium | |
| CN103440288A (en) | Big data storage method and device | |
| Wang et al. | Research and implementation on spatial data storage and operation based on Hadoop platform | |
| CN103699656A (en) | GPU-based mass-multimedia-data-oriented MapReduce platform | |
| Liu et al. | Cross-attentional spatio-temporal semantic graph networks for video question answering | |
| CN111078952B (en) | Cross-modal variable-length hash retrieval method based on hierarchical structure | |
| Luo et al. | Big-data analytics: challenges, key technologies and prospects | |
| Jing et al. | Improved U-Net model for remote sensing image classification method based on distributed storage | |
| Karode et al. | Performance analysis of trustworthy online review system using blockchain | |
| CN108319604B (en) | Optimization method for association of large and small tables in hive | |
| CN109783465B (en) | Mass three-dimensional model integration system under cloud computing framework | |
| Jiugen et al. | Cloud computing-based big data mining connotation and solution | |
| CN112580355B (en) | News information topic detection and real-time aggregation method | |
| Kong et al. | Application Research of Personalized Recommendation Technology in College English Teaching Reform under The Background of Big Data | |
| Zhao et al. | Impact of virtual reality technology on digital media in the context of big data and artificial intelligence | |
| Xu et al. | Visualization digital system of digital museum based on big data technology | |
| Liu | Visualized Analysis of Tourism Big Data based on Real-Time Analysis and Complexity Measurement | |
| Chen | Construction of Ideological and Political Teaching Resource Integration Platform Based on Big Data |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20191119 Address after: Room 804, block a, Jilin animation and game original industrial park, 2888 Silicon Valley Street, Changchun hi tech Industrial Development Zone, 130000 Jilin Province Applicant after: Changchun Samai Animation Design Co.,Ltd. Address before: 200092 Shanghai City, Yangpu District Siping Road No. 1239 Applicant before: Tongji University |
|
| TA01 | Transfer of patent application right | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20200617 Address after: 130012 Jilin province city Changchun well-informed high tech Industrial Development Zone, Road No. 168 Applicant after: JILIN ANIMATION INSTITUTE Applicant after: Jilin Jidong Pangu Network Technology Co.,Ltd. Address before: Room 804, block a, Jilin animation and game original industrial park, 2888 Silicon Valley Street, Changchun hi tech Industrial Development Zone, 130000 Jilin Province Applicant before: Changchun Samai Animation Design Co.,Ltd. |
|
| TA01 | Transfer of patent application right | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |