CN101957968A - Online transaction service aggregation method based on Hadoop - Google Patents
Online transaction service aggregation method based on Hadoop Download PDFInfo
- Publication number
- CN101957968A CN101957968A CN2010102703056A CN201010270305A CN101957968A CN 101957968 A CN101957968 A CN 101957968A CN 2010102703056 A CN2010102703056 A CN 2010102703056A CN 201010270305 A CN201010270305 A CN 201010270305A CN 101957968 A CN101957968 A CN 101957968A
- Authority
- CN
- China
- Prior art keywords
- service
- module
- hadoop
- polymerization
- web api
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Landscapes
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The invention discloses an online transaction service aggregation method based on a Hadoop. The method comprises the following steps of: submitting a Web API (Application Programming Interface) sorting module, an online transaction service discovery module, an online transaction service aggregation module, a personalized data mining and analyzing module and an aggregation service personalized recommendation module as computation tasks of the Hadoop to a platform, parallelizing the computation-intensive tasks therein, and shortening task execution time. By utilizing a Web API with higher scores, the discovery of an electronic commerce service is realized, a large number of services with the same or similar functions are aggregated into different levels of aggregation services, therefore, the access efficiency of an electronic commerce site can be enhanced, more visitors are attracted, and the demand for electronic commerce service aggregation is satisfied.
Description
Technical field
The present invention relates to a kind of method of commerce, specifically a kind of on-net transactions polymerization based on Hadoop.
Background technology
Along with Internet fast development with popularize, quick, safety, characteristics become all generally accepted mode of doing business of consumer and businessman for ecommerce sustainable development at home and abroad provides advantage cheaply.The recent statistics report that CNNIC announces shows that China On Line transaction userbase reached 1.08 hundred million people in 2009, and trading volume reaches 2,500 hundred million.Though ecommerce has obtained development fast in the whole world, various e-commerce websites occur in succession, and Internet user's number sharply increases, and increasing netizen is also adapting to shopping online and consumption gradually.The click number that shopping website and online retail shop obtain the consumer rises significantly, and still, what wherein really can transform into effective purchase does not but make us optimistic.At first to understand the information of the own commodity of purchasing when in fact the user does shopping by network, mainly comprise the evaluation of the price, quality, manufacturer, user of performance, specification, the product of commodity, the evaluation to the website of the prestige, user of on the net bandwagon effect of the website of the commodity of purchasing, the service that provides, product, mode of doing business, transaction security, response speed, logistics distribution, maintenance, website then is provided this producer and products thereof.Simultaneously, participant's behavior is not fully rational in the e-commerce environment, and there are certain deviation in its behavior and cognition, in the cognition with behavior on feature make the participant colony of E-commerce market form the problem of a high complexity.
How being the user finds useful data in the electronic commerce information of magnanimity, and the individual demand that the user is satisfied in the existing service of polymerization becomes a challenging problem.Although can be used for finding electronic commerce data and service as universal search engines such as Google, Baidu, Yahoo, but, these universal search engines only provide the Web address based on key word, and the user need visit these Web addresses one by one could find required electronic commerce data and service.And universal search engine does not provide a kind of and specially comes assisting users to serve at the ordering mechanism of E-business service to choose, also fail to consider QoS (the Quality of Service) attribute of E-business service, therefore, utilize universal search engine to carry out service discovery and choose not only lacking flexibility and dirigibility, and can not satisfy user's individual demand.This just needs to seek a kind of new method provides the adaptive service to satisfy the demand of user individual easily, the service aggregating (Service Mashup) that proposes along with the rise of Web 2.0 technology is exactly a kind of Perfected process that the adaptive service is provided, service aggregating can be collected data and the service with integrated a plurality of services source, utilizes the lightweight programming technique to create various application fast.The adaptive service aggregating is meant the service of initiatively recommending specific function and different brackets according to factors such as user's interest and demands, and people are its individual demand of easier embodiment when shopping, thereby the adaptive service aggregating seems particularly important in e-commerce field.Research in e-commerce field in the past concentrates on aspects such as ecommerce agreement, raising E-business service performance more, the research of E-business service polymerization but rarely has the people to pay close attention to, yet, the E-business service polymerization but can improve the access efficiency of e-commerce site, attracts more visitor.Simultaneously, traditional integrated system can not satisfy the demand of E-business service polymerization, and it is imperative to make up the E-business service aggregation platform by means of distributed computing technology.Be E-business service polymerization based on the on-net transactions polymerization of Hadoop based on Hadoop.Hadoop be one can easier exploitation and operation handle the software platform of large-scale data, as up-to-date distributed computing technology-Hadoop technology, for adaptive E-business service polymerization studies provides platform.
Summary of the invention
The purpose of this invention is to provide a kind of on-net transactions polymerization based on Hadoop, this method is chosen the representational service of e-commerce field, utilizes instruments such as WordNet to extract its text feature, sets up the semantic base of e-commerce field.Based on the ecommerce semantic base, the Web API that is obtained from service sources (Web) such as ProgrammableWeb, Amazon, Google, Baidu, Yahoo is mated, Web API is divided in the specific classification, utilize the higher Web API of mark to realize the discovery of E-business service, the service aggregating that has same or similar function is in a large number become the different brackets aggregated service.The present invention can improve the access efficiency of e-commerce site, attracts more visitor, satisfies the demand of E-business service polymerization.
The objective of the invention is to be achieved through the following technical solutions:
A kind of on-net transactions polymerization based on Hadoop, it is characterized in that: this method with Web API classification and ordination module, on-net transactions find module, on-net transactions polymerization module, individuation data excavation and analysis module, and aggregated service personalized recommendation module be submitted to platform as the calculation task of Hadoop, for computation-intensive task realization parallelization wherein, shorten task execution time, concrete workflow is as follows:
A), the Web API classification and ordination module API that will obtain from the service source classifies according to the Hadoop algorithm and sorts;
B), E-business service finds classification and the ranking results of module according to Web API, chooses the high Web API of correlativity and importance, and the Web API that chooses is sent on the service source, utilizes these Web API retrieval on-net transactions;
C1), on-net transactions polymerization module obtains the information on services on the service source, and according to functional character and service aggregating;
C2), individuation data excavates and analysis module obtains data message on the service source, and the data during service source socialization used are excavated and are analyzed;
D), aggregated service personalized recommendation module is according to the result that the result of service aggregating and individuation data excavate and analyze, and makes personalized recommendation according to the Hadoop algorithm to the user.
Service of the present invention source comprises ProgrammableWeb, Amazon, Google, Baidu, the Yahoo on the Internet.The service aggregating that on-net transactions polymerization module will have same or similar function in a large number becomes the different brackets aggregated service.E-business service is found data mining algorithm and the technology that module utilizes cluster, similarity to calculate, sort and recommend to calculate, extract and analyze implicit a large number of users customized information in the various socialization Web application, select a kind of basic vector distance function to measure the similarity of these services.
The present invention utilizes the higher Web API of mark to realize the discovery of E-business service, the service aggregating that has same or similar function is in a large number become the different brackets aggregated service, can improve the access efficiency of e-commerce site, attract more visitor, satisfy the demand of E-business service polymerization.
Description of drawings
Fig. 1 is a formation frame diagram of the present invention.
Embodiment
A kind of on-net transactions polymerization based on Hadoop of the present invention, see Fig. 1, this method with Web API classification and ordination module, on-net transactions find module, on-net transactions polymerization module, individuation data excavation and analysis module, and aggregated service personalized recommendation module be submitted to platform as the calculation task of Hadoop, utilize the MapReduce programming framework to realize parallelization for computation-intensive task wherein, to shorten task execution time.Concrete workflow is as follows:
A), the Web API classification and ordination module API that will obtain from the service source classifies according to the Hadoop algorithm and sorts; The service source comprises ProgrammableWeb, Amazon, Google, Baidu, the Yahoo on the Internet.
B), E-business service finds classification and the ranking results of module according to Web API, chooses the high Web API of correlativity and importance, and the Web API that chooses is sent on the service source, utilizes these Web API retrieval on-net transactions;
C1), on-net transactions polymerization module obtains the information on services on the service source, and according to functional character and service aggregating;
C2), individuation data excavates and analysis module obtains data message on the service source, and the data during service source socialization used are excavated and are analyzed;
D), aggregated service personalized recommendation module is according to the result that the result of service aggregating and individuation data excavate and analyze, and makes personalized recommendation according to the Hadoop algorithm to the user.
When the personalized recommendation of E-business service, analyse in depth the contact between user capture service and the Web service object, utilize text vector statement Web service object, thereby set up the mathematical model of describing user access pattern and Web service functional objective.On basis based on the mathematical model of vector space, utilize data mining algorithm and technology such as cluster, similarity calculating, ordering and recommendation calculating, extract and analyze implicit a large number of users customized information in the various socialization Web application, select a kind of basic vector distance function to measure the similarity of these services, such as the most frequently used cosine function.By the distance function of definition, we can utilize clustering method to obtain different user access patterns.Calculate and user clustering based on similarity, propose to recommend strategy based on the hybrid electronic business service of coordinating (collaborative filtering) technology of filtering.
Analyse in depth the algorithmic procedure and the data flow of these application, formulate the required parameter of Hadoop framework, as input parameter, Map and Reduce function, task granularity etc., broken needle is not finished the E-business service polymerization and is realized the performance of concrete optimizing application Hadoop.
The present invention utilizes the higher Web API of mark to realize the discovery of E-business service, the service aggregating that has same or similar function is in a large number become the different brackets aggregated service, can improve the access efficiency of e-commerce site, attract more visitor, satisfy the demand of E-business service polymerization.
Claims (4)
1. on-net transactions polymerization based on Hadoop, it is characterized in that: this method with WebAPI classification and ordination module, on-net transactions find module, on-net transactions polymerization module, individuation data excavation and analysis module, and aggregated service personalized recommendation module be submitted to platform as the calculation task of Hadoop, for computation-intensive task realization parallelization wherein, shorten task execution time, concrete workflow is as follows:
A), the Web API classification and ordination module API that will obtain from the service source classifies according to the Hadoop algorithm and sorts;
B), E-business service finds classification and the ranking results of module according to Web API, chooses the high Web API of correlativity and importance, and the Web API that chooses is sent on the service source, utilizes these Web API retrieval on-net transactions;
C1), on-net transactions polymerization module obtains the information on services on the service source, and according to functional character and service aggregating;
C2), individuation data excavates and analysis module obtains data message on the service source, and the data during service source socialization used are excavated and are analyzed;
D), aggregated service personalized recommendation module is according to the result that the result of service aggregating and individuation data excavate and analyze, and makes personalized recommendation according to the Hadoop algorithm to the user.
2. the on-net transactions polymerization based on Hadoop according to claim 1 is characterized in that: described service source comprises ProgrammableWeb, Amazon, Google, Baidu, Yahoo.
3. the on-net transactions polymerization based on Hadoop according to claim 1 is characterized in that: the service aggregating that on-net transactions polymerization module will have same or similar function in a large number becomes the different brackets aggregated service.
4. the on-net transactions polymerization based on Hadoop according to claim 1, it is characterized in that: E-business service is found data mining algorithm and the technology that module utilizes cluster, similarity to calculate, sort and recommend to calculate, extract and analyze implicit a large number of users customized information in the various socialization Web application, select a kind of basic vector distance function to measure the similarity of these services.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2010102703056A CN101957968A (en) | 2010-08-31 | 2010-08-31 | Online transaction service aggregation method based on Hadoop |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2010102703056A CN101957968A (en) | 2010-08-31 | 2010-08-31 | Online transaction service aggregation method based on Hadoop |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN101957968A true CN101957968A (en) | 2011-01-26 |
Family
ID=43485287
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN2010102703056A Pending CN101957968A (en) | 2010-08-31 | 2010-08-31 | Online transaction service aggregation method based on Hadoop |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN101957968A (en) |
Cited By (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102866911A (en) * | 2012-09-12 | 2013-01-09 | 北京航空航天大学 | Mashup application establishing method and device |
| CN102880503A (en) * | 2012-08-24 | 2013-01-16 | 新浪网技术(中国)有限公司 | Data analysis system and data analysis method |
| CN103425707A (en) * | 2012-05-25 | 2013-12-04 | 中兴通讯股份有限公司 | Data analyzing method and data analyzing device |
| CN103577403A (en) * | 2012-07-19 | 2014-02-12 | 镇江雅迅软件有限责任公司 | Cloud computing technology based recommendation system implementation method |
| CN103605718A (en) * | 2013-11-15 | 2014-02-26 | 南京大学 | Hadoop improvement based goods recommendation method |
| CN105354327A (en) * | 2015-11-26 | 2016-02-24 | 中山大学 | Interface API recommendation method and system based on massive data analysis |
| CN106503140A (en) * | 2016-10-20 | 2017-03-15 | 安徽大学 | One kind is based on Hadoop cloud platform web resource personalized recommendation system and method |
| CN107566495A (en) * | 2017-09-06 | 2018-01-09 | 国云科技股份有限公司 | A kind of chance distribution method based on micro services |
| CN109672626A (en) * | 2019-01-09 | 2019-04-23 | 中南大学 | A kind of service aggregating method utilized based on queueing delay |
| CN110096553A (en) * | 2019-03-28 | 2019-08-06 | 北京华成智云软件股份有限公司 | A kind of the big data analysis system and analysis method of integration across database |
-
2010
- 2010-08-31 CN CN2010102703056A patent/CN101957968A/en active Pending
Cited By (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103425707A (en) * | 2012-05-25 | 2013-12-04 | 中兴通讯股份有限公司 | Data analyzing method and data analyzing device |
| CN103577403A (en) * | 2012-07-19 | 2014-02-12 | 镇江雅迅软件有限责任公司 | Cloud computing technology based recommendation system implementation method |
| CN102880503B (en) * | 2012-08-24 | 2015-04-15 | 新浪网技术(中国)有限公司 | Data analysis system and data analysis method |
| CN102880503A (en) * | 2012-08-24 | 2013-01-16 | 新浪网技术(中国)有限公司 | Data analysis system and data analysis method |
| CN102866911B (en) * | 2012-09-12 | 2015-03-25 | 北京航空航天大学 | Mashup application establishing method and device |
| CN102866911A (en) * | 2012-09-12 | 2013-01-09 | 北京航空航天大学 | Mashup application establishing method and device |
| CN103605718A (en) * | 2013-11-15 | 2014-02-26 | 南京大学 | Hadoop improvement based goods recommendation method |
| CN105354327A (en) * | 2015-11-26 | 2016-02-24 | 中山大学 | Interface API recommendation method and system based on massive data analysis |
| CN106503140A (en) * | 2016-10-20 | 2017-03-15 | 安徽大学 | One kind is based on Hadoop cloud platform web resource personalized recommendation system and method |
| CN107566495A (en) * | 2017-09-06 | 2018-01-09 | 国云科技股份有限公司 | A kind of chance distribution method based on micro services |
| CN109672626A (en) * | 2019-01-09 | 2019-04-23 | 中南大学 | A kind of service aggregating method utilized based on queueing delay |
| CN110096553A (en) * | 2019-03-28 | 2019-08-06 | 北京华成智云软件股份有限公司 | A kind of the big data analysis system and analysis method of integration across database |
| CN110096553B (en) * | 2019-03-28 | 2021-05-18 | 北京华成智云软件股份有限公司 | Cross-database big data analysis system and analysis method |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN101957968A (en) | Online transaction service aggregation method based on Hadoop | |
| WO2021025926A1 (en) | Digital content prioritization to accelerate hyper-targeting | |
| Xin et al. | Using multi-features to partition users for friends recommendation in location based social network | |
| Dong et al. | Self-adaptive semantic focused crawler for mining services information discovery | |
| Dhingra et al. | Spam analysis of big reviews dataset using Fuzzy Ranking Evaluation Algorithm and Hadoop | |
| CN104866969A (en) | Personal credit data processing method and device | |
| Goel et al. | Discovering similar users on twitter | |
| US9269112B1 (en) | Integrating location-based social media data with enterprise business intelligence applications | |
| Dragoni | A three-phase approach for exploiting opinion mining in computational advertising | |
| Ouyang et al. | CompetitiveBike: Competitive analysis and popularity prediction of bike-sharing apps using multi-source data | |
| Liang et al. | RETRACTED ARTICLE: Research on consumers online shopping decision-making and recommendation of commodity based on social media network | |
| Xu et al. | Novel model of e-commerce marketing based on big data analysis and processing | |
| Ennaji et al. | Social intelligence framework: Extracting and analyzing opinions for social CRM | |
| Goel et al. | Folksonomy-based user profile enrichment using clustering and community recommended tags in multiple levels | |
| CN118071400A (en) | Application method and system based on graph computing technology in information consumption field | |
| Li et al. | Big data gathering and mining pipelines for CRM using open-source | |
| Balaraman et al. | E-commerce trends and future analytics tools | |
| Zhang | Intelligent recommendation model of contemporary pop music based on knowledge map | |
| Su | Accurate Marketing Algorithm of Network Video Based on User Big Data Analysis | |
| Kumar et al. | Web Mining and Web Usage Mining for Various Human-Driven Applications | |
| Sun | Music Individualization Recommendation System Based on Big Data Analysis | |
| Wei et al. | A study of customer behaviour through web mining | |
| Sheriff | Big data revolution: Is it a business disruption? | |
| Ding et al. | [Retracted] Clustering Merchants and Accurate Marketing of Products Using the Segmentation Tree Vector Space Model | |
| Zhu et al. | Tackling data sparseness in recommendation using social media based topic hierarchy modeling |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C12 | Rejection of a patent application after its publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20110126 |