CN107154984A - The distributed control method of shops Saas systems on Novel wire - Google Patents
The distributed control method of shops Saas systems on Novel wire Download PDFInfo
- Publication number
- CN107154984A CN107154984A CN201710532345.5A CN201710532345A CN107154984A CN 107154984 A CN107154984 A CN 107154984A CN 201710532345 A CN201710532345 A CN 201710532345A CN 107154984 A CN107154984 A CN 107154984A
- Authority
- CN
- China
- Prior art keywords
- data
- shops
- redis
- data cached
- control method
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 17
- 238000012546 transfer Methods 0.000 claims abstract description 7
- 230000004044 response Effects 0.000 claims abstract description 4
- 230000006870 function Effects 0.000 claims description 12
- 238000013507 mapping Methods 0.000 claims description 8
- 238000004891 communication Methods 0.000 claims description 3
- 230000001934 delay Effects 0.000 claims 1
- 238000004364 calculation method Methods 0.000 description 5
- 238000005516 engineering process Methods 0.000 description 3
- 241001269238 Data Species 0.000 description 2
- 230000008901 benefit Effects 0.000 description 2
- 230000009467 reduction Effects 0.000 description 2
- 238000013500 data storage Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 239000000686 essence Substances 0.000 description 1
- 238000013467 fragmentation Methods 0.000 description 1
- 238000006062 fragmentation reaction Methods 0.000 description 1
- 238000007726 management method Methods 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000008520 organization Effects 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
- 210000004291 uterus Anatomy 0.000 description 1
- 239000002699 waste material Substances 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1001—Protocols in which an application is distributed across nodes in the network for accessing one among a plurality of replicated servers
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1097—Protocols in which an application is distributed across nodes in the network for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/56—Provisioning of proxy services
- H04L67/568—Storing data temporarily at an intermediate stage, e.g. caching
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Information Transfer Between Computers (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a kind of distributed control method of shops's Saas systems on Novel wire, comprise the following steps：S1, customer in response request；S2, is asked according to the client, is judged that requested shops is data cached with the presence or absence of corresponding program level, is, then no into step S6, then into step S3；S3, the requested shops of judgement is data cached with the presence or absence of redis, is, then the program level for the data cached deposits of the redis being requested into shops is data cached and into step S6, no, then into step S4；S4, obtains data and according to the shop individuation data of the data cached middle acquisitions of the redis from requested shops from total Database, transfers to corresponding service to carry out computing data and draw final data；S5, the redis that the final data is stored in into requested shops is data cached and program level is data cached, and enters step S6；And S6, return to request data to client.
Description
Technical field
The present invention relates to a kind of distributed control of shops's Saas systems in technical field of electronic commerce, more particularly to Novel wire
Method processed.
Background technology
Xian Shang shops Saas systems can easily provide the user with shop displaying and merchandise sales service.On existing line
Shops's Saas systems are low for the displaying efficiency of shop content, merchandise news, marketing activity, and the data in cervical orifice of uterus shop are all independently deposited
Storage, shared data can not be shared and repeat to store, and lack hierarchical cache mechanism.
When establishing substantial amounts of shop in system, and user's visit capacity it is larger when, system can be caused to occur following
Problem：
1st, the storage of system mass data, comprising mass of redundancy data, takes memory space and computing resource so that service
Device system resources consumption is big, reduces the performance of system.
2nd, the separate storage of a large amount of repeated datas, storage organization is complicated, and the recall precision to data is low, is consumed when reading data
Duration, also causes the waste of calculation resources.
3rd, data can not be effectively synchronous between many shops, and such as stock, identical merchandise news cause bulk redundancy number
According to generation, it is impossible to allow the shared of corporate data, reduce the performance and stability of system.
The content of the invention
It is an object of the invention to the shortcoming for overcoming prior art, there is provided a kind of distribution of shops's Saas systems on Novel wire
Formula control method.
The distributed control method of shops Saas systems, comprises the following steps on a kind of Novel wire：
S1, customer in response request；
S2, is asked according to the client, is judged that requested shops is data cached with the presence or absence of corresponding program level, is, then
It is no into step S6, then into step S3；
S3, judges that requested shops is data cached with the presence or absence of redis, is, then by the data cached deposit quilts of the redis
Ask the program level of shops data cached and into step S6, it is no, then into step S4；
S4, data are obtained and according to the shop of the data cached middle acquisitions of the redis from requested shops from total Database
Property data, transfer to corresponding service to carry out computing data and draw final data；
S5, the redis that the final data is stored in into requested shops is data cached and program level is data cached, and enters
Step S6；
S6, request data is returned to client.
It is used as further improved, the data cached and described redis of described program level data cached delaying for json forms
Deposit data.
It is used as further improved, data cached use hash, set or key-value storage side of the json forms
What formula was hashed is stored in redis buffer services and local cache service.
It is described to transfer to corresponding service to carry out the step of computing draws final data data to wrap as further improved
Include：It will ask to be distributed to distributed Web Server progress computing according to IPHash algorithms using Nginx load balancing modes
Draw final data.
As further improved, the distributed Web Server carry out the step of computing draws final data and wrapped
Include：The distributed Web Server draw final data using Map-Reduce computings.
As further improved, the Map-Reduce computings comprise the following steps：
Data, mapping key, the minimum unit of value forms are decomposed by Mapping functions by S41；
S42, carries out repeat key merging by the mapping key-value pairs gone out by Combine functions, is opened with reducing data communication
Pin；
S43, is grouped the data of different nodes according to key by Partitioner functions；And
S44, is received by Reducing functions, and the data after packet is finally calculated, returning result.
Compared with prior art, the present invention has advantages below：
First, according to system data burst, by public and sharable data storage in total Database, it substantially reduce the number and be
The occupancy for storage resource of uniting, and the reduction of data volume, then the when reduction retrieval of data acquisition take resource and system I/O operations etc.
The occupancy of resource.
Second, the shop of the public data from total data fast cache to storekeeper, shop such as commodity when store system is run a shop
Main shop is created immediately, and another storekeeper is directed to shop individual cultivation according to oneself marketing mode, and the Data Data is obtained by computing
The data finally shown to shop, so as to obtain the data cached of json forms, using hash, set, key-value etc. is a variety of
Storage mode, is stored in redis buffer services and local cache service, the wherein use of public data is reduced by data hash
The consumption of data retrieval, service amount of calculation is also largely reduced, and realizes the quick-searching to data.
Third, using Nginx load balancing modes, Nginx will ask to be distributed to distributed Web according to IPHash algorithms
Server, the problem of so solving single Web Server project performance bottlenecks, the robustness of the high concurrent access of lifting system.
Fourth, using MapReduce Distributed Calculations in system, a large amount of storekeeper's data are subjected to data fragmentation, are dispersed in
On each node, facilitate the calculating of MapReduce programs.Realize that big data parallel processing is calculated by the technology, solve and be
The requirement united to big data computing.
Program level caching five, is added, and program level caching can have multigroup to realize dilatation in groups, in burst focus
Under the business scenario for causing flow abruptly increase, the extending transversely of caching can be achieved.
Brief description of the drawings
Accompanying drawing 1 is the flow chart of the distributed control method of shops Saas systems on Novel wire.
Accompanying drawing 2 is the structural representation of multi-level buffer in the distributed control method of shops Saas systems on Novel wire.
Accompanying drawing 3 is obtained most using Map-Reduce computings in the distributed control method of shops Saas systems on Novel wire
The flow chart of whole data.
Embodiment
To make the purpose, technical scheme and advantage of embodiment of the present invention clearer, below in conjunction with present invention implementation
Accompanying drawing in mode, the technical scheme in embodiment of the present invention is clearly and completely described, it is clear that described reality
The mode of applying is a part of embodiment of the invention, rather than whole embodiments.Based on the embodiment in the present invention, ability
The every other embodiment that domain those of ordinary skill is obtained under the premise of creative work is not made, belongs to the present invention
The scope of protection.Therefore, the detailed description of embodiments of the present invention below to providing in the accompanying drawings, which is not intended to limit, wants
The scope of the present invention of protection is sought, but is merely representative of the selected embodiment of the present invention.Based on the embodiment in the present invention,
The every other embodiment that those of ordinary skill in the art are obtained under the premise of creative work is not made, belongs to this
Invent the scope of protection.
The present invention is described in further detail with embodiment below in conjunction with the accompanying drawings.
The distributed control method of shops Saas systems, comprises the following steps on a kind of Novel wire：
S1, customer in response request；
S2, is asked according to the client, is judged that requested shops is data cached with the presence or absence of corresponding program level, is, then
It is no into step S6, then into step S3；
S3, judges that requested shops is data cached with the presence or absence of redis, is, then by the data cached deposit quilts of the redis
Ask the program level of shops data cached and into step S6, it is no, then into step S4；
S4, data are obtained and according to the shop of the data cached middle acquisitions of the redis from requested shops from total Database
Property data, transfer to corresponding service to carry out computing data and draw final data；
S5, the redis that the final data is stored in into requested shops is data cached and program level is data cached, and enters
Step S6；
S6, request data is returned to client.
Most systems business scenario is met comprising master and slave structure deployment buffer service, but is deposited advertising campaign is this
Need caching extending transversely in the case where burst focus causes the business scenario of flow abruptly increase.Fig. 2 is refer to, in order to solve horizontal expansion
The problem of exhibition, program level caching (Cache) is added, multi-level buffer is realized, program level Cache capacity is generally less than
Redis buffer memory capacity, that is to say, that the data temperature in program level Cache is higher；Program level Cache can have many simultaneously
Group is, it is necessary to when extending transversely, it is only necessary to dilatation program level Cache in groups.
The public datas such as the commodity when store system is run a shop, shop that can be from total data fast cache to storekeeper, shop
Main shop is created immediately, and another storekeeper is directed to shop individual cultivation according to oneself marketing mode, and the Data Data is obtained by computing
The data finally shown to shop, so as to obtain the data cached of json forms, using hash, set, key-value etc. is a variety of
Storage mode, is stored in redis buffer services and local cache service, the wherein use of public data is reduced by data hash
The consumption of data retrieval, service amount of calculation is also largely reduced, and realizes the quick-searching to data.
When user accesses, system respond request returns to the data of needs, program level buffer memory in being cached from program level
Be the very hot point data of system.If now the data of needs can not be found in program level caching, request is submitted to by system
Redis buffer services, obtain data in redis principals and subordinates service and return., will be from total Database if can not still get data
Obtain data and shop individuation data is obtained from redis, transfer to corresponding service to carry out computing data and draw final number
According to, by data buffer storage to redis caching and program level cache, finally return to user request data.
It is described to transfer to corresponding service to carry out the step of computing draws final data data to include：Loaded using Nginx
Balanced way, which will ask to be distributed to distributed Web Server according to IPHash algorithms, to carry out computing and draws final data.
Fig. 3 is refer to, as further improved, the distributed Web Server carry out computing and draw final data
The step of using Map-Reduce computings carry out.Wherein, the Map-Reduce computings comprise the following steps：
S41, decomposes data by Mapping functions and is mapped to key, the minimum unit of value forms；
S42, carries out repeat key merging by the mapping key-value pairs gone out by Combine functions, is opened with reducing data communication
Pin, in this step, also relative to a precomputation part is carried out in advance, accelerates overall calculating speed；
S43, is grouped the data of different nodes according to key by Partitioner functions；And
S44, is received by Reducing functions, and the data after packet is finally calculated, returning result.
System Distributed Calculation is realized using Map-Reduce technologies, the management computing of shop big data is realized, by sum
Calculated according to storehouse and individuation data and obtain the data that shop displaying needs, and the focus that data in data is accessed is classified and stored.
When shop is run a shop, what is recorded in total Database predominantly shares shared data, and the data are main to be used because all shops are shared, number
According to also redis and program level caching is stored in by hot-spot mode, when storekeeper accesses, data directly can obtain calculating from caching and obtain
As a result, the data result in caching or is directly returned into storekeeper.
The foregoing is merely illustrative of the preferred embodiments of the present invention, is not intended to limit the invention, all essences in the present invention
God is with principle, and any modification, equivalent substitution and improvements done etc. should be included within the scope of protection of the invention.
Claims (7)
1. the distributed control method of shops Saas systems on a kind of Novel wire, it is characterised in that comprise the following steps：
S1, customer in response request；
S2, is asked according to the client, is judged that requested shops is data cached with the presence or absence of corresponding program level, is then to enter
Step S6, it is no, then into step S3；
S3, judges that requested shops is data cached with the presence or absence of redis, is, then the data cached deposits of the redis is requested
The program level of shops is data cached and into step S6, no, then into step S4；
S4, data are obtained and personalized according to the shop of the data cached middle acquisitions of the redis from requested shops from total Database
Data are transferred to corresponding service to carry out computing and draw final data by data；
S5, the redis that the final data is stored in into requested shops is data cached and program level is data cached, and enters step
S6；And
S6, request data is returned to client.
2. control method according to claim 1, it is characterised in that：The data cached and described redis of described program level delays
Deposit data is data cached for json forms.
3. control method according to claim 1, it is characterised in that：Described program level is data cached with multigroup.
4. control method according to claim 2, it is characterised in that：The data cached use hash of the json forms,
What set or key-value storage modes were hashed is stored in redis buffer services and local cache service.
5. control method according to claim 1, it is characterised in that：It is described to transfer to corresponding service to carry out computing data
The step of drawing final data includes：It will ask to be distributed to distribution according to IPHash algorithms using Nginx load balancing modes
Web Server carry out computing draw final data.
6. control method according to claim 5, it is characterised in that：The distributed Web Server carry out computing and obtained
The step of going out final data includes：The distributed Web Server draw final data using Map-Reduce computings.
7. control method according to claim 6, it is characterised in that：The Map-Reduce computings comprise the following steps：
Data, mapping key, the minimum unit of value forms are decomposed by Mapping functions by S41；
S42, carries out repeat key merging, to reduce data communication expense by Combine functions by the mapping key-value pairs gone out；
S43, is grouped the data of different nodes according to key by Partitioner functions；And
S44, is received by Reducing functions, and the data after packet is finally calculated, returning result.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710532345.5A CN107154984A (en) | 2017-07-03 | 2017-07-03 | The distributed control method of shops Saas systems on Novel wire |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710532345.5A CN107154984A (en) | 2017-07-03 | 2017-07-03 | The distributed control method of shops Saas systems on Novel wire |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN107154984A true CN107154984A (en) | 2017-09-12 |
Family
ID=59795182
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710532345.5A Pending CN107154984A (en) | 2017-07-03 | 2017-07-03 | The distributed control method of shops Saas systems on Novel wire |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107154984A (en) |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106528448A (en) * | 2016-10-11 | 2017-03-22 | 杭州数强网络科技有限公司 | Distributed caching mechanism for multi-source heterogeneous electronic commerce big data |
| CN106777085A (en) * | 2016-12-13 | 2017-05-31 | 东软集团股份有限公司 | A kind of data processing method, device and data query system |
-
2017
- 2017-07-03 CN CN201710532345.5A patent/CN107154984A/en active Pending
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106528448A (en) * | 2016-10-11 | 2017-03-22 | 杭州数强网络科技有限公司 | Distributed caching mechanism for multi-source heterogeneous electronic commerce big data |
| CN106777085A (en) * | 2016-12-13 | 2017-05-31 | 东软集团股份有限公司 | A kind of data processing method, device and data query system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11349940B2 (en) | Server side data cache system | |
| CN103380421B (en) | Distributed caching for graph data | |
| CN104508639B (en) | Use the coherency management of coherency domains table | |
| CN109240946A (en) | The multi-level buffer method and terminal device of data | |
| CN103067297B (en) | A kind of dynamic load balancing method based on resource consumption prediction and device | |
| Baker et al. | Distributed cooperative Web servers | |
| EP3443471B1 (en) | Systems and methods for managing databases | |
| CN105450579B (en) | Internet resources pre-add support method, client and middleware server | |
| CN110110006A (en) | Data managing method and Related product | |
| CN103312624A (en) | Message queue service system and method | |
| CN105354250A (en) | Data storage method and device for cloud storage | |
| EP3049940B1 (en) | Data caching policy in multiple tenant enterprise resource planning system | |
| CN104270412A (en) | Three-level caching method based on Hadoop distributed file system | |
| CN106446099A (en) | Distributed cloud storage method and system and uploading and downloading method thereof | |
| CN109376318A (en) | A kind of page loading method, computer readable storage medium and terminal device | |
| CN101916289A (en) | Method for establishing digital library storage system supporting mass small files and dynamic backup number | |
| CN103297490B (en) | Information processing apparatus, distributed processing system, and distributed processing method | |
| CN108259603A (en) | A kind of load-balancing method and device | |
| CN109933585A (en) | Data query method and data query system | |
| CN104424325A (en) | Data inquiry method and device | |
| Tenzakhti et al. | Replication algorithms for the world-wide web | |
| CN110109931A (en) | It is a kind of for preventing the method and system that data access clashes between RAC example | |
| CN110119396A (en) | Data managing method and Related product | |
| CN110196680A (en) | Data processing method, device and storage medium | |
| CN110020373A (en) | The method and apparatus that static page is stored, browsed |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20170912 |