CN107465574A - Internet site group plateform system and its parallel isolation streaming computational methods - Google Patents
Internet site group plateform system and its parallel isolation streaming computational methods Download PDFInfo
- Publication number
- CN107465574A CN107465574A CN201710666875.9A CN201710666875A CN107465574A CN 107465574 A CN107465574 A CN 107465574A CN 201710666875 A CN201710666875 A CN 201710666875A CN 107465574 A CN107465574 A CN 107465574A
- Authority
- CN
- China
- Prior art keywords
- data
- layer
- internet site
- site group
- unit
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L43/00—Arrangements for monitoring or testing data switching networks
- H04L43/04—Processing captured monitoring data, e.g. for logfile generation
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L43/00—Arrangements for monitoring or testing data switching networks
- H04L43/02—Capturing of monitoring data
- H04L43/028—Capturing of monitoring data by filtering
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L43/00—Arrangements for monitoring or testing data switching networks
- H04L43/08—Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters
Abstract
The invention discloses a kind of internet site group plateform system and its parallel isolation streaming computational methods, system includes stream data Access Layer, stream data process layer, intermediate data storage layer and result data filing layer；Stream data Access Layer is used for the data for receiving acquisition system collection, and send to stream data process layer, stream data process layer is cleaned and filtered to the data of arrival, it is compared according to current time and weeds out stale data, then using parallel isolation streaming computational methods, the visit capacity for unit of having reached the standard grade is calculated, result data filing layer is used for the result of calculation for storing stream data process layer.The present invention counts parallel to more unit visit capacities of internet site group's plateform system and data analysis contributes to the reliability service of unit network and controls the customer group of website in real time, and different clients are in the detection of different geographical and period to website visiting.
Description
Technical field
The present invention relates to a kind of internet site group plateform system and its parallel isolation streaming computational methods, belong to enterprise's door
Family web technology field.
Background technology
In traditional flow chart of data processing, data are always first collected, are then placed data into DB.Needed as people
When query done to data by DB, obtain answer or carry out related processing.Although so seeming very rationally,
As a result but very it is compact and, especially some in real time search application environments in some particular problems, be similar to
The processed offline of MapReduce modes can not solve problem well.This has just drawn a kind of new data and has calculated structure ---
Stream calculation mode.It can be analyzed extensive flow-data in real time in continually changing motion process well,
The information to come in handy is captured, and result is sent to next calculate node.
Representing system more in early days has IBM System S, and it is a complete computing architecture, passes through " stream
Computing " technologies, the data of stream forms can be carried out with real-time analysis." initial system possesses about
800 microprocessors, but IBM claims, and according to demand, this numeral is also possible to up to ten thousand.Researcher talks about, wherein the portion of most critical
It is System S softwares to divide, and it can separate task, for example be divided into image recognition and text identification, then by the knot after processing
Fruit fragment forms complete answer.The director Nagui Halim of IBM laboratories high-performance stream computing project are spoken of：System
S is a brand-new operational pattern, and its flexibility and speed has much advantage.And compared with legacy system, its mode is more
It intellectuality, can suitably change, need to solve the problems, such as to be applicable it.
Commercial search engine, as Google, Bing and Yahoo！Deng the offer structuring generally in user's inquiry response
Web results, while it is also inserted into the text advertisements of the pay-per-click model based on flow.In order to which optimum position shows most on the page
Related advertisement, the possibility that an advertisement is clicked in context is given come dynamic estimation by some algorithms.Context can
The information such as user preference, geographical position, historical query, history click can be included.One main search engine may processing each second
Thousands of inquiries, each page may include multiple advertisements.For timely processing user feedback, it is necessary to which one low is prolonged
Late, expansible, highly reliable processing engine.However, for the very high application of these requirement of real-time, although MapReduce makees
Real-time improvement, but still be very unstable to meet application demand.Because Hadoop, which is batch processing, has made height optimization,
Map Reduce system operates static data typically via scheduling batch tasks；And one of typical normal form of streaming computing is
The flow of event of uncertain data speed flows into system, and system processing power or must pass through approximate calculate with event flow matches
The methods of method graceful degradation, commonly referred to as load bridging (load-shedding).Certainly, except load bridging, streaming computing
The mechanism such as fault-tolerant processing are also calculated with batch processing and are not quite similar.
MR also has the real-time numerical procedure of oneself, such as HOP.It is but this kind of based on MapReduce progress Stream Processings
Scheme have three major defects.
Input data is separated into the fragment of fixed size, then handled by MapReduce platform, shortcoming is to handle delay
Length, the expense of initialization process task to data slot is directly proportional.Small segmentation can reduce delay, increase additional overhead,
And the dependence management between being segmented is more complicated (such as a segmentation may may require that the information of previous segmentation)；Conversely,
Big segmentation can increase delay.Optimal fragment size depends on concrete application.
In order to support Stream Processing, MapReduce needs to be transformed into Pipeline pattern, rather than Reduce direct
Output；In view of efficiency, it is medium that intermediate result is preferably only stored in internal memory.These, which are changed, causes original MapReduce frameworks
Complexity greatly increase, be unfavorable for the maintenance and expansion of system.
User is forced to use MapReduce interface to define streaming operation, and this causes the scalability of user program to drop
It is low.
In summary, the pattern of Stream Processing, which determines, to use very different framework with batch processing, it is intended to build one
The individual general-purpose platform for being not only adapted to streaming computing but also being adapted to batch processing to calculate, as a result may be a highly complex system, and
And final system may be all undesirable to two kinds of calculating.
The content of the invention
The technical problems to be solved by the invention are the defects of overcoming prior art, there is provided a kind of internet site group platform
System and its parallel isolation streaming computational methods, more unit visit capacities of Web group are carried out using parallel isolation streaming algorithm
Line analysis, and the change of the access state of each unit of Web group system is recorded into preservation in real time, calculate in Web group and run
Deviation ratio of each unit under averagely daily running status, it is more single to efficiently solve internet site group's plateform system
Position operation causes the inaccurate problem when counting the visit capacity of each unit.
In order to solve the above technical problems, the present invention provides a kind of internet site group plateform system, including stream data connects
Enter layer, stream data process layer, intermediate data storage layer and result data filing layer；
The stream data Access Layer is used for the data for receiving acquisition system collection, and sends to stream data process layer；
There is a message-oriented middleware between the acquisition system and stream data Access Layer, the data flow for buffering collection enters streaming number
According to the speed of Access Layer；
The stream data process layer is cleaned and filtered to the data of arrival, and rejecting is compared according to current time
Fall stale data, then using the parallel visit capacity isolated streaming computational methods, calculate unit of having reached the standard grade；
The result data filing layer is used for the result of calculation for storing stream data process layer；
The intermediate data that the intermediate data storage layer is used in data storage processing procedure.
Foregoing message-oriented middleware is an ordered queue, by the way of first in first out that data are past from message-oriented middleware
Stream data Access Layer injects.
Foregoing internet site group plateform system can realize given query, calculate maximum, minimum value, average value, row
Sequence, window inside counting, non-repetition counting, special index filtering, hot statistics and ranking list.
Foregoing given query refers to, an element into internet site group's plateform system is character string one by one
Right, given query is exactly the value of character string pair under comparison, meets the requirements and does the processing of next step, and knot is counted until needing
Fruit；The given query digital independent number is：Read 0 and write 1.
Foregoing calculating maximum, minimum value, the method for average value are to preserve a middle anaplasia in intermediate data storage layer
Amount, only needs to take out every time, is updated after being calculated；In calculating process, digital independent number is：Read 1 and write 1.
The implementation of foregoing sequence is to preserve a data structure heap in intermediate data storage layer, renewal every time exists
Corresponding insert and delete is carried out thereon；Digital independent number is：Read 1 and write 1.
Foregoing window inside counting uses DGIM algorithms.
Foregoing non-repetition counting uses hash tables, search tree, FM algorithms or combinational estimation method.
Foregoing special index filtering uses bloom filter.
The parallel isolation streaming computational methods of internet site group's plateform system, comprise the following steps：
1) list of all units of having reached the standard grade is obtained, encoding orgCode by the organization of unit carries out uniqueness area
Point；
2) according to the quantity of the unit of having reached the standard grade obtained in the configurations and step 1) of Web group server, calculating needs
The Thread Count of unlatching, calculating formula are：T=U/C,
Wherein, T represents to need the Thread Count opened, and U represents to have reached the standard grade the quantity of unit, and C represents Web group server
CPU core number；
3) each thread is responsible for the visit capacity statistics of one or more unit of having reached the standard grade, and visit capacity statistics is mutually isolated simultaneously
And be all parallel computation, the calculation formula of the daily flowing of access statistics of a unit of having reached the standard grade is as follows：
Q=FP*0.4+SP*0.3+TP*0.3
Wherein, Q represents a unit flowing of access number of one day of having reached the standard grade, and FP represents the homepage visit of one day of unit network
Flow is asked, SP represents one day flowing of access of the two level page of unit network, and TP represents one day of the three-level page of unit network
Flowing of access, corresponding coefficient is then multiplied by respectively and can obtain flowing of access.
The beneficial effect that the present invention is reached：
The more unit features of single system of internet site group are converted into isolating streaming computation model parallel by the present invention, are used
Parallel isolation streaming algorithm carries out on-line analysis to more unit visit capacities of Web group, and by the visit of each unit of Web group system
Ask that the change of state records preservation in real time, calculate each unit run in Web group under averagely daily running status
Deviation ratio, efficiently solving the more unit operations of internet site group's plateform system causes when counting the visit capacity of each unit
Inaccurate problem.
More unit visit capacities of internet site group's plateform system are counted the present invention parallel and data analysis contributes to
The reliability service of unit network and in real time control website customer group, and different clients in different geographical and period to net
Stand access detection.
Brief description of the drawings
Fig. 1 is internet site group's plateform system structure chart of the present invention.
Embodiment
The invention will be further described below.Following examples are only used for the technical side for clearly illustrating the present invention
Case, and can not be limited the scope of the invention with this.
The internet site group plateform system of the present invention includes stream data Access Layer, stream data process layer, mediant
According to accumulation layer and result data filing layer.
The data of acquisition system collection continuously enter this internet site group by stream data Access Layer
Plateform system, there is a message-oriented middleware between acquisition system and stream data Access Layer, for buffering the data flow of collection
Enter the speed of streaming data access layer so that system loading is unlikely to excessive, is maintained on a somewhat gentle processing speed.
Herein, message-oriented middleware is exactly an ordered queue, by data from message-oriented middleware toward streaming by the way of first in first out
Data access layer injects, it is ensured that data are to arrive one by one.Corresponding calculate is obtained in stream data process layer to store, here
Calculating mainly from two in terms of progress：The data of arrival are cleaned and filtered, is compared and picked according to current time
Remove stale data；Data in same hour are merged (summation).Result of calculation is finally written to result data
File layer, for next system queries or used for other purposes.The mediant that intermediate data storage layer is used in data storage processing procedure
According to.
Following difference be present compared to data warehouse, database system in the internet site group plateform system of the present invention：
1. data are to arrive one by one, the data of next period are unknown；
2. the speed that data arrive is also uncontrollable；
3. data are effectual property, it is necessary to timely processing；
4. the order that data reach stream data Access Layer from acquisition system cannot be guaranteed；
5. task is endless.
If by its map-reducer task versus with hadoop, difference is：
Map and reduce are being run always, and map endlessly sends data, reducer also ceaselessly processing datas,
There is no the concept that tasks carrying is complete.
The internet site group plateform system of the present invention can realize following common data service：
Select--------------------------- fixed datas inquire about (exception or dirty data processing)
Max/min/avg------------------- maximins
Count/sum---------------------- sums or number statistics (such as pv etc.)
Count (distinct) --- --- --- --- --- --- non-repetition countings (typical such as UV)
Order by------------------------ sort (taking the user closely accessed)
(the most topN commodity of such as access times) are sorted after group by+ clustering function+order by----- clusters.
Specific implementation is as follows：
A, given query
This is simplest processing mode in streaming systems, it is however generally that into one of internet site group's plateform system
Element be one by one character string to (such as arg1, arg2, arg3 ... ...), then given query, is exactly relatively lower arg value, symbol
Close the processing for requiring to do next step, the statistical result until needing.Digital independent number：Read 0 and write 1.
B, maximum, minimum value, average value --- --- ----intermediate variable
An intermediate variable is preserved on intermediate store, only needs to take out every time, is updated after being calculated.
Digital independent number：Read 1 and write 1.
C, TopN sequences --- --- --- --- ----minimax heap
A data structure heap is preserved on intermediate store, renewal every time is also required for corresponding insert and delete realization.
Digital independent number：Read 1 and write 1.
D, window inside counting --- --- --- --- --- --- -- DGIM algorithms
Time window, it is possible to understand that into a queue, comprising two operations, add and remove.
It is also considered that, the time is not the time into system simultaneously, it may be possible to the logging time carried, this
Being can out of order arrival.
Here talk and count, just further comprises an operation, isContainsKey and get.
E, non-repetition counting --- --- ----hash tables, search tree, FM algorithms, combinational estimation
The logic of four kinds of modes is consistent：One will save historical data, second, compression histories data are wanted, third, to facilitate
Inquiry (whether judgement is present, and any time can summarized results).
And space, time, three indexs of accuracy can not be scrupled all again, it is small that you can not require that space-consuming has,
Judge that the time is short, at the same it is again accurate.
F, special index filtering --- --- -- bloom filter
Bloom filter are really an ancient and popular things.The system contacted at present, if using filtering, greatly
Part all very first times consider bloom filter filterings.Bloom filter are an extensive hash (multiple hash letters
Number), save space, time, while accuracy and ensure that general (can leak, but will not judge by accident).
G, --- --- --- --- specified time window counts hot statistics
Statistics granularity is first confirmd that, is Flow Record rank, or minute rank, or hour rank, when corresponding to
Between window, that is, time indicator slide most basic unit.
H, ranking list --- --- --- --- with the time decay by --- --
If the website of forum's property, there is ten big hot topics patch, be designated as t1, t2, t3 ... ..., t10.If accessing,
Or if new record comes, renewal.
But also a kind of situation is, midnight up to it is several it is small in the case of, be possible to no any access.
So order or that original order
Not necessarily, because the weight of each time slice is different.May sequentially can be：t3,t1,t2,t10……
At this time, it may be necessary to construct the data of some timer-triggered schedulers, such as 5 minutes once empty data, calculating process is triggered, again
Update weight values.
The present invention can also realize the processing of advanced analysis function, specific as follows：
Where-------------------------- designated statistics scopes；
Group by+having------------- segment the statistics of different dimensions；
The how individual data of join+union-------------------- merge；
As for the advanced analysis function such as rollup, cube, it can treat and.
The parallel isolation streaming computational methods of above-mentioned internet site group plateform system, comprise the following steps：
The first step, the list of all units of having reached the standard grade is obtained, encoding orgCode by the organization of unit is carried out uniquely
Property distinguish.
Second step, according to the quantity of the unit of having reached the standard grade obtained in the configurations and the first step of Web group server, root
The Thread Count for needing to open is calculated according to following algorithmic formula：T=U/C,
Wherein, T represents to need the Thread Count opened, and U represents to have reached the standard grade the quantity of unit, and C represents Web group server
CPU core number.
3rd step, each thread are responsible for the visit capacity statistics of one or more unit of having reached the standard grade, and visit capacity statistics is mutual
Isolate and be all parallel computation, the calculation formula of the daily flowing of access statistics of a unit of having reached the standard grade is as follows：
Q=FP*0.4+SP*0.3+TP*0.3
Wherein, Q represents a unit flowing of access number of one day of having reached the standard grade, and FP represents the homepage visit of one day of unit network
Flow is asked, SP represents one day flowing of access of the two level page of unit network, and TP represents one day of the three-level page of unit network
Flowing of access, corresponding coefficient is then multiplied by respectively and can obtain flowing of access.
Described above is only the preferred embodiment of the present invention, it is noted that for the ordinary skill people of the art
For member, without departing from the technical principles of the invention, some improvement and deformation can also be made, these are improved and deformation
Also it should be regarded as protection scope of the present invention.
Claims (10)
1. internet site group's plateform system, it is characterised in that middle including stream data Access Layer, stream data process layer
Data storage layer and result data filing layer；
The stream data Access Layer is used for the data for receiving acquisition system collection, and sends to stream data process layer；It is described
There is a message-oriented middleware between acquisition system and stream data Access Layer, the data flow for buffering collection enters stream data and connect
Enter the speed of layer；
The stream data process layer is cleaned and filtered to the data of arrival, is compared and weeded out according to current time
Issue evidence, then using the parallel visit capacity isolated streaming computational methods, calculate unit of having reached the standard grade；
The result data filing layer is used for the result of calculation for storing stream data process layer；
The intermediate data that the intermediate data storage layer is used in data storage processing procedure.
2. internet site group plateform system according to claim 1, it is characterised in that the message-oriented middleware is one
Ordered queue, data are injected from message-oriented middleware toward stream data Access Layer by the way of first in first out.
3. internet site group plateform system according to claim 1, it is characterised in that the internet site group platform
System can realize given query, calculate maximum, minimum value, average value, sequence, window inside counting, non-repetition counting, special index
Filtering, hot statistics and ranking list.
4. internet site group plateform system according to claim 3, it is characterised in that the given query refers to, enters
An element for entering internet site group's plateform system is that character string pair, given query are exactly character string pair under comparison one by one
Value, meets the requirements and does the processing of next step, the statistical result until needing；The given query digital independent number is：Read 0
Write 1.
5. internet site group plateform system according to claim 3, it is characterised in that the calculating maximum, minimum
Value, the method for average value are to preserve an intermediate variable in intermediate data storage layer, only need to take out every time, after being calculated
Renewal；In calculating process, digital independent number is：Read 1 and write 1.
6. internet site group plateform system according to claim 3, it is characterised in that the implementation of the sequence
To preserve a data structure heap in intermediate data storage layer, renewal every time carries out corresponding insert and delete thereon；Data are read
The number is taken to be：Read 1 and write 1.
7. internet site group plateform system according to claim 3, it is characterised in that the window inside counting uses
DGIM algorithms.
8. internet site group plateform system according to claim 3, it is characterised in that the non-repetition counting uses hash
Table, search tree, FM algorithms or combinational estimation method.
9. internet site group plateform system according to claim 3, it is characterised in that the special index filtering uses
bloom filter。
10. the parallel isolation streaming computational methods of internet site group's plateform system described in claim 1 to 9 any one,
It is characterised in that it includes following steps：
1）The list of all units of having reached the standard grade is obtained, encoding orgCode by the organization of unit carries out uniqueness differentiation；
2）According to the configurations and step 1 of Web group server）The quantity of the unit of having reached the standard grade of middle acquisition, calculating need to open
Thread Count, calculating formula is：T=U/C,
Wherein, T represents to need the Thread Count opened, and U represents to have reached the standard grade the quantity of unit, and C represents the CPU core of Web group server
Number；
3）Each thread is responsible for the visit capacity statistics of one or more unit of having reached the standard grade, and visit capacity statistics is mutually isolated and all
It is parallel computation, the calculation formula of the daily flowing of access statistics of a unit of having reached the standard grade is as follows：
Q=FP*0.4+SP*0.3+TP*0.3
Wherein, Q represents a unit flowing of access number of one day of having reached the standard grade, and FP represents the homepage access stream of one day of unit network
Amount, SP represent one day flowing of access of the two level page of unit network, and what TP represented the three-level page of unit network accesses for one day
Flow, corresponding coefficient is then multiplied by respectively and can obtain flowing of access.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710666875.9A CN107465574B (en) | 2017-08-07 | 2017-08-07 | Internet website group platform system and parallel isolation streaming computing method thereof |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710666875.9A CN107465574B (en) | 2017-08-07 | 2017-08-07 | Internet website group platform system and parallel isolation streaming computing method thereof |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107465574A true CN107465574A (en) | 2017-12-12 |
| CN107465574B CN107465574B (en) | 2020-11-10 |
Family
ID=60548358
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710666875.9A Active CN107465574B (en) | 2017-08-07 | 2017-08-07 | Internet website group platform system and parallel isolation streaming computing method thereof |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107465574B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110765222A (en) * | 2019-10-24 | 2020-02-07 | 成都路行通信息技术有限公司 | Interest point self-driving heat degree calculation method and platform based on Geohash codes |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7616625B1 (en) * | 2003-10-22 | 2009-11-10 | Sprint Communications Company L.P. | System and method for selective enhanced data connections in an asymmetrically routed network |
| CN104320311A (en) * | 2014-11-20 | 2015-01-28 | 国电南京自动化股份有限公司 | Heartbeat detection method of SCADA distribution type platform |
| CN104657502A (en) * | 2015-03-12 | 2015-05-27 | 浪潮集团有限公司 | System and method for carrying out real-time statistics on mass data based on Hadoop |
| CN105608144A (en) * | 2015-12-17 | 2016-05-25 | 山东鲁能软件技术有限公司 | Big data analysis platform device and method based on multilayer model iteration |
| CN105786864A (en) * | 2014-12-24 | 2016-07-20 | 国家电网公司 | Offline analysis method for massive data |
| CN105959151A (en) * | 2016-06-22 | 2016-09-21 | 中国工商银行股份有限公司 | High availability stream processing system and method |
-
2017
- 2017-08-07 CN CN201710666875.9A patent/CN107465574B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7616625B1 (en) * | 2003-10-22 | 2009-11-10 | Sprint Communications Company L.P. | System and method for selective enhanced data connections in an asymmetrically routed network |
| CN104320311A (en) * | 2014-11-20 | 2015-01-28 | 国电南京自动化股份有限公司 | Heartbeat detection method of SCADA distribution type platform |
| CN105786864A (en) * | 2014-12-24 | 2016-07-20 | 国家电网公司 | Offline analysis method for massive data |
| CN104657502A (en) * | 2015-03-12 | 2015-05-27 | 浪潮集团有限公司 | System and method for carrying out real-time statistics on mass data based on Hadoop |
| CN105608144A (en) * | 2015-12-17 | 2016-05-25 | 山东鲁能软件技术有限公司 | Big data analysis platform device and method based on multilayer model iteration |
| CN105959151A (en) * | 2016-06-22 | 2016-09-21 | 中国工商银行股份有限公司 | High availability stream processing system and method |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110765222A (en) * | 2019-10-24 | 2020-02-07 | 成都路行通信息技术有限公司 | Interest point self-driving heat degree calculation method and platform based on Geohash codes |
| CN110765222B (en) * | 2019-10-24 | 2022-04-19 | 成都路行通信息技术有限公司 | Interest point self-driving heat degree calculation method and platform based on Geohash codes |
Also Published As
| Publication number | Publication date |
|---|---|
| CN107465574B (en) | 2020-11-10 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11934409B2 (en) | Continuous functions in a time-series database | |
| Huang et al. | Tencentrec: Real-time stream recommendation in practice | |
| US10409650B2 (en) | Efficient access scheduling for super scaled stream processing systems | |
| US20200167355A1 (en) | Edge processing in a distributed time-series database | |
| CN111475509A (en) | Big data-based user portrait and multidimensional analysis system | |
| CN100375088C (en) | Segmentation and processing of continuous data streams using transactional semantics | |
| CN110869968A (en) | Event processing system | |
| CN109670116A (en) | A kind of intelligent recommendation system based on big data | |
| CN107515927A (en) | A kind of real estate user behavioural analysis platform | |
| Kumar | An encyclopedic overview of ‘big data’analytics | |
| US20140337274A1 (en) | System and method for analyzing big data in a network environment | |
| CN102662986A (en) | System and method for microblog message retrieval | |
| US10025645B1 (en) | Event Processing System | |
| CN106649687A (en) | Method and device for on-line analysis and processing of large data | |
| CN110795613A (en) | Commodity searching method, device and system and electronic equipment | |
| Bakaev et al. | Intelligent information system to support decision-making based on unstructured web data | |
| Choudhary et al. | A real-time fault tolerant and scalable recommender system design based on Kafka | |
| CN107465574A (en) | Internet site group plateform system and its parallel isolation streaming computational methods | |
| Prakash et al. | Big data preprocessing for modern world: opportunities and challenges | |
| Anusha et al. | Big data techniques for efficient storage and processing of weather data | |
| US20170004402A1 (en) | Predictive recommendation engine | |
| US11947545B2 (en) | Systems and methods for configuring data stream filtering | |
| Madaan et al. | Big data analytics: A literature review paper | |
| Kolici et al. | Scalability, memory issues and challenges in mining large data sets | |
| Prashanthi et al. | Generating analytics from web log |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CP03 | Change of name, title or address | ||
| CP03 | Change of name, title or address |
Address after: No. 38, New Model Road, Gulou District, Nanjing City, Jiangsu Province, 210000 Patentee after: Nanjing Nanzi Huadun Digital Technology Co.,Ltd. Address before: No. 38 Xinmofan Road, Gulou District, Nanjing City, Jiangsu Province, 211103 Patentee before: NANJING HUADUN POWER INFORMATION SECURITY EVALUATION CO.,LTD. |