CN106776837A - A kind of method of the security real-time deal association analysis based on MongoDB - Google Patents
A kind of method of the security real-time deal association analysis based on MongoDB Download PDFInfo
- Publication number
- CN106776837A CN106776837A CN201611062583.6A CN201611062583A CN106776837A CN 106776837 A CN106776837 A CN 106776837A CN 201611062583 A CN201611062583 A CN 201611062583A CN 106776837 A CN106776837 A CN 106776837A
- Authority
- CN
- China
- Prior art keywords
- time
- data
- security
- real
- analysis
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2228—Indexing structures
- G06F16/2255—Hash tables
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q40/00—Finance; Insurance; Tax strategies; Processing of corporate or income taxes
- G06Q40/04—Trading; Exchange, e.g. stocks, commodities, derivatives or currency exchange
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Engineering & Computer Science (AREA)
- Business, Economics & Management (AREA)
- Software Systems (AREA)
- Accounting & Taxation (AREA)
- Finance (AREA)
- Computational Linguistics (AREA)
- Computing Systems (AREA)
- Development Economics (AREA)
- Economics (AREA)
- Marketing (AREA)
- Strategic Management (AREA)
- Technology Law (AREA)
- General Business, Economics & Management (AREA)
- Financial Or Insurance-Related Operations Such As Payment And Settlement (AREA)
Abstract
It is a kind of method for being stored to security real-time transaction data based on MongoDB and being carried out real-time analysis the present invention relates to the big data Treatment Analysis field of software information.The inventive method includes：Build an independent MongoDB or the cluster formed by multiple MongoDB；Obtain the details of each security；Design one permanently stores table and an effective temporary storage table within half an hour；The interval acquiring time is set, real time security data are obtained in foregoing spaced points, and processed by duplicate removal, the real time data for obtaining is respectively stored on permanent table and interim table；By inquiring about the data of interim table, analysis obtains the form of expression of each security within various time periods, for the various exceptions occurred in each time period simultaneously, exports corresponding unusual fluctuation type and is encoded with security, forms real-time security Real-time Association Analyzing.The shortcoming that search efficiency is low, response speed is slow when being stored in the data analysis process of the inventive method solution big data quantity, improves the operation promptness of client.
Description
Technical field
It is that one kind is based on MongoDB to the real-time friendship of security the present invention relates to the big data Treatment Analysis field of software information
The method that easy data are stored and carried out real-time analysis.
Background technology
The real-time deal information of securities market, with information is more, quantity big, the characteristics of conclude the business frequent, frequently hands at this
Many transaction linkage information are there are in easily, for carrying out the people of short-term trading in the market, the pass of each securities information
Connection linkage has good reference value, the related information that securities trading process is is known sooner, in fast changing security city
In, more can rapidly obtain the judgement of dealing, so as to make a profit in the market, but the approach of the acquisition information content of people limit, very
Hardly possible obtains substantial amounts of unusual action information in a short time, and by taking the A-share of China's securities market as an example, cut-off in November, 2016 probably has
3000 stocks, there is 4 exchange hours of hour daily, and a Transaction Information was obtained by every 3 seconds, just probably have 4.5G within one day
Data volume, so big data volume there are many no unusual fluctuation or do not have related proper motion, can largely filter substantially
Fall, but people cannot in time process so many data quickly, by this method, may be by the energy for automatically processing of computer
Power, helps clerk that useful association unusual action information is obtained from substantial amounts of Transaction Information.
Analysis has the characteristics of data volume is big, requirement of real-time is high to security in real time, enters for traditional relevant database
The access of row data, is far not by far up to the mark when data volume is big in performance, so as to the situation of real-time cannot be met.
The content of the invention
Present invention solves the technical problem that being to provide a kind of side of the security real-time deal association analysis based on MongoDB
Method, search efficiency is low when being stored using traditional relational in the data analysis process for solving big data quantity, response speed is slow lacks
Point, and from the angle of multithreading, realization is obtained and is analyzed in a short time in substantial amounts of securities data.
The present invention solve above-mentioned technical problem technical scheme be,
Described method includes following steps：
Step 1：By MongoDB (database based on distributed document storage) as an efficient data access
Space, builds an independent MongoDB or the cluster that is formed by multiple MongoDB is used for after security Real time data acquisition efficiently
Accessing operation；
Step 2：The details of each security are obtained, acquisition makes with when being analyzed in real time for carrying out securities data
With；
Step 3：Design one permanently store table, every time obtain data storage in the table, for follow-up data validation
Analysis；An effective temporary storage table within half an hour is designed simultaneously, for carrying out each association analysis in the short time
Data acquisition；
Step 4：The interval acquiring time is set, real time security data is obtained in foregoing spaced points, and processed by duplicate removal,
The real time data for obtaining is respectively stored into and is permanently stored on table and temporary storage table；
Step 5：By the data inquired about on temporary storage table, analysis obtains the performance of each security within various time periods
Form, for the various exceptions occurred in each time period simultaneously, exports corresponding unusual fluctuation type and is encoded with security, forms real-time
Security Real-time Association Analyzing, the guidance for carrying out securities trading is referred to.
Described accessing operation, including：Independent MongoDB or MongoDB clusters are built, storage security real time data is obtained
Historical data and ephemeral data after taking off, it is big for historical data, MongoDB clusters are built, and historical data is pressed
Date carries out subregion reading.
Described details, the information such as including coding (prefix), title, capital stock, big shareholder's accounting.
The step 3, concretely comprises the following steps：
Step one：One is designed on MongoDB and permanently store table, for storing the continuous security real-time deal accumulated
Historical data, the time carries out subregion by date, and designs storage on different data spaces；For being deployed in cluster,
The Hash storage of security coding can be increased, encoded by security and stored on multiple servers different securities data Hash；
Step 2：For the analysis of real-time securities data, a temporary storage table is designed on MongoDB, for storing
Securities data in half an hour, using the TTL indexes of MongoDB, a TTL index is set up on a time row, is being faced
When table on time half an hour is set, the time point is automatically deleted the data of interim table.
Described step 4, concretely comprises the following steps：
Step one：Obtain related security write and prefix on the basis of call security Real-time data interface, all of card
Certificate data, by existing detailed data, complete the initialization that real time data is called, and operation securities data obtains program, leads in real time
The concurrent form of thread is crossed to transmit data to obtain data on interface；
Step 2：The security real time data that multithreading is obtained, including the opening price on the same day, highest price, lowest price, in real time
Valency, exchange hand, transaction value, the quantity for buying five grades and price, the quantity and price of selling five grades, each thread for obtaining data are being obtained
After the real time information of each security, parsed, at the same inquire about the real time information that obtains whether with recent acquisition
Information is identical, what identical expression had been present, is no longer preserved；
Step 3：By the security real time data after duplicate removal, it is stored in permanently stores table and temporary storage table respectively, is used for
Follow-up historical query and Real-time Association Analyzing；
The interval acquiring time of described security Real time data acquisition can be configured to every 3 seconds or 5 seconds and obtain once, for obtaining
Frequency data high are taken, storage and the treatment of system need high request.
It is described to realize security real-time deal unusual fluctuation association analysis, concretely comprise the following steps：
Step one：All securities datas are divided into multithreading, various time periods, various unusual fluctuation types are carried out on each thread
Analysis, analysis result there is the information of unusual fluctuation, being aggregated into unified show area carries out unified displaying；
Step 2：In each thread for having divided security, the unified current transaction letter for obtaining each security on interim table
Breath, and the real-time deal information in various time periods is obtained respectively, rising range, drop-off range, envelope are carried out respectively limit up, seal
Limit down, strike a bargain hand-off etc. analysis, the big simultaneously handle that is ranked up in order of the various intensity of anomalys within each time period is arranged
The output of former of name is on unified collection procedure；
Step 3：Each thread is in the unusual action information output in specific time interval to unified collection procedure, receiving
Collection program collects the data of all threads, including unusual fluctuation type is encoded with security, then carries out unified sequence, then suitable by sorting
Sequence, obtains in each time interval respectively, the rise of association, drop, envelope limit up, seal limit down, the very fast information such as hand-off, unite
One is input to show area, so as to realize the association analysis of each security in each time interval；
Step 4：For various time periods, User Defined configuration can be on demand carried out；Various abnormal types, can after
It is continuous to be increased；Various abnormal amounts of increase, drop range, turnover rate etc., can on demand carry out custom-configuring for user；Each parameter
Configuration can modify by the concrete condition of market and be set to optimal setting.
Described various exceptions, can be configured to：Quickly go up, quickly drop, envelope are limited up, seal and limit down, strike a bargain quickly
Amplify.
Described various time periods, can be configured to：First 30 seconds, first 1 minute, first 2 minutes, first 5 minutes, first 10 minutes, it is preceding
15 minutes.
The described real-time data of acquisition and the analysis being associated, refer to two kinds of programs of each self-operating, are obtained at interval
Completed in the time of 3 seconds or 5 seconds of access evidence, the securities data more than quantity carries out the data acquisition of multithreading respectively
With analysis, and for the separated time journey of all securities datas, can be by a point multiple when the security quantity that each thread is got is more
Thread is realized.
Described method, may also include step 6：After security real-time transaction data is stored in permanent table, subsequently carrying out
Turn over analysis, can analyze the security real-time deal incidence relation in bigger time interval, can analyze several recently by certain model
The quotation information of it or longer time, can be used as larger range of securities trading association analysis.
Beneficial effects of the present invention are as follows：
1st, the method for the present invention is using the efficient storage of MongoDB and abundant inquiry support, compared with the index branch of polymorphic type
The function of holding, MongoDB as an efficient data access space, for the storage by security real-time transaction data with
Analysis is obtained, and solves in the data analysis process of big data quantity that search efficiency when being stored using traditional relational is low, response speed
Slow shortcoming, and from the angle of multithreading, realization is obtained and is analyzed, obtained in a short time in substantial amounts of securities data
Go out the process of result, be fast changing securities trading, the information of unusual fluctuation is therefrom obtained in time, there is provided a kind of efficient side
Method.
2nd, the method for the present invention from MongoDB be based on MongoDB NoSQL types database, be one based in
Deposit, there is dsc data in physical memory so as to reach the database of the Data Physical mode of adjustment read-write, and its TTL index
The deletion after data over-time can be very quickly carried out, is that analysis in real time is the efficient management for having carried out outdated data, tool
There is good big data process performance, be the security in big data quantity so as to carry out efficient security real-time deal association analysis
In process of exchange, the unusual action information of burst is therefrom obtained, for clerk provides a kind of method of efficient association unusual fluctuation analysis,
The judgement of securities trading is carried out for client, Transaction apparatus meeting therein is found.
Brief description of the drawings
The present invention is further described below in conjunction with the accompanying drawings：
Accompanying drawing 1 is the flow chart of computer software functional unit of the present invention.
Specific embodiment
Fig. 1 is referred to, is the flow chart of computer software functional unit of the present invention, separately below to its each stream
Journey is implemented and is described：
Step 1：MongoDB environment is built, independent a MongoDB or cluster is set up, for the storage of real time data
Obtained with analysis；
Step 2：On the MongoDB for building, a permanent storage table is designed, the data storage for obtaining every time is at this
Permanently store on table, analyzed for follow-up data validation；An effective temporary storage table within half an hour is designed simultaneously,
TTL indexes are set up, the data more than half an hour are automatically deleted by MongoDB, for carrying out each association analysis in the short time
Data acquisition；
Step 3：The details of each security are initialized, (prefix), title, capital stock, big shareholder's accounting letter is such as encoded
Breath, for carrying out securities data, acquisition is used with when being analyzed in real time；
Step 4：Startup program, real time security data were obtained every 3 seconds or 5 seconds by way of multithreading, and through the past
Process again, the real time data for obtaining is respectively stored into and is permanently stored on table and temporary storage table；
Step 5：Analysis program is stabbed by being spaced minimum analysis time, is inquired about on temporary storage table in the way of multithreading
Data, the performance of each security in past 30 seconds, 1 minute, 2 minutes, 5 minutes, 10 minutes, the time such as 15 minutes is analyzed respectively
Form, limits up, seals and limit down, strike a bargain and quickly put simultaneously for occurring quick rise, quick drop, envelope in each time interval
It is big to wait abnormal, export corresponding unusual fluctuation type and encoded with security, in output to unified total program, total program is being obtained
After the unusual fluctuation type of each thread is encoded with security, by sequence and the threshold values of analysis unusual fluctuation, show area is finally shown to, is formed in fact
When security Real-time Association Analyzing, for carry out securities trading guidance refer to；
Step 6：After security real-time transaction data is stored in permanent table, the analysis that turns over subsequently is being carried out, can analyzed bigger
Security real-time deal incidence relation in time interval, can analyze these last few days or the market of longer time are believed by certain model
Breath, can be used as larger range of securities trading association analysis.
By building independent MongoDB or MongoDB clusters, for storing going through after security Real time data acquisition gets off
History data and ephemeral data；
MongoDB is with the Data Physical for adjusting read-write in being based on internal memory, dsc data being existed physical memory so as to reach
Mode, analysis has very big advantage to the security higher for requirement of real-time in real time, so selection MongoDB is used to carry out height
The real-time analysis of effect；
For historical data than it is larger in the case of, can be by setting up cluster and carrying out subregion by date to historical data
Mode, so as to improve the efficiency that data are read out.
When carrying out security Real-time data interface and calling, the security that need to first obtain correlation are write and prefix, are carrying out data
During analysis, the information such as capital stock, big shareholder's share-holding need to be related to, so before carrying out securities data and obtaining program operation in real time, needing
The initialization of underlying securities information is carried out, obtaining complete details is used for data acquisition, analysis.
It is described that two storage tables are set up on MongoDB, concretely comprise the following steps：
Step one, table of the design one for storing security real-time deal on MongoDB, for depositing for historical data
Storage, this data are the data of continuous accumulation；In view of historical data than larger and subsequently data need to be read out, this is permanent
Table needs the time by date to carry out subregion, and designs storage on different data spaces；If be deployed on cluster, can
Consider to carry out Hash storage by increase security coding, encoded by security and different securities data Hash is stored in many services
On device；
Step 2, for real-time securities data analysis, it is poor if the real time data being related to is within the time of half an hour
Real-time can seldom be determined, so an interim table is set up, for storing real-time securities data；Interim table by using
The TTL indexes of MongoDB, set up a TTL index (time-to-liveindex), interim by being arranged in a time
Time half an hour is set on table, and this time of the data of interim table will be automatically left out.
The method characteristic of described acquisition securities data is：
Step one, all of securities data, by existing detailed data, complete the initialization that real time data is called, and lead to
The concurrent form of thread is crossed to transmit data to obtain data on interface；
Step 2, multithreading obtain security real time information, include the same day opening price, highest price, lowest price, in real time
Valency, exchange hand, transaction value, the quantity for buying five grades and price, the quantity and price of selling five grades, each thread for obtaining data are being obtained
After the real time information of each security, parsed, at the same inquire about the real time information that obtains whether with recent acquisition
Information is identical, if identical expression has been present, is no longer preserved；
Step 3, by the security real time data after duplicate removal, preserve take on permanent table and interim table respectively, for follow-up
Historical query and Real-time Association Analyzing；
Step 4, security Real time data acquisition above, the acquisition time at interval can obtain by being configured to every 3 seconds or 5 seconds
Once, the storage of data and the requirement of real time of security are mainly in view of for obtaining frequency, obtaining frequency data storage high will
Ask comparing high, have requirement higher to the treatment of system.
It is described to realize concretely comprising the following steps for security real-time deal unusual fluctuation association analysis：
Step one, all security are divided into multithreading, various time intervals, various unusual fluctuation types are carried out on each thread
, is there is the information of unusual fluctuation by analysis in the result of analysis, and being aggregated into unified show area carries out unified displaying；
Step 2, in each thread for having divided security, the unified current transaction letter for obtaining each security on interim table
Breath, and from the timestamp for obtaining, before obtaining 30 seconds of timestamp respectively, before 1 minute, before 2 minutes, before 5 minutes, 10 minutes
Before, before 15 minutes etc. the time real-time deal information, carry out that rising range, drop-off range, envelope are limited up, envelope is limited down respectively,
The analysis of the hand-off grade that strikes a bargain, the rising range within each time period than larger, drop-off range than it is larger, shut and limit up
, shut it is limiting down, the hand-off very fast amplification of striking a bargain be ranked up in order and the output of former of ranking to unified
On collection procedure；
After step 3, each thread are exported onto unified collection procedure to the unusual action information in specific time interval,
Collection procedure collects the data of all threads, then carries out unified sequence, then by clooating sequence, obtains respectively in each time interval
It is interior, the rise of association, drop, envelope limit up, seal limit down, the very fast information such as hand-off, unify to be input to show area, so as to realize
In the association analysis of each security of each time interval；
Step 4, for each interlude section, can on demand carry out User Defined configuration；The type of each unusual fluctuation, can
Subsequently increased；Amount of increase, drop range, turnover rate of each unusual fluctuation etc., also can on demand carry out custom-configuring for user, each parameter
Configuration can modify by the concrete condition of market and be set to optimal setting.
Security real-time transaction data association analysis, obtains real-time data and the analysis being associated, and is divided to two kinds of programs each
Self-operating, and to a fairly large number of securities data, carries out data acquisition and the analysis of multithreading respectively, the acquisition of data with point
Analysis, completes substantially within the time of 3 seconds of interval acquiring data or 5 seconds, can just reach the effect for realizing analysis in real time, so
For the separated time journey of all securities datas, the security quantity that each thread is got specifically is seen, if security quantity can pass through too much
Multiple threads are divided to be realized, it is ensured that the acquisition and analysis of data are realized in the time interval for obtaining data.
After security real-time transaction data is stored in permanent table, the analysis that turns over subsequently can be being carried out, can analyzed bigger
Time interval in security real-time deal incidence relation, the row of these last few days or longer time can be analyzed by certain model
Feelings information, can be used as larger range of securities trading association analysis.
Using MongoDB with the data for adjusting read-write in being based on internal memory, dsc data being existed physical memory so as to reach
Physics mode, realizes that the analysis in real time of requirement of real-time security higher has very big advantage, and with using the super of its TTL index
The function that the time of mistake is automatically deleted, creates interim table, carries out efficient security real-time transaction data association, is in big data quantity
During securities trading, the unusual action information of burst is therefrom obtained, there is provided a kind of method for having friendship in time.
Claims (10)
1. a kind of method of the security real-time deal association analysis based on MongoDB, it is characterised in that comprise the following steps：
Step 1：By MongoDB as an efficient data access space, an independent MongoDB is built or by multiple
The cluster that MongoDB is formed is used for efficient accessing operation after security Real time data acquisition；
Step 2：The details of each security are obtained, acquisition is used with when being analyzed in real time for carrying out securities data；
Step 3：Design one permanently stores table, and the data storage for obtaining every time is analyzed in the table for follow-up data validation；
An effective temporary storage table within half an hour is designed simultaneously, and the data for carrying out each association analysis in the short time are obtained
Take；
Step 4：The interval acquiring time is set, real time security data are obtained in foregoing spaced points, and processed by duplicate removal, obtaining
The real time data for taking is respectively stored into and permanently stores on table and temporary storage table；
Step 5：By the data inquired about on temporary storage table, analysis obtains the performance shape of each security within various time periods
Formula, for the various exceptions occurred in each time period simultaneously, exports corresponding unusual fluctuation type and is encoded with security, forms card in real time
Certificate Real-time Association Analyzing, the guidance for carrying out securities trading is referred to.
2. method according to claim 1, it is characterised in that described accessing operation, including：Store and read its history
Data and ephemeral data, it is big for historical data, MongoDB clusters are built, and subregion reading is carried out by date to historical data
Take；
Described details, the information such as including coding, title, capital stock, big shareholder's accounting.
3. method according to claim 1, it is characterised in that the step 3, concretely comprises the following steps：
Step one：One is designed on MongoDB permanently store table, the history of the security real-time deal for storing continuous accumulation
Data, the time carries out subregion by date, and designs storage on different data spaces；For being deployed in cluster, can increase
Plus the Hash storage of security coding, encoded by security and store on multiple servers different securities data Hash；
Step 2：For the analysis of real-time securities data, a temporary storage table is designed on MongoDB, for storing half
Securities data in hour, using the TTL indexes of MongoDB, sets up a TTL index, in interim table on a time row
Upper setting time half an hour, the time point is automatically deleted the data of interim table.
4. method according to claim 2, it is characterised in that the step 3, concretely comprises the following steps：
Step one：One is designed on MongoDB permanently store table, the history of the security real-time deal for storing continuous accumulation
Data, the time carries out subregion by date, and designs storage on different data spaces；For being deployed in cluster, can increase
Plus the Hash storage of security coding, encoded by security and store on multiple servers different securities data Hash；
Step 2：For the analysis of real-time securities data, a temporary storage table is designed on MongoDB, for storing half
Securities data in hour, using the TTL indexes of MongoDB, sets up a TTL index, in interim table on a time row
Upper setting time half an hour, the time point is automatically deleted the data of interim table.
5. the method according to claim any one of 1-4, it is characterised in that described step 4, concretely comprises the following steps：
Step one：Obtain related security write and prefix on the basis of call security Real-time data interface, all of Number
According to, by existing detailed data, the initialization that real time data is called being completed, operation securities data obtains program in real time, by line
The form of Cheng Bingfa transmits data to obtain data on interface；
Step 2：The security real time data that multithreading is obtained, including the opening price on the same day, highest price, lowest price, real-time valency, into
Friendship amount, transaction value, the quantity for buying five grades and price, the quantity and price of selling five grades, each thread for obtaining data are being obtained respectively
After the real time information of individual security, parsed, at the same inquire about obtain real time information whether the information with recent acquisition
It is identical, what identical expression had been present, no longer preserved；
Step 3：By the security real time data after duplicate removal, it is stored in permanently stores table and temporary storage table respectively, for follow-up
Historical query and Real-time Association Analyzing；
The interval acquiring time of described security Real time data acquisition can be configured to every 3 seconds or 5 seconds and obtain once, for obtaining frequency
Rate data high, storage and the treatment of system need high request.
6. the method according to claim any one of 1-4, it is characterised in that described step 5, concretely comprises the following steps：
Step one：All securities datas are divided into multithreading, carried out on each thread various time periods, various unusual fluctuation types point
, is there is the information of unusual fluctuation by analysis in the result of analysis, and being aggregated into unified show area carries out unified displaying；
Step 2：In each thread for having divided security, the unified currency transaction information for obtaining each security on interim table, and
The real-time deal information in various time periods is obtained respectively, rising range, drop-off range, envelope is carried out respectively and limited up, sealed limit down
Plate, strike a bargain hand-off etc. analysis, the various intensity of anomalys within each time period it is big be ranked up in order and ranking before
The output of several is on unified collection procedure；
Step 3：Each thread is in the unusual action information output in specific time interval to unified collection procedure, collecting journey
Sequence collects the data of all threads, including unusual fluctuation type is encoded with security, then carries out unified sequence, then by clooating sequence, point
Do not obtain in each time interval, the rise of association, drop, envelope limit up, seal limit down, the very fast information such as hand-off, unify defeated
Enter to show area, so as to realize the association analysis of each security in each time interval；
Step 4：For various time periods, User Defined configuration can be on demand carried out；Various abnormal types, can subsequently enter
Row increases；Various abnormal amounts of increase, drop range, turnover rate etc., can on demand carry out custom-configuring for user；The configuration of each parameter
Can modify by the concrete condition of market and be set to optimal setting
Described various exceptions, can be configured to：Quickly go up, quickly drop, envelope are limited up, seal and limit down, strike a bargain and quickly amplify；
Described various time periods, can be configured to：First 30 seconds, first 1 minute, first 2 minutes, first 5 minutes, first 10 minutes, first 15 points
Clock；
The described real-time data of acquisition and the analysis being associated, refer to two kinds of programs of each self-operating, in interval acquiring number
According to time of 3 seconds or 5 seconds in complete, the securities data more than quantity, carry out respectively the data acquisition of multithreading with point
Analysis, and for the separated time journey of all securities datas, can be by a point multiple threads when the security quantity that each thread is got is more
Realized.
7. method according to claim 5, it is characterised in that described step 5, concretely comprises the following steps：
Step one：All securities datas are divided into multithreading, carried out on each thread various time periods, various unusual fluctuation types point
, is there is the information of unusual fluctuation by analysis in the result of analysis, and being aggregated into unified show area carries out unified displaying；
Step 2：In each thread for having divided security, the unified currency transaction information for obtaining each security on interim table, and
The real-time deal information in various time periods is obtained respectively, rising range, drop-off range, envelope is carried out respectively and limited up, sealed limit down
Plate, strike a bargain hand-off etc. analysis, the various intensity of anomalys within each time period it is big be ranked up in order and ranking before
The output of several is on unified collection procedure；
Step 3：Each thread is in the unusual action information output in specific time interval to unified collection procedure, collecting journey
Sequence collects the data of all threads, including unusual fluctuation type is encoded with security, then carries out unified sequence, then by clooating sequence, point
Do not obtain in each time interval, the rise of association, drop, envelope limit up, seal limit down, the very fast information such as hand-off, unify defeated
Enter to show area, so as to realize the association analysis of each security in each time interval；
Step 4：For various time periods, User Defined configuration can be on demand carried out；Various abnormal types, can subsequently enter
Row increases；Various abnormal amounts of increase, drop range, turnover rate etc., can on demand carry out custom-configuring for user；The configuration of each parameter
Can modify by the concrete condition of market and be set to optimal setting.
Described various exceptions, can be configured to：Quickly go up, quickly drop, envelope are limited up, seal and limit down, strike a bargain and quickly amplify；
Described various time periods, can be configured to：First 30 seconds, first 1 minute, first 2 minutes, first 5 minutes, first 10 minutes, first 15 points
Clock；
The described real-time data of acquisition and the analysis being associated, refer to two kinds of programs of each self-operating, in interval acquiring number
According to time of 3 seconds or 5 seconds in complete, the securities data more than quantity, carry out respectively the data acquisition of multithreading with point
Analysis, and for the separated time journey of all securities datas, can be by a point multiple threads when the security quantity that each thread is got is more
Realized.
8. the method according to claim any one of 1-4, it is characterised in that described method, may also include step 6：Card
After certificate real-time transaction data is stored in permanent table, the analysis that turns over subsequently is being carried out, the security in bigger time interval can analyzed
Real-time deal incidence relation, the quotation information of these last few days or longer time can be analyzed by certain model, can be used as bigger model
The securities trading association analysis enclosed.
9. method according to claim 5, it is characterised in that described method, may also include step 6：Security are handed in real time
After easy data are stored in permanent table, the analysis that turns over subsequently is being carried out, the security real-time deal in bigger time interval can analyzed
Incidence relation, the quotation information of these last few days or longer time can be analyzed by certain model, can be used as larger range of security
Transaction association is analyzed.
10. method according to claim 7, it is characterised in that described method, may also include step 6：Security are handed in real time
After easy data are stored in permanent table, the analysis that turns over subsequently is being carried out, the security real-time deal in bigger time interval can analyzed
Incidence relation, the quotation information of these last few days or longer time can be analyzed by certain model, can be used as larger range of security
Transaction association is analyzed.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201611062583.6A CN106776837A (en) | 2016-11-25 | 2016-11-25 | A kind of method of the security real-time deal association analysis based on MongoDB |
| PCT/CN2016/109349 WO2018094777A1 (en) | 2016-11-25 | 2016-12-10 | Method for association analysis of real-time trading of security |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201611062583.6A CN106776837A (en) | 2016-11-25 | 2016-11-25 | A kind of method of the security real-time deal association analysis based on MongoDB |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN106776837A true CN106776837A (en) | 2017-05-31 |
Family
ID=58901801
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201611062583.6A Pending CN106776837A (en) | 2016-11-25 | 2016-11-25 | A kind of method of the security real-time deal association analysis based on MongoDB |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN106776837A (en) |
| WO (1) | WO2018094777A1 (en) |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107862031A (en) * | 2017-11-02 | 2018-03-30 | 东软集团股份有限公司 | Method for processing business, device and server |
| CN108197301A (en) * | 2018-01-26 | 2018-06-22 | 北京博睿宏远数据科技股份有限公司 | A kind of method for being used to quantify different stock trader APP market refresh rates |
| CN108470045A (en) * | 2018-03-06 | 2018-08-31 | 平安科技（深圳）有限公司 | The method and storage medium that electronic device, data chain type are filed |
| CN110019436A (en) * | 2017-07-14 | 2019-07-16 | 北京国双科技有限公司 | Data import deriving method and device, tables of data treating method and apparatus |
| CN110278260A (en) * | 2019-06-17 | 2019-09-24 | 武汉灯塔之光科技有限公司 | A kind of forwarding recorded broadcast mthods, systems and devices of difference securities market data |
| CN112835930A (en) * | 2021-03-03 | 2021-05-25 | 上海渠杰信息科技有限公司 | Database query method and device |
| CN114661771A (en) * | 2022-04-14 | 2022-06-24 | 广州经传多赢投资咨询有限公司 | Stock data storage and reading method, equipment and readable storage medium |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109636611A (en) * | 2018-10-19 | 2019-04-16 | 深圳平安财富宝投资咨询有限公司 | Clear acquisition methods, server, storage medium and the device of configuration information |
| CN111062810A (en) * | 2019-11-12 | 2020-04-24 | 上交所技术有限责任公司 | Interface-based multidimensional index data processing method suitable for security trading system |
| CN112750040B (en) * | 2021-01-13 | 2023-07-14 | 国泰君安证券股份有限公司 | Data processing method, system, application, device, processor and storage medium for realizing thread security applied to memory transaction system |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102609464A (en) * | 2012-01-16 | 2012-07-25 | 北京亿赞普网络技术有限公司 | Method and device for associative table query of MONGODB shards |
| CN103207919A (en) * | 2013-04-26 | 2013-07-17 | 北京亿赞普网络技术有限公司 | Method and device for quickly inquiring and calculating MangoDB cluster |
| CN104112010A (en) * | 2014-07-16 | 2014-10-22 | 深圳市国泰安信息技术有限公司 | Data storage method and device |
| WO2016032548A1 (en) * | 2014-08-25 | 2016-03-03 | Hewlett Packard Enterprise Development Lp | Providing transactional support to a data storage system |
| CN105959169A (en) * | 2016-07-19 | 2016-09-21 | 中国银联股份有限公司 | Transaction data processing system and method |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR100645513B1 (en) * | 2004-10-15 | 2006-11-15 | 삼성전자주식회사 | Performance Management Data Processor for single table and Method thereof |
| CN102024010A (en) * | 2010-06-04 | 2011-04-20 | 西本新干线股份有限公司 | Data processing system and data processing method |
-
2016
- 2016-11-25 CN CN201611062583.6A patent/CN106776837A/en active Pending
- 2016-12-10 WO PCT/CN2016/109349 patent/WO2018094777A1/en active Application Filing
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102609464A (en) * | 2012-01-16 | 2012-07-25 | 北京亿赞普网络技术有限公司 | Method and device for associative table query of MONGODB shards |
| CN103207919A (en) * | 2013-04-26 | 2013-07-17 | 北京亿赞普网络技术有限公司 | Method and device for quickly inquiring and calculating MangoDB cluster |
| CN104112010A (en) * | 2014-07-16 | 2014-10-22 | 深圳市国泰安信息技术有限公司 | Data storage method and device |
| WO2016032548A1 (en) * | 2014-08-25 | 2016-03-03 | Hewlett Packard Enterprise Development Lp | Providing transactional support to a data storage system |
| CN105959169A (en) * | 2016-07-19 | 2016-09-21 | 中国银联股份有限公司 | Transaction data processing system and method |
Non-Patent Citations (1)
| Title |
|---|
| 雷萍 等: "《电子金融与支付实例教程》", 31 December 2010 * |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110019436A (en) * | 2017-07-14 | 2019-07-16 | 北京国双科技有限公司 | Data import deriving method and device, tables of data treating method and apparatus |
| CN107862031A (en) * | 2017-11-02 | 2018-03-30 | 东软集团股份有限公司 | Method for processing business, device and server |
| CN108197301A (en) * | 2018-01-26 | 2018-06-22 | 北京博睿宏远数据科技股份有限公司 | A kind of method for being used to quantify different stock trader APP market refresh rates |
| CN108470045A (en) * | 2018-03-06 | 2018-08-31 | 平安科技（深圳）有限公司 | The method and storage medium that electronic device, data chain type are filed |
| CN108470045B (en) * | 2018-03-06 | 2020-02-18 | 平安科技（深圳）有限公司 | Electronic device, data chain archiving method and storage medium |
| CN110278260A (en) * | 2019-06-17 | 2019-09-24 | 武汉灯塔之光科技有限公司 | A kind of forwarding recorded broadcast mthods, systems and devices of difference securities market data |
| CN112835930A (en) * | 2021-03-03 | 2021-05-25 | 上海渠杰信息科技有限公司 | Database query method and device |
| CN114661771A (en) * | 2022-04-14 | 2022-06-24 | 广州经传多赢投资咨询有限公司 | Stock data storage and reading method, equipment and readable storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| WO2018094777A1 (en) | 2018-05-31 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106776837A (en) | A kind of method of the security real-time deal association analysis based on MongoDB | |
| CN101853299B (en) | Image searching result ordering method based on perceptual cognition | |
| CN110020091B (en) | Medical search engine system based on block chain | |
| WO2005084234A3 (en) | Method of and system for obtaining data from multiple sources and raking documents based on meta data obtained through collaborative filtering and other matching techniques | |
| WO2011025696A1 (en) | Method and system of information matching in electronic commerce website | |
| CN111475509A (en) | Big data-based user portrait and multidimensional analysis system | |
| CN111127105A (en) | User hierarchical model construction method and system, and operation analysis method and system | |
| CN106250287A (en) | A kind of log information processing means | |
| CN104112010B (en) | A kind of date storage method and device | |
| CN101706926A (en) | Method for investigating and processing cigarette consumption information | |
| CN105677549B (en) | A kind of software testing management method and system | |
| CN106169959A (en) | A kind of log processing device | |
| CN104361109B (en) | The method and apparatus for determining picture the selection result | |
| CN110471903A (en) | Heterogeneous system nodal information method of summary, data library generating method, data creation method and device | |
| CN104331493A (en) | Method and device for generating trend interpretation data by virtue of computer | |
| CN104615734B (en) | A kind of community management service big data processing system and its processing method | |
| CN109460775A (en) | A kind of data filling method and device based on comentropy | |
| CN107818156A (en) | A kind of real time individual news recommends method and system | |
| TWI557662B (en) | The method of proactively advising events | |
| CN104182539A (en) | Abnormal information batch processing method and system | |
| CN107783974A (en) | Data handling system and method | |
| CN106250405A (en) | A kind of magnanimity information processing system | |
| CN106250406A (en) | A kind of log processing method | |
| CN109960701A (en) | A kind of big data processing method and system based on mixing engine | |
| CN110457357A (en) | Lawyer's recommended method neural network based and relevant device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20170531 |
|
| RJ01 | Rejection of invention patent application after publication |