CN110415027A - A kind of big data market plateform system - Google Patents
A kind of big data market plateform system Download PDFInfo
- Publication number
- CN110415027A CN110415027A CN201910640372.3A CN201910640372A CN110415027A CN 110415027 A CN110415027 A CN 110415027A CN 201910640372 A CN201910640372 A CN 201910640372A CN 110415027 A CN110415027 A CN 110415027A
- Authority
- CN
- China
- Prior art keywords
- data
- module
- user
- market
- processing module
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5083—Techniques for rebalancing the load in a distributed system
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/16—Error detection or correction of the data by redundancy in hardware
- G06F11/20—Error detection or correction of the data by redundancy in hardware using active fault-masking, e.g. by switching out faulty elements or by switching in spare elements
- G06F11/202—Error detection or correction of the data by redundancy in hardware using active fault-masking, e.g. by switching out faulty elements or by switching in spare elements where processing functionality is redundant
- G06F11/2023—Failover techniques
- G06F11/2028—Failover techniques eliminating a faulty processor or activating a spare
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2455—Query execution
- G06F16/24552—Database cache management
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/25—Integrating or interfacing systems involving database management systems
- G06F16/252—Integrating or interfacing systems involving database management systems between a Database Management System and a front-end application
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q30/00—Commerce
- G06Q30/02—Marketing; Price estimation or determination; Fundraising
- G06Q30/0201—Market modelling; Market analysis; Collecting market data
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q40/00—Finance; Insurance; Tax strategies; Processing of corporate or income taxes
- G06Q40/06—Asset management; Financial planning or analysis
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Business, Economics & Management (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Development Economics (AREA)
- Accounting & Taxation (AREA)
- Databases & Information Systems (AREA)
- Finance (AREA)
- Strategic Management (AREA)
- General Engineering & Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Entrepreneurship & Innovation (AREA)
- Marketing (AREA)
- Game Theory and Decision Science (AREA)
- Economics (AREA)
- Software Systems (AREA)
- General Business, Economics & Management (AREA)
- Computational Linguistics (AREA)
- Quality & Reliability (AREA)
- Human Resources & Organizations (AREA)
- Operations Research (AREA)
- Technology Law (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Financial Or Insurance-Related Operations Such As Payment And Settlement (AREA)
Abstract
The invention discloses big data market plateform systems, simplify exchange's ticker, reduce docking cost and market delay, investor is enable more fully to analyze market situation.Its technical solution are as follows: the difficulty that user accesses more market is reduced by the design of user's api interface.Lower system delay is brought for the simple binary coding SBE and inside high-performance UDP communication frame of low latency optimization.The present invention also provides the diversification data such as K line, timesharing, index, other open market data, public sentiment data and other analysis data, help investor that can more thoroughly understand capital market, provide more reference frames for the transaction of numerous investors.Have benefited from multiple nodes, by fens technology, user can be connected to according to load balancing it is different it is preposition on, it is preposition to obtain data from different nodes according to load balancing so that investor obtains very fast experience.
Description
Technical field
The present invention relates to big data fields, and in particular to can be applied to financial instrument futures scene has big data processing
The market plateform system of ability.
Background technique
In financial transaction field, market data play a crucial role various policy-drivens, and market speed is determined
Determine transactions velocity.
However, there are technical thresholds and access threshold for each exchange's market of existing market Customer Acquisition, while itself
Maintenance cost is higher, reliability is lower, causes existing market platform latency larger.International finance information exchange tissue in 2005
(FIX Protocol Limited abbreviation FPL) has issued the FAST for supporting the communication of the high-throughput data between financial institution
(FIX Adapted for STreaming) coding method.Mainly there is New York Securities Exchange using the exchange that FAST is encoded
Institute, Chicago Merchandise Exchange, Shanghai Stock Exchange etc..Most of market are all based on FAST coding, master currently on the market
Wanting target is to carry out high compression to data, to reduce bandwidth utilization rate, thus also results in delay and increases.
In recent years, big data, AI (Artificial Intelligence artificial intelligence) and intelligent finance technological applications high speed
Development, existing system can not support history mass data query function, extensive batch to calculate service, calculate clothes in real time
Business is unable to satisfy the demands such as customer analysis other open market data, public sentiment datas.
Summary of the invention
A brief summary of one or more aspects is given below to provide to the basic comprehension in terms of these.This general introduction is not
The extensive overview of all aspects contemplated, and be both not intended to identify critical or decisive element in all aspects also non-
Attempt to define the range in terms of any or all.Its unique purpose is to provide the one of one or more aspects in simplified form
A little concepts are with the sequence for more detailed description given later.
The purpose of the present invention is to solve the above problem, provides a kind of big data market plateform system, solves user
The problem of market and enhancing data are obtained in process of exchange, can simplify the ticker of more exchanges, reduction is butted into
This, and the delay for obtaining market is greatly reduced.Simultaneously available index, other open market data, public sentiment data and its
He analyzes data, so that investor analyzes market situation more fully hereinafter, distributed structure/architecture also improves reliability, reduces use
The maintenance cost at family.
The technical solution of the present invention is as follows: system includes core clothes present invention discloses a kind of big data market plateform system
Business module, database module, cache module, messaging bus, Service Processing Module, preposition processing module, user's api interface mould
Block, in which:
Core services module includes initial data reception resolution unit, SBE coding unit, divides theme release unit, wherein
Initial data receives resolution unit and is used for from the acquired raw line feelings data of transaction and carries out data parsing, and SBE coding unit is used
In the original market data progress SBE volume received and parsed through out using simple binary coding SBE to initial data receiving unit
Code divides theme release unit to be divided into different themes for the market data after encoding SBE and is published to messaging bus；
Cache module, for the market data after obtaining SBE coding on messaging bus or from Service Processing Module
Treated data are obtained, are stored in itself memory and/or database module, and, it receives and comes from Service Processing Module
Or the request of the reading data of preposition processing module, data are obtained from the memory of itself and are returned, if do not had in memory
Then inquire in database module to return to the data inquired；
User's api interface module, is used for subscriber access system, user by api interface initiates include user log in, it is several
According to subscribe to, the request including data query, preposition processing module is responded the request treated that data return to user；
Preposition processing module receives user and passes through user for pushing real time data, inquiry same day data and historical data
The request and response that api interface module is sent；
Service Processing Module carries out after obtaining storing data from cache module for obtaining market data from messaging bus
Processing result is sent to cache module and stored by alignment processing, also from preposition processing module receive include user authentication,
Request including enquiry of historical data obtains data from cache module and returns to preposition processing module.
One embodiment of big data market plateform system according to the present invention, core services module and messaging bus carry out single
It being transmitted to data, messaging bus carries out one-way data transfer with preposition processing module, cache module and Service Processing Module respectively,
Cache module carries out bidirectional data transfers, cache module and preposition processing module with database module, Service Processing Module respectively
Carry out one-way data transfer.Preposition processing module carries out bi-directional data biography with Service Processing Module, user's api interface module respectively
It is defeated.
One embodiment of big data market plateform system according to the present invention, system further include:
Fens module and preposition processing module carry out bidirectional data transfers, provide load balancing clothes for preposition processing module
Business.
One embodiment of big data market plateform system according to the present invention, the load balancing clothes configured in fens module
Business is to obtain the current loading condition of each preposition processing module by communicating with each preposition processing module, selects preposition processing
The smallest preposition address is loaded in module, carries out data communication back to user's api interface module.
One embodiment of big data market plateform system according to the present invention is to adopt to the data after SBE coding in system
It is transmitted with udp protocol.
One embodiment of big data market plateform system according to the present invention, preposition processing module include that general market are preposition
Unit, inquiry data front end units and historical data front end units, in which:
General market front end units receive the subscription request of user's api interface module, in the number by user authentication unit
After authentication, long connection is established with user's api interface module, data are obtained from messaging bus, cache module, and according to user
The theme of subscription carries out the push of different data to corresponding user's api interface module；
Data front end units are inquired, the inquiry same day for receiving user's api interface module specifies the request of data, passes through
Data, which are obtained, from cache module after the authentication of user authentication unit returns to user's api interface module；
Historical data front end units, the query history for receiving user's api interface module are specified the request of data, are passed through
Data are obtained from cache module after the authentication of user authentication unit and return to user's api interface module, if the history number obtained
According to before certain time, then the directly link of return file download is for Customer Acquisition data.
One embodiment of big data market plateform system according to the present invention, Service Processing Module include but is not limited to user
Authentication unit, K line market unit, indicator calculating unit, data analysis unit, returns and surveys unit at historical data unit, in which:
User authentication unit identifies for the permission to user accesses data；
K line market unit, for the calculation processing to K line and timesharing；
Data analysis unit, for analyzing data such as market；
Historical quotes unit, for storing historical quotes；
Indicator calculating unit, for the calculating to the various indexs of market；
Return and survey unit, for calculate under certain condition including rate of return (RMT), it is maximum withdraw rate including return measured data.
One embodiment of big data market plateform system according to the present invention, system are distributed structure/architecture, are deployed in multiple
Node is automatically switched by hot standby realization failure in real time.
The present invention, which compares the prior art, to be had following the utility model has the advantages that the present invention is by designing succinct easy-to-use API, greatly
Reduce the difficulty that user accesses more market.Meanwhile having benefited from encoding SBE for the simple binary of low latency optimization
(Simple Binary Encoding) and internal high-performance UDP (User Datagram Protocol, User Datagram Protocol
View) communication frame, bring lower system delay.Due to using the standardized finance data exchange agreement of FPL publication
SBE, user can reduce the number of copy data when selecting SBE categorical data as returning, so that whole system obtains
The speed of data is faster.In addition, the present invention also provides K line, timesharing, index, other open cities in addition to basic market data
The diversification data such as field data, public sentiment data and other analysis data, help investor that can more thoroughly understand capital city
, more reference frames are provided for the transaction of numerous investors.The present invention is distributed structure/architecture, can neatly be disposed multiple
Node, hot standby realization failure automatically switches in real time, it is ensured that market are not interrupted, and the reliability of whole system is improved, and reduce user
Maintenance cost.Have benefited from multiple nodes, passes through fens (preposition gateway, for being responsible for the technology of access-in management and load balancing)
User can be connected to according to load balancing it is different it is preposition on, it is preposition to obtain number from different service nodes according to load balancing
According to so that investor obtains very fast experience.
Detailed description of the invention
After the detailed description for reading embodiment of the disclosure in conjunction with the following drawings, it better understood when of the invention
Features described above and advantage.In the accompanying drawings, each component is not necessarily drawn to scale, and has similar correlation properties or feature
Component may have same or similar appended drawing reference.
Fig. 1 shows the system architecture diagram of an embodiment of big data market plateform system of the invention.
Fig. 2 shows the data subscription flow charts that an embodiment of big data market plateform system of the invention is configured.
Fig. 3 shows the data query flow chart that an embodiment of big data market plateform system of the invention is configured.
Specific embodiment
Below in conjunction with the drawings and specific embodiments, the present invention is described in detail.Note that below in conjunction with attached drawing and specifically real
The aspects for applying example description is merely exemplary, and is understood not to carry out any restrictions to protection scope of the present invention.
Fig. 1 shows the system architecture of an embodiment of big data market plateform system of the invention.Referring to Figure 1, originally
The system of embodiment includes: core services module, database module, cache module, messaging bus, Service Processing Module, preposition
Processing module, user's api interface module, fens module.
The present invention is distributed structure/architecture, can neatly dispose multiple nodes, and hot standby realization failure automatically switches in real time.Core
Central server module and messaging bus carry out one-way data transfer, messaging bus respectively with preposition processing module, cache module and industry
Processing module of being engaged in carries out one-way data transfer, and cache module carries out bi-directional data with database module, Service Processing Module respectively
Transmission, cache module and preposition processing module carry out one-way data transfer.Preposition processing module respectively with Service Processing Module, use
Family api interface module, fens module carry out bidirectional data transfers.
Core services module further comprises: initial data receives resolution unit, SBE coding unit, divides theme publication single
Member.
Initial data receives the market seat that resolution unit obtains exchange, uses (the application of API provided by each transaction
Program Interfaces) or network packet communications protocol, original market data are collected from each exchange and carry out data parsing.
Each exchange includes but is not limited to: Shanghai Stock Exchange, Shenzhen Stock Exchange, Chinese Financial Futures Market exchange, Shanghai
Futures exchange, Shanghai international energy trade center, Dalian Commodity Exchange, Zhengzhou Commodity Exchange etc..
What SBE coding unit was used to receive and parse through out initial data receiving unit using simple binary coding SBE
Original market data carry out SBE coding.
Market data after dividing theme release unit to be used to encode SBE are divided into different themes and are published to messaging bus
On.It is using UDP (User Datagram Protocol, User Datagram Protocol to the transmission of the data after SBE coding in system
View).
Cache module is specific to handle are as follows: cache module is obtained from messaging bus for carrying out data buffer storage and storage
The market data of different themes after SBE coding, are stored in the memory of cache module after decoding process, while will acquire
Market data are written in database module.In addition, cache module is also obtained from each processing unit in Service Processing Module
Data after processing unit processes, and being deposited into memory and database module, such as K line, timesharing, Relative Strength Index, random
The data such as index.These data are deposited into the memory of cache module itself, convenient for accelerating the read or write speed of data, and are being set
Data from deletion in memory or deletion while being saved in database module after the period threshold set.Cache module, which receives, to be come
From the request of Service Processing Module or the reading data of preposition processing module, data are obtained from the memory of itself and are returned to
Corresponding module, cache module can be inquired from database module to return if the data for being not intended to obtain in memory
Return the data in the database module inquired.
Market data of the Service Processing Module for the different themes after obtaining SBE coding from messaging bus, from caching
The various data such as user information, K line are obtained in module.It is laggard that these processing units in Service Processing Module obtain market data
Processing result is sent to cache module and stored by row alignment processing.In addition, these processing units in Service Processing Module
The request such as user authentication, enquiry of historical data is also obtained from preposition processing module, and data are obtained from cache module and is returned
To preposition processing module.
Service Processing Module further comprises but is not limited to: user authentication unit, historical data unit, K line market list
Member, data analysis unit, returns and surveys unit at indicator calculating unit.
User authentication unit is used to identify the permission of user accesses data.The specific processing identified are as follows: user authentication list
Member receive from each front end units of preposition processing module (including general market front end units, inquiry data front end units or
Historical data front end units) request that sends, inquire whether user has corresponding data access authority from cache module,
Query result return is sent to preposition processing module.
K line market unit is for the calculation processing to K line and timesharing.K line market unit is specifically handled are as follows: K line market
Unit obtains each exchange's market data from messaging bus, and data are carried out SBE volume after carrying out K line, the corresponding algorithm of timesharing
Code, is sent to general market front end units in real time, and at the same time being written to cache module for inquiring data front end units, history
Data front end units obtain data.User only need to subscribe to K line, time data by user's api interface module or
Inquiry can take data and carry out front end displaying, calculate without oneself.
Data analysis unit is for analyzing the data such as market.Data analysis unit is specifically handled are as follows: data point
It analyses unit and obtains each exchange's market data, other open market data, public sentiment datas etc. from messaging bus, taken from cache module
Historical data is obtained, calculate according to preset algorithm and data are inquired into the data of calculated result storage to cache module
Front end units, historical data front end units obtain data.For example, stream of the data analysis unit for Chinese Financial Futures Market exchange
Dynamic property analysis, the liquidity scale as calculated result can help user understand the product in the market mobility how.
Historical quotes unit is implemented for storing historical quotes are as follows: historical quotes unit each day of trade from
Messaging bus obtains market data in real time, writes data into cache module, and additionally lands portion according to preset format
File.
Indicator calculating unit is specific to handle for calculating to the various indexs of market are as follows: indicator calculating unit is from disappearing
Breath bus obtains each exchange's market data, calculated according to the different algorithm of various indexs (such as power refers to relatively
Mark, random index, trend index, smooth and similar average line etc.), calculated result is finally written to cache module.It is logical in user
Cross that user's api interface module is subscribed to or inquiry obtains data, these calculated results can be shown together with K line etc. in front end,
It can be used for the analysis of strategies of oneself.
It returns survey unit and withdraws the data such as rate for calculating rate of return (RMT), maximum under certain condition, it is specific to handle are as follows: time
Unit is surveyed according to the input of user, the historical quotes since input time point are obtained from cache module, further according to specified stock
Ticket combination is selected stocks, and buy in selling according to trading rules, finally calculates rate of return (RMT) in a period, most
The data such as rate are withdrawn greatly and are exported to user.
Preposition processing module further comprises: general market front end units, inquiry data front end units and historical data
Front end units.
General market front end units are specific to handle for pushing real time data are as follows: general market front end units, which receive, to be used
The subscription of family api interface module is requested, and after the data authentication by user authentication unit, is established with user's api interface module
Long connection, obtains data from messaging bus, cache module, and the theme subscribed to according to user is to corresponding user's api interface
The push of module progress different data.These themes include but is not limited to: by exchange's market theme, single contract market master
Topic, exchange K line theme, single and about K line theme etc..
Inquiry data front end units are specific to handle for inquiring same day data are as follows: inquiry data front end units, which receive, to be used
The inquiry same day of family api interface module specifies the request of data, by obtaining after the authentication of user authentication unit from cache module
Data return to user's api interface module.
Historical data front end units are used for enquiry of historical data, specific to handle are as follows: historical data front end units, which receive, to be used
The query history of family api interface module specifies the request of data, by obtaining after the authentication of user authentication unit from cache module
Data return to user's api interface module, if the historical data obtained will directly return to file download before certain time
Link for Customer Acquisition data.
User's api interface module is used for subscriber access system, and the interface of setting includes but is not limited to: API management connects
Mouth, request interface, data callback interface.User's api interface module inputs user name, password, the address fens for user, in user
The users such as Client-initiated user login, data subscription, data query request is initialized and successively received after inputting above- mentioned information,
Data can return in data callback interface, and user handles the data oneself received in data callback interface.
Fens module provides load balancing service, specific processing mode for preposition processing module are as follows: fens module passes through
It is communicated with each preposition processing module and obtains the current loading condition of each preposition processing module, select to bear in preposition processing module
The smallest preposition address is carried, user's api interface module is returned to and carries out data communication.
Fig. 2 shows the data subscription process of system configuration of the invention, system obtains the propelling datas such as market and mainly walks
It is rapid that details are as follows:
Step S11: server-side operation: starting exchange's ticker process of this system first, and exchange can starting row
Feelings push.In the process, due to being the form using dynamic link library, optional number can be configured for a ticker process
The reception of exchange is measured, disposing an exchange respectively if there is more machines can not influence mutually, and obtain faster
Speed.After received market of trading, exchange's data are directly copied, are calculated by the core services module in system,
Finally encoded using SBE, it will be after coding using UDP (User Datagram Protocol, User Datagram Protocol)
Data are sent to preposition processing module.Internal high-performance frame is relied on, UDP is communicated up to 5,000,000 QPS (Queries Per
Second, inquiry number per second), significantly reduce communication delay.The data issued out are received by historical data unit simultaneously,
After SBE decoding process, part hot spot data is stored in the data query in cache module for low latency, remainder data will be deposited
Enter in database module and file.
Step S12: user calls user's api interface module of this system: as shown in table 1, this system provides several be easy to
The api interface used.It is worth noting that the sequence of calling interface: 1. inheriting CFfexGmdDyMdUserSpi, realize wherein
4 interfaces, the data process method of oneself is added in call back function；2. being created using CreateGmdDyMdUserApi real
Example；3. registering front server address using RegisterFrontAddress, obtain current API's using GetVersion
Version；4. receiving the type of data using SetExtConfig setting, the data after alternative construction body, text, SBE coding；⑤
The Data subject of oneself needs is registered using RegisterTopics, while can subscribe to multiple themes；6. using
The example realized in the RegisterSpi registration first step；7. starting to start API using Start and Join and API being waited to execute
It finishes；8. Release can be called to delete example after having used API.
Step S13: processing user's request: after correctly calling user's api interface module in step s 12, preposition processing mould
Block receives the subscription market request of user, then transmits data in user authentication unit and is verified, preposition processing module
It receives and returns.According to verifying as a result, if permission or user's error will send out error message to user's api interface module, at this time
Detailed error information will be returned in the OnEvent interface of user.
Step S14: market data-pushing: in step s 13 if verifying all passes through, preposition processing module and user API
Interface module establishes connection, and according to the theme propelling data that user subscribes to, user inherits and realizes in step S12 at this time
Interface in CFfexGmdDyMdUserSpi will all have data return.Preferably, optional in the 4. operation of step S12
Selection is encoded with SBE in mode, will obtain faster speed.
Fig. 3 shows the data query process that an embodiment of big data market plateform system of the invention is configured.Please
The implementation steps of the inquiry data such as historical quotes are obtained referring to Fig. 3, in the present embodiment, and details are as follows.
Step S21: identical as the step S11 in Fig. 2.That is, server-side is run: starting exchange's market of this system first
Receiving process, exchange can start market push.It in the process, can be one due to being the form using dynamic link library
A ticker process configures the reception of optional quantity exchange, and disposing an exchange respectively if there is more machines can be mutual
It does not influence, and obtains faster speed.After received market of trading, the core services module in system will directly be handed over
Easy institute data are copied, are calculated, and are finally encoded using SBE, use UDP (User Datagram Protocol, user
Datagram protocol) by the data after coding send preposition processing module.Internal high-performance frame is relied on, UDP communication is up to 500
Ten thousand QPS (Queries Per Second, inquiry number per second), significantly reduce communication delay.The data while quilt issued out
Historical data unit is received, and after SBE decoding process, part hot spot data is stored in the number in cache module for low latency
It is investigated that asking, remainder data is stored into database module and file.
Step S22: user calls user's api interface module of this system.As shown in table 1, this system provides several be easy to
The interface used.It is worth noting that the sequence of calling interface: 1. inheriting CFfexGmdDyMdUserSpi, realize therein 4
The data process method of oneself is added in a interface in call back function；2. creating example using CreateGmdDyMdUserApi；
3. registering front server address using RegisterFrontAddress, the version of current API is obtained using GetVersion
This；4. receiving the type of data using SetExtConfig setting, the data after alternative construction body, text, SBE coding；5.
The data that oneself Calling SendRequest interface requests oneself needs；6. using in the RegisterSpi registration first step
The example of realization；7. starting to start API using Start and Join and API being waited to be finished；8. adjustable after having used API
Example is deleted with Release.
Step S23: processing user's request.After correctly calling user's api interface module in step S22, preposition processing mould
Block receives the request of user query data, sends checking request then to user authentication unit, is looked into after verifying is legal according to request
Different processing units of the data type of inquiry into Service Processing Module issue inquiry request of data, such as historical data, K line
Calculating, timesharing calculating, data analysis etc..Some of them processing unit can be gone from file, database module, cache module
Data required for obtaining.And many hot spot datas are stored in cache module, the speed for inquiring reading is very fast, reduces
The delay of data acquisition.Finally, preposition processing module obtains the data that inquiry returns.
Step S24: query result returns.In step S23, if Authority Verification is illegal, by hair error message to use
Family api interface module will return to detailed error information in the OnEvent interface of user at this time.If the verification passes, data at this time
The interface for the CFfexGmdDyMdUserSpi for inheriting and realizing by user in step S22 is returned.
Although for simplify explain the above method is illustrated to and is described as a series of actions, it should be understood that and understand,
The order that these methods are not acted is limited, because according to one or more embodiments, some movements can occur in different order
And/or with from it is depicted and described herein or herein it is not shown and describe but it will be appreciated by those skilled in the art that other
Movement concomitantly occurs.
Those skilled in the art will further appreciate that, the various illustratives described in conjunction with the embodiments described herein
Logic plate, module, circuit and algorithm steps can be realized as electronic hardware, computer software or combination of the two.It is clear
Explain to Chu this interchangeability of hardware and software, various illustrative components, frame, module, circuit and step be above with
Its functional form makees generalization description.Such functionality be implemented as hardware or software depend on concrete application and
It is applied to the design constraint of total system.Technical staff can realize every kind of specific application described with different modes
Functionality, but such realization decision should not be interpreted to cause departing from the scope of the present invention.
General place can be used in conjunction with various illustrative logic plates, module and the circuit that presently disclosed embodiment describes
Reason device, digital signal processor (DSP), specific integrated circuit (ASIC), field programmable gate array (FPGA) other are compiled
Journey logical device, discrete door or transistor logic, discrete hardware component or its be designed to carry out function described herein
Any combination is realized or is executed.General processor can be microprocessor, but in alternative, which, which can be, appoints
What conventional processor, controller, microcontroller or state machine.Processor is also implemented as calculating the combination of equipment, example
Such as DSP and the combination of microprocessor, multi-microprocessor, one or more microprocessors to cooperate with DSP core or any
Other such configurations.
The step of method or algorithm for describing in conjunction with embodiment disclosed herein, can be embodied directly in hardware, in by processor
It is embodied in the software module of execution or in combination of the two.Software module can reside in RAM memory, flash memory, ROM and deposit
Reservoir, eprom memory, eeprom memory, register, hard disk, removable disk, CD-ROM or known in the art appoint
In the storage medium of what other forms.Exemplary storage medium is coupled to processor so that the processor can be from/to the storage
Medium reads and writees information.In alternative, storage medium can be integrated into processor.Pocessor and storage media can
It resides in ASIC.ASIC can reside in user terminal.In alternative, pocessor and storage media can be used as discrete sets
Part is resident in the user terminal.
In one or more exemplary embodiments, described function can be in hardware, software, firmware, or any combination thereof
Middle realization.If being embodied as computer program product in software, each function can be used as one or more item instructions or generation
Code may be stored on the computer-readable medium or be transmitted by it.Computer-readable medium includes computer storage medium and communication
Both media comprising any medium for facilitating computer program to shift from one place to another.Storage medium can be can quilt
Any usable medium of computer access.It is non-limiting as example, such computer-readable medium may include RAM, ROM,
EEPROM, CD-ROM or other optical disc storages, disk storage or other magnetic storage apparatus can be used to carry or store to refer to
The desirable program code of order or data structure form and any other medium that can be accessed by a computer.Any connection is also proper
Ground is known as computer-readable medium.For example, if software is using coaxial cable, fiber optic cables, twisted pair, digital subscriber line
(DSL) or the wireless technology of such as infrared, radio and microwave etc is passed from web site, server or other remote sources
It send, then the coaxial cable, fiber optic cables, twisted pair, DSL or such as infrared, radio and microwave etc is wireless
Technology is just included among the definition of medium.Disk (disk) and dish (disc) as used herein include compression dish
(CD), laser disc, optical disc, digital versatile disc (DVD), floppy disk and blu-ray disc, which disk (disk) are often reproduced in a manner of magnetic
Data, and dish (disc) with laser reproduce data optically.Combinations of the above should also be included in computer-readable medium
In the range of.
Offer is to make any person skilled in the art all and can make or use this public affairs to the previous description of the disclosure
It opens.The various modifications of the disclosure all will be apparent for a person skilled in the art, and as defined herein general
Suitable principle can be applied to other variants without departing from the spirit or scope of the disclosure.The disclosure is not intended to be limited as a result,
Due to example described herein and design, but should be awarded and principle disclosed herein and novel features phase one
The widest scope of cause.
Claims (8)
1. a kind of big data market plateform system, which is characterized in that including core services module, database module, cache module,
Messaging bus, Service Processing Module, preposition processing module, user's api interface module, in which:
Core services module includes that initial data receives resolution unit, SBE coding unit, divides theme release unit, wherein original
Data receiver resolution unit is used for from the acquired raw line feelings data of transaction and carries out data parsing, and SBE coding unit is for adopting
SBE coding is carried out to the original market data that initial data receiving unit receives and parses through out with simple binary coding SBE, point
Theme release unit is divided into different themes for the market data after encoding SBE and is published to messaging bus；
Cache module is obtained for the market data after obtaining SBE coding on messaging bus or from Service Processing Module
Data that treated, are stored in itself memory and/or database module, and, receive from Service Processing Module or
The request of the reading data of preposition processing module is obtained from the memory of itself and data and is returned, if in memory not if
Inquiry is in database module to return to the data inquired；
User's api interface module, is used for subscriber access system, and user initiates to include that user logs in, data are ordered by api interface
It reads, the request including data query, preposition processing module is responded the request treated that data return to user；
Preposition processing module receives user and passes through user API for pushing real time data, inquiry same day data and historical data
The request and response that interface module is sent；
Service Processing Module is corresponded to after obtaining storing data from cache module for obtaining market data from messaging bus
Processing, is sent to cache module for processing result and stores, and receiving also from preposition processing module includes user authentication, inquiry
Request including historical data obtains data from cache module and returns to preposition processing module.
2. big data market plateform system according to claim 1, which is characterized in that core services module and messaging bus
One-way data transfer is carried out, messaging bus is unidirectionally counted with preposition processing module, cache module and Service Processing Module respectively
According to transmission, cache module carries out bidirectional data transfers, cache module and preposition place with database module, Service Processing Module respectively
It manages module and carries out one-way data transfer.Preposition processing module carries out with Service Processing Module, user's api interface module two-way respectively
Data transmission.
3. big data market plateform system according to claim 1, which is characterized in that system further include:
Fens module and preposition processing module carry out bidirectional data transfers, provide load balancing service for preposition processing module.
4. big data market plateform system according to claim 3, which is characterized in that the load configured in fens module is equal
Weighing apparatus service is to obtain the current loading condition of each preposition processing module by communicating with each preposition processing module, is selected preposition
The smallest preposition address is loaded in processing module, carries out data communication back to user's api interface module.
5. big data market plateform system according to claim 1, which is characterized in that the number after SBE coding in system
According to being transmitted using udp protocol.
6. big data market plateform system according to claim 1, which is characterized in that preposition processing module includes general row
Feelings front end units, inquiry data front end units and historical data front end units, in which:
General market front end units receive the subscription request of user's api interface module, reflect by the data of user authentication unit
Quan Hou establishes long connection with user's api interface module, data is obtained from messaging bus, cache module, and subscribe to according to user
Theme to corresponding user's api interface module carry out different data push；
Data front end units are inquired, the inquiry same day for receiving user's api interface module specifies the request of data, passes through user
Data, which are obtained, from cache module after the authentication of authentication unit returns to user's api interface module；
Historical data front end units, the query history for receiving user's api interface module specify the request of data, pass through user
Data are obtained from cache module after the authentication of authentication unit and return to user's api interface module, if the historical data obtained exists
Before certain time, then the link of file download is directly returned for Customer Acquisition data.
7. big data market plateform system according to claim 1, which is characterized in that Service Processing Module includes but unlimited
In user authentication unit, historical data unit, K line market unit, indicator calculating unit, data analysis unit, returns and surveys unit,
In:
User authentication unit identifies for the permission to user accesses data；
K line market unit, for the calculation processing to K line and timesharing；
Data analysis unit, for analyzing data such as market；
Historical quotes unit, for storing historical quotes；
Indicator calculating unit, for the calculating to the various indexs of market；
Return and survey unit, for calculate under certain condition including rate of return (RMT), it is maximum withdraw rate including return measured data.
8. big data market plateform system according to claim 1, which is characterized in that system is distributed structure/architecture, deployment
In multiple nodes, automatically switched by hot standby realization failure in real time.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910640372.3A CN110415027B (en) | 2019-07-16 | 2019-07-16 | Big data quotation platform system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910640372.3A CN110415027B (en) | 2019-07-16 | 2019-07-16 | Big data quotation platform system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110415027A true CN110415027A (en) | 2019-11-05 |
| CN110415027B CN110415027B (en) | 2023-05-26 |
Family
ID=68361685
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910640372.3A Active CN110415027B (en) | 2019-07-16 | 2019-07-16 | Big data quotation platform system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110415027B (en) |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111709839A (en) * | 2020-06-19 | 2020-09-25 | 上海金融期货信息技术有限公司 | Wind control complex service implementation system and method based on clearing core |
| CN112116466A (en) * | 2020-06-11 | 2020-12-22 | 上海金融期货信息技术有限公司 | Unified platform system for stock futures financial data exchange |
| CN112132681A (en) * | 2020-07-02 | 2020-12-25 | 上海金融期货信息技术有限公司 | Market source unified access system |
| CN112347020A (en) * | 2020-10-26 | 2021-02-09 | 东方证券股份有限公司 | FAST market analysis system and method based on CGRA |
| CN113065958A (en) * | 2021-02-20 | 2021-07-02 | 上海金融期货信息技术有限公司 | Theme market situation business data streaming system |
| CN114124668A (en) * | 2021-11-03 | 2022-03-01 | 上证所信息网络有限公司 | System and method for ensuring consistency of market quotation slices of multiple hosts |
| CN114331445A (en) * | 2022-03-15 | 2022-04-12 | 上海金仕达软件科技有限公司 | API (application programming interface), method, storage medium and electronic equipment for accessing massive users |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20070288662A1 (en) * | 2006-06-12 | 2007-12-13 | Telepaq Technology Inc. | Distributed push-pull information service system |
| CN103247003A (en) * | 2013-05-15 | 2013-08-14 | 上海大智慧股份有限公司 | Distributed program trading system oriented to event processing |
| CN106294486A (en) * | 2015-06-04 | 2017-01-04 | 交通银行股份有限公司 | The processing method of finance market data and system |
| WO2017008604A1 (en) * | 2015-07-15 | 2017-01-19 | 中兴通讯股份有限公司 | Data processing method, device and system |
| CN108269188A (en) * | 2016-12-30 | 2018-07-10 | 上海金融期货信息技术有限公司 | A kind of exchange's quotation information processing method and system based on FPGA |
-
2019
- 2019-07-16 CN CN201910640372.3A patent/CN110415027B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20070288662A1 (en) * | 2006-06-12 | 2007-12-13 | Telepaq Technology Inc. | Distributed push-pull information service system |
| CN103247003A (en) * | 2013-05-15 | 2013-08-14 | 上海大智慧股份有限公司 | Distributed program trading system oriented to event processing |
| CN106294486A (en) * | 2015-06-04 | 2017-01-04 | 交通银行股份有限公司 | The processing method of finance market data and system |
| WO2017008604A1 (en) * | 2015-07-15 | 2017-01-19 | 中兴通讯股份有限公司 | Data processing method, device and system |
| CN108269188A (en) * | 2016-12-30 | 2018-07-10 | 上海金融期货信息技术有限公司 | A kind of exchange's quotation information processing method and system based on FPGA |
Non-Patent Citations (1)
| Title |
|---|
| 张靖宇等: "高性能大宗商品行情报价系统设计与研究", 《计算机应用与软件》 * |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112116466A (en) * | 2020-06-11 | 2020-12-22 | 上海金融期货信息技术有限公司 | Unified platform system for stock futures financial data exchange |
| CN111709839A (en) * | 2020-06-19 | 2020-09-25 | 上海金融期货信息技术有限公司 | Wind control complex service implementation system and method based on clearing core |
| CN111709839B (en) * | 2020-06-19 | 2023-11-14 | 上海金融期货信息技术有限公司 | Wind control complex service implementation system and method based on clearing core |
| CN112132681A (en) * | 2020-07-02 | 2020-12-25 | 上海金融期货信息技术有限公司 | Market source unified access system |
| CN112132681B (en) * | 2020-07-02 | 2024-03-26 | 上海金融期货信息技术有限公司 | Unified access system for quotation sources |
| CN112347020A (en) * | 2020-10-26 | 2021-02-09 | 东方证券股份有限公司 | FAST market analysis system and method based on CGRA |
| CN113065958A (en) * | 2021-02-20 | 2021-07-02 | 上海金融期货信息技术有限公司 | Theme market situation business data streaming system |
| CN114124668A (en) * | 2021-11-03 | 2022-03-01 | 上证所信息网络有限公司 | System and method for ensuring consistency of market quotation slices of multiple hosts |
| CN114331445A (en) * | 2022-03-15 | 2022-04-12 | 上海金仕达软件科技有限公司 | API (application programming interface), method, storage medium and electronic equipment for accessing massive users |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110415027B (en) | 2023-05-26 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110415027A (en) | A kind of big data market plateform system | |
| CN109246211B (en) | Resource uploading and resource requesting method in block chain | |
| CN103329113B (en) | Configuration is accelerated and custom object and relevant method for proxy server and the Dynamic Website of hierarchical cache | |
| CN103716326B (en) | Resource access method and URG | |
| US8738741B2 (en) | Brokering network resources | |
| US7249195B2 (en) | Apparatus and methods for correlating messages sent between services | |
| CN109937420B (en) | De-identified distributed bridging network platform | |
| CN109995713A (en) | Service processing method and relevant device in a kind of micro services frame | |
| EP1517258A2 (en) | Content download to wireless devices | |
| BRPI0612847A2 (en) | method for switching transactions using a connection point, system for switching customer transactions, and connection point | |
| CN109005220B (en) | Service implementation method of Internet of things terminal and Internet of things access gateway | |
| CN102257777A (en) | Route decision method, content distribution device, content distribution network interconnection system | |
| CN110110269A (en) | A kind of event subscription method and device based on block chain | |
| US20070192431A1 (en) | Method and apparatus for service oriented architecture infrastructure switch | |
| US20200014632A1 (en) | Resource path monitoring | |
| KR20170062244A (en) | Api managing apparatus | |
| CN103428041B (en) | A kind of end-to-end flux content detection system and detection method based on cloud | |
| CN114363352A (en) | Block chain-based Internet of things system cross-chain interaction method | |
| US20090158047A1 (en) | High performance secure caching in the mid-tier | |
| US20220414088A1 (en) | Performance efficient blockchain application programming interfaces | |
| KR101653685B1 (en) | Computer-excutable method for managing api | |
| US7464070B2 (en) | Database query operations using storage networks | |
| CN112100590A (en) | Tourism big data cloud platform and user authority management method thereof | |
| US20080114767A1 (en) | Trading Files Via Locking and Unlocking | |
| CN110515975A (en) | Risk detecting system, method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |