CN108874962A - A kind of finance big data processing method - Google Patents
A kind of finance big data processing method Download PDFInfo
- Publication number
- CN108874962A CN108874962A CN201810578953.4A CN201810578953A CN108874962A CN 108874962 A CN108874962 A CN 108874962A CN 201810578953 A CN201810578953 A CN 201810578953A CN 108874962 A CN108874962 A CN 108874962A
- Authority
- CN
- China
- Prior art keywords
- data
- financial market
- financial
- market
- converted
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q40/00—Finance; Insurance; Tax strategies; Processing of corporate or income taxes
- G06Q40/04—Trading; Exchange, e.g. stocks, commodities, derivatives or currency exchange
Landscapes
- Business, Economics & Management (AREA)
- Accounting & Taxation (AREA)
- Finance (AREA)
- Engineering & Computer Science (AREA)
- Development Economics (AREA)
- Economics (AREA)
- Marketing (AREA)
- Strategic Management (AREA)
- Technology Law (AREA)
- Physics & Mathematics (AREA)
- General Business, Economics & Management (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Financial Or Insurance-Related Operations Such As Payment And Settlement (AREA)
Abstract
The invention discloses a kind of financial big data processing methods, include the following steps：1) financial market are obtained by a data from one or more data sources；2) the financial market are extracted and is parsed by a data, to obtain one or more data fields；3) one or more of data fields are converted according to predefined data format；4) financial market data are obtained by data according to converted financial market and the financial market data is stored in storage equipment and send the financial market data to user, one aspect of the present invention can carry out storage compression to data file on hard disk, save the hard disk resources of server, on the other hand when carrying out network transmission, efficiency of transmission can also be greatly improved.Therefore it can receive as early as possible and handle data, to carry out dealing decision in time, help to improve investment achievement.
Description
Technical field
The present invention relates to financial technology fields, and in particular to a kind of finance big data processing method.
Background technique
In financial field, financial market data class is various, such as stock, futures, fund, noble metal, foreign exchange etc..
The variation of financial market data is able to reflect the turn of the market in a certain field, while being also the basis of customer transaction.For amount
Change transaction, to real-time and historical financial market data readings and analysis, facilitates User Exploitation, management, maintenance or modification and hand over
Easily strategy, to make correctly dealing decision.
Traditionally, financial market data are stored in third party database, can be read from database when user needs
Out.Database is usually the relevant database such as mysql or using redis as the memory database of representative.These data
Library is often individual server, and the communication between quotation service and database server occupies certain time.Also, data
Library needs separately installed, deployment, O&M, spends human and material resources；Database be abnormal it is unpredictable, will lead to data lack
It loses；Over time, the data accumulated over a long period can sharply expand, and incite somebody to action the migration of data, backup, management and its troublesome.
In addition, data store internal has done synchronization to multithreading access, it is inefficient.And for relevant database, data store internal
Many references can be interted in design, to rapidly locating.Therefore, at least due to the above reason, uses traditional data
It is slow that library will lead to the financial market data of reading, thus is perhaps able to satisfy and needs when obtaining minute grade or the other data of even lower level
It wants, but if on the level of high frequency (such as mono- data of 500ms by a data (tick data)), it is clear that it is not able to satisfy
Demand.
Summary of the invention
(1) technical problems to be solved
In order to overcome the shortage of prior art, it is proposed that a kind of finance big data processing method, it on the one hand can be on hard disk
Storage compression is carried out to data file, saves the hard disk resources of server, it, can also on the other hand when carrying out network transmission
To greatly improve efficiency of transmission.Therefore it can receive as early as possible and handle data, to carry out dealing decision in time, help to improve
Invest achievement.
(2) technical solution
The present invention is achieved through the following technical solutions：The invention proposes a kind of financial big data processing methods, including such as
Lower step：1) financial market are obtained by a data from one or more data sources；2) the financial market are carried out by data
It extracts and parses, to obtain one or more data fields；3) by one or more of data fields according to predefined number
It is converted according to format；4) according to converted financial market by data financial market data are obtained and by the financial market
Data are stored in storage equipment and send the financial market data to user.
Further, the converted financial market are stored directly in by data as the financial market data
In the storage equipment, and/or by more converted financial market acquired in predetermined time section by a Data Synthesis
For interval censored data and store in the storage device.
Further, after obtaining the financial market by data from one or more of data sources, to described
Financial market are pre-processed by a data, and the pretreatment includes：The financial market are filtered by a data and/or
The financial market are converted by the time format of data.
(3) beneficial effect
The present invention compared with the existing technology, has the advantages that：
A kind of financial big data processing method that the present invention mentions, by the storage of financial market data at binary file without
It is stored in database, on the one hand storage compression can be carried out to data file on hard disk in this way, save the hard of server
Disk resource can also greatly improve efficiency of transmission on the other hand when carrying out network transmission.Therefore it can receive as early as possible simultaneously
It handles data and helps to improve investment achievement to carry out dealing decision in time.
Specific embodiment
In order to make the objectives, technical solutions, and advantages of the present invention clearer, with reference to embodiments, to the present invention
It is further elaborated.It should be appreciated that the specific embodiments described herein are merely illustrative of the present invention, it is not used to
Limit the present invention.
A kind of finance big data processing method, includes the following steps：1) financial market are obtained from one or more data sources
By a data；2) the financial market are extracted and is parsed by a data, to obtain one or more data fields；3) will
One or more of data fields are converted according to predefined data format；4) according to converted financial market by pen
Data obtain financial market data and the financial market data are stored in storage equipment and send the gold to user
Melt market data.
Wherein, the converted financial market are stored directly in by data as the financial market data described
It stores in equipment, and/or more converted financial market acquired in predetermined time section is synthesized into area by a data
Between data and store in the storage device；The financial market are being obtained by a data from one or more of data sources
Later, the financial market are pre-processed by a data, the pretreatment includes：To the financial market by data into
Row is filtered and/or is converted to the financial market by the time format of data.
A kind of financial big data processing method that the present invention mentions, in one embodiment, can only by financial market by
Data are stored in memory can only deposit the interval censored data of synthesis in another embodiment without synthesizing interval censored data
Storage is in memory.In other embodiments, more usually situation is, has both needed by a data, it is also desirable to interval censored data.Therefore both
It needs to store by a data, it is also desirable to more in predetermined time section be synthesized interval censored data by a data and stored
With for users to use.
The above-described embodiments are merely illustrative of preferred embodiments of the present invention, not to structure of the invention
Think and range is defined.Without departing from the design concept of the invention, ordinary people in the field is to technology of the invention
The all variations and modifications that scheme is made, should all drop into protection scope of the present invention, the claimed technology contents of the present invention,
It is all described in the claims.
Claims (3)
1. a kind of finance big data processing method, it is characterised in that：Include the following steps：1) it is obtained from one or more data sources
Financial market are by a data；2) the financial market are extracted and is parsed by a data, to obtain one or more data
Field；3) one or more of data fields are converted according to predefined data format；4) according to converted gold
Melt market to obtain financial market data by data and the financial market data are stored in storage equipment and to user
Send the financial market data.
2. a kind of financial big data processing method according to claim 1, it is characterised in that：By the converted finance
Market are stored directly in the storage equipment by data as the financial market data, and/or by predetermined time section
Interior more acquired converted financial market synthesize interval censored data by a data and store in the storage device.
3. a kind of financial big data processing method according to claim 1, it is characterised in that：From one or more of
After data source obtains the financial market by data, the financial market are pre-processed by a data, the pre- place
Reason includes：The financial market are filtered by data and/or to the financial market by data time format into
Row conversion.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810578953.4A CN108874962A (en) | 2018-06-07 | 2018-06-07 | A kind of finance big data processing method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810578953.4A CN108874962A (en) | 2018-06-07 | 2018-06-07 | A kind of finance big data processing method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN108874962A true CN108874962A (en) | 2018-11-23 |
Family
ID=64337105
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810578953.4A Pending CN108874962A (en) | 2018-06-07 | 2018-06-07 | A kind of finance big data processing method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108874962A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110830437A (en) * | 2019-09-25 | 2020-02-21 | 平安科技（深圳）有限公司 | Data compression method, device, equipment and storage medium for high-frequency service data |
-
2018
- 2018-06-07 CN CN201810578953.4A patent/CN108874962A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110830437A (en) * | 2019-09-25 | 2020-02-21 | 平安科技（深圳）有限公司 | Data compression method, device, equipment and storage medium for high-frequency service data |
| CN110830437B (en) * | 2019-09-25 | 2022-06-10 | 平安科技（深圳）有限公司 | Data compression method, device, equipment and storage medium for high-frequency service data |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Johannes et al. | Financial development and economic growth in Cameroon, 1970-2005 | |
| Kim | Macroeconomic effects of household debt: an empirical analysis | |
| Kanbur et al. | Spatial inequality and development | |
| CN102496126B (en) | Custody asset transaction data monitoring equipment | |
| CN106776837A (en) | A kind of method of the security real-time deal association analysis based on MongoDB | |
| CN107748797A (en) | The processing method of financial market data | |
| Görg et al. | Multinational companies, technology spillovers and firm survival: Evidence from Irish manufacturing | |
| Karahan et al. | Financial deepening and economic growth in Turkey | |
| CN104112010A (en) | Data storage method and device | |
| KR101435096B1 (en) | Apparatus and method for prediction of merchandise demand using social network service data | |
| CN110471903A (en) | Heterogeneous system nodal information method of summary, data library generating method, data creation method and device | |
| Mariana | Mergers and acquisitions waves from the European Union perspective | |
| CN108874962A (en) | A kind of finance big data processing method | |
| Avouyi-Dovi et al. | Interactions between business cycles, financial cycles and monetary policy: stylised facts | |
| CN106383911A (en) | Contracted order compression method and system | |
| Fasanya et al. | Trivariate analysis of oil revenue, government spending and economic growth in Nigeria | |
| Ventosa‐Santaulària et al. | Recessions and potential GDP: The case of Mexico | |
| Gupta | Trends and macro-economic determinants of FDI inflows to India | |
| Sharma et al. | Evidence on relationship between economic growth and financial development of selected northern Indian states: using panel VEC approach | |
| Owolabi et al. | Economic growth may affect agriculture sector output: evidence from Nigeria | |
| CN114647555B (en) | Data early warning method, device, equipment and medium based on multi-service system | |
| Gharaibeh et al. | Factors affecting profitability of the Jordanian banking sector | |
| Deeken et al. | Integration as a spatial institution: Implications for agglomeration and growth | |
| Bibas et al. | Modelling the circular economy: designing a global baseline of economic activity and material flows | |
| Srivastava et al. | ASSESSING THE IMPACT OF CENTRAL BANKING POLICIES ON FINANCIAL MARKET PERFORMANCE IN EMERGING ECONOMIES: A CASE STUDY OF INDIA |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20181123 |
|
| WD01 | Invention patent application deemed withdrawn after publication |