CN109063167A - A kind of unstructured finance data storage method - Google Patents
A kind of unstructured finance data storage method Download PDFInfo
- Publication number
- CN109063167A CN109063167A CN201810933306.0A CN201810933306A CN109063167A CN 109063167 A CN109063167 A CN 109063167A CN 201810933306 A CN201810933306 A CN 201810933306A CN 109063167 A CN109063167 A CN 109063167A
- Authority
- CN
- China
- Prior art keywords
- fund
- information
- unstructured
- finance data
- record
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q40/00—Finance; Insurance; Tax strategies; Processing of corporate or income taxes
- G06Q40/06—Asset management; Financial planning or analysis
Landscapes
- Engineering & Computer Science (AREA)
- Business, Economics & Management (AREA)
- Finance (AREA)
- Accounting & Taxation (AREA)
- Development Economics (AREA)
- Operations Research (AREA)
- Game Theory and Decision Science (AREA)
- Human Resources & Organizations (AREA)
- Entrepreneurship & Innovation (AREA)
- Economics (AREA)
- Marketing (AREA)
- Strategic Management (AREA)
- Technology Law (AREA)
- Physics & Mathematics (AREA)
- General Business, Economics & Management (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a kind of unstructured finance data storage methods, comprising: database and operating system, the finance data storing step includes: 1) to store funds data in the database；2) fund document is created in operating system memory；3) a line record information that different field compositions is not fixed column is recorded according to fund information；4) whether fund information record includes fund information of holding position；If it is, entering in next step；It 5) is fund information record creation index；6) finance data information is inquired.The present invention provides a kind of storage of finance data and inquiry is more convenient, and a kind of low, the high unstructured finance data storage method of search efficiency is lost in query performance.
Description
Technical field
The present invention relates to field of computer technology more particularly to a kind of unstructured finance data storage methods.
Background technique
Storage for finance data presently mainly passes through relevant database, such as Microsoft
The relevant databases such as SQLServer/Oracle/MySQL.For relevant database, the mode of storing data are as follows:
1. establishing database.
2. design data table structure.When design data table structure, for the data of different scenes, due to relational data
Library requires column name predefined good, will fix in design phase table structure.
3. after designing database table structure, what data must be given in strict accordance with the structure set by database
API is written to inside database.
For different data class, need strictly to carry out division table according to type.
Since relevant database is substantially a kind of general data storage solution party inside traditional relational data
Case is closed to guarantee that relevant database can obtain relatively good usage experience in multiple occasions such as finance, industry, agriculturals
It is that type database finally can be all landed on physical disk in real time to obtain the optimal of comprehensive experience.
The prior art is largely able to solve finance data storage and efficiency using relevant database as core
Problem.But existing technology equally has several obvious defects:
1. relational data when carrying out data query, needs to be associated inquiry by way of associate field.And
Data for being under the jurisdiction of the same logic are actually potentially stored in complete different place.Such as investment manager needs to obtain
The information of fund is taken, then just needing the information for inquiring fund inside database according to the information association of fund to fund asset
Table obtains fund asset information, is associated with fund table of holding position according to fund information table and gets the information that fund holds position.Actually
Fund asset and fund hold position the position stored on disk be it is uncertain, this has been decided in its sole discretion most by relevant database
Excellent storage mode
2. the field that same type of table is related to also can be different due to the difference of financial business.Such as stock and phase
Goods, futures possess warehousing orientation and occupy the attribute of guarantee fund, and stock is just not present.Dividing table storage bring is the unification of data
(data of holding position have been split to two tables), same table store the problem of bringing field redundancies again (relational database table knot
Structure is fixed).
3. the considerations of due to relational data itself for versatility, all data finally can all unify landing and arrive physics
Disk, and need to inquire every time when, equally also must first load data into from physical disk and carry out Directory Enquiries
In reason, generation, have carried out the huge loss of query performance, especially when financial transaction data runs up to hundreds of thousands, up to a million or even up to ten million
When, this access physical disk inquiry bring loss is then more obvious.
Chinese Patent Application No. is 201410234903.6, the applying date: on May 29th, 2014, publication date: 2014 10
The moon 01, patent name is: a kind of method of unstructured data storage and inquiry, the invention provides a kind of unstructured numbers
According to the method, apparatus and system of storage and inquiry, when unstructured data stores, the Master of distributed system receives the industry
Be engaged in metadata information and electron image bill carries out compression processing and obtains compressed data packets, and the Master is by the compressed data
Packet is stored in any one back end for being distributed in the Master, realizes unstructured data using distributed system
Storage, due to the limited storage space of each back end, not will lead to the decline of storage efficiency；Non-structural data enquiry
When, the Master in distributed system receives the inquiry request that enquiry module is sent, according to inquiry entrained in inquiry request
Information is in all back end acquisition of the Master and the matched compressed data packets of the query information, using distribution system
System realizes the inquiry of unstructured data, since the data information that each back end being distributed in Master is stored has
Limit, all back end parallel queries not will lead to search efficiency decline.
Although above patent document discloses the method for a kind of unstructured data storage and inquiry, the unstructured data
Storage method storage mode falls behind, and query performance loss is big, and inquiry is inconvenient, and search efficiency is inconvenient, is not able to satisfy user's
It needs.
Summary of the invention
In view of this, the present invention provides a kind of storage of finance data and inquiry is more convenient, query performance loss is low, inquiry
A kind of high-efficient unstructured finance data storage method.
In order to achieve the object of the present invention, following technical scheme can be taken:
A kind of unstructured finance data storage method, comprising: database and operating system, the finance data storage step
Suddenly include:
Step 1) stores funds data in the database；
Step 2) creates fund document in operating system memory；
Step 3) records a line record information that different field compositions is not fixed column according to fund information；
Whether step 4) fund information record includes fund information of holding position；If it is, entering in next step；
Step 5) is fund information record creation index；
Step 6) inquires finance data information.
Step 3) the record information includes recording information according to fund number or fund flows or fund name.
If the step 4) includes fund information record not comprising fund information of holding position, recorded in information in fund
Create subdocument of holding position.
Creation index described in the step 5) includes creation hash index or creation b-tree indexed.
The step 5) includes the following steps:
1) by the physical disk of index record write operation system；
2) by the physical disk of entire fund recording documents write operation system.
The step 6) includes the following steps:
1) fund record information is loaded into document information；
2) document information whether include inquiry in need document information；If so, then entering in next step；
3) document information being loaded onto is loaded into the memory of operating system by index information；
4) according to match information in memory is indexed, query result is obtained.
If the document information does not include the document information of inquiry in need, inquiry record is not present.
The index includes comment tag in the step 5).
The beneficial effects of the present invention are: 1) present invention can be by database purchase in a base by associated finance data
Inside inscription on ancient bronze objects shelves, facilitate inquiry data information；2) the specific column of the unlimited system of data storage of the present invention, for the data of no respective column
Do not have to directly be loaded into, reduces query performance loss；3) each data is stored in famous mode, and storing data is more convenient,
Search efficiency greatly improves.
Detailed description of the invention
Fig. 1 is a kind of unstructured finance data storage method flow chart of the embodiment of the present invention；
Fig. 2 is a kind of inquiry finance data method flow of unstructured finance data storage method of the embodiment of the present invention
Figure.
Specific embodiment
With reference to the accompanying drawing and the embodiment of the present invention is described in further detail invention.
Embodiment 1
Referring to Fig. 1, a kind of unstructured finance data storage method, comprising: database and operating system, the finance number
Include: according to storing step
Step 1) stores funds data S1 in the database；
Step 2) creates fund document S2 in operating system memory；
Step 3) records a line record information S3 that different field compositions is not fixed column according to fund information；
Whether step 4) fund information record includes the fund information S5 that holds position；If it is, entering in next step；
Step 5) is fund information record creation index S6；
Step 6) inquires finance data information S7.
Preferably, step 3) the record information includes being remembered according to fund number or fund flows or fund name
Record information.
Preferably, if the step 4) includes fund information record not comprising fund information of holding position, remember in fund
The subdocument S51 that holds position is created in record information.
Preferably, creation index described in the step 5) includes creation hash index or creation b-tree indexed.
Financial transaction data of the present invention, the data with certain relevance, the data storage of the relevance is arrived
Place similar in position.For this kind of associated datas, we term it documents.Such as fund information, we are just generated
One is named as the document of fund.Inside each document, following a few class data can be possessed:
A. specific business datum.Such as fund number, fund name, fund flows etc.
B. subdocument.Such as fund is it is considered that fund has fund asset information, has fund to hold position information, then we will
Fund asset, fund flows are embedded into inside the document of fund as subdocument.
The document, which column definition document does not specifically include to the present invention, but actual selection insertion column as needed
Name；The considerations of for reliability, all documents equally can all be written to physical disk, but since related data can be referred to
Inside unified document, when needing to do data query, entire document can be fully loaded to inside memory, inside memory into
Row inquiry and analysis, to bring the promotion in performance.
In the present embodiment, when needing non expendable fund information, one is created in the operating system memory and is named as fund
Document, the inside be it is empty, without any field or actual data information；Believe when needing to be actually inserted into a fund basis
When breath, a fund information record is written to document, the format of fund information record is: fund number: specific fund number
Value, fund flows: specific fund flows numerical value, fund name: specific fund name.
After the insertion of fund basic information, some funds are held position information, some information of not holding position of holding position.For having
The record for information of holding position, we hold position to a data, fund is inserted into inside corresponding fund basic information: fund is held position Ziwen
Shelves.
After have passed through above-mentioned steps, following two records are obtained inside fund document:
Fund number: specific fund number A, fund flows: specific fund flows A, fund name: specific fund name
Claim A；
Fund number: specific fund number B, fund flows: specific fund flows B, fund name: specific fund name
Claim B, fund holds position: fund is held position subdocument B；
Pay attention to just producing the difference with relational data here, it is above-mentioned two record be not possess it is identical
Column, but selection insertion data column according to the actual needs.Meanwhile a Ziwen is inserted inside the record of fund number B
Associated data of the shelves as fund B, that is, there is the mode of document nesting document, this is inside relevant database
It is not allowed to.
We have two records inside the document of the fund, when record number becomes millions of up to ten million data
When, ergodic data is obviously creation impossible, that we are indexed the information of fund inside memory at this time.To fund
Number creation hash index and b-tree indexed.
Embodiment 2
Referring to Fig. 1, the difference is that, the step 5) includes the following steps: with above-described embodiment
1) by the physical disk S61 of index record write operation system；
2) by the physical disk S62 of entire fund recording documents write operation system.
The step 5) is fund information record creation index S6, and after which finishes, we are by index record
And physical disk is written from memory to guarantee the reliability of data in the record of document.
When needing to retrieve data, when first having to inquiry fund information, by the document of entire fund all from object
Reason disk is loaded into memory, has all been stored in inside the document of fund in the associated data of step 3) inside basic gold, at this time
It, can be according to the data inside index traversal memory after entire document is loaded into memory.Since associated data is unified, we
It can be inquired inside memory when inquiry, to ensure that search efficiency.
Embodiment 3
Referring to Fig. 2, the difference is that, the step 6) includes the following steps: with above-described embodiment
1) fund record information is loaded into document information S71；
2) document information whether include inquiry in need document information S72；If so, then entering in next step；
3) document information being loaded onto is loaded into the memory S73 of operating system by index information；
4) according to match information in memory is indexed, query result S74 is obtained.
Preferably, in the step 2), if the document information does not include the document information of inquiry in need,
S721 is not present in inquiry record.
Preferably, the index includes comment tag in the step 5).
In the present embodiment, when checking fund data, a label has been owned by for each data.On for example,
Following two records are obtained inside fund document described in stating:
Fund number: specific fund number A, fund flows: specific fund flows A, fund name: specific fund name
Claim A；
Fund number: specific fund number B, fund flows: specific fund flows B, fund name: specific fund name
Claim B, fund holds position: fund is held position subdocument B；
Every data line is all by label: the mode of numerical value stores, and for each data, we can be defined not
Same label, is equal to us to each data and has done an annotation, for the people checked it can be seen that different
The corresponding meaning of data.
The foregoing is only a preferred embodiment of the present invention, is not intended to limit the scope of the present invention.
Claims (8)
1. a kind of unstructured finance data storage method, comprising: database and operating system, it is characterised in that: the finance
Data storing steps include:
Step 1) stores funds data in the database；
Step 2) creates fund document in operating system memory；
Step 3) records a line record information that different field compositions is not fixed column according to fund information；
Whether step 4) fund information record includes fund information of holding position；If it is, entering in next step；
Step 5) is fund information record creation index；
Step 6) inquires finance data information.
2. a kind of unstructured finance data storage method according to claim 1, it is characterised in that: the step 3) note
Record information includes recording information according to fund number or fund flows or fund name.
3. a kind of unstructured finance data storage method according to claim 1, it is characterised in that: the step 4) packet
If including fund information record not comprising fund information of holding position, is recorded in fund and create subdocument of holding position in information.
4. a kind of unstructured finance data storage method according to claim 1, it is characterised in that: in the step 5)
The creation index includes creation hash index or creation b-tree indexed.
5. according to claim 1 or a kind of 4 any unstructured finance data storage methods, it is characterised in that: described
Step 5) includes the following steps:
1) by the physical disk of index record write operation system；
2) by the physical disk of entire fund recording documents write operation system.
6. a kind of unstructured finance data storage method according to claim 1, it is characterised in that: the step 6) packet
Include following steps:
1) fund record information is loaded into document information；
2) document information whether include inquiry in need document information；If so, then entering in next step；
3) document information being loaded onto is loaded into the memory of operating system by index information；
4) according to match information in memory is indexed, query result is obtained.
7. a kind of unstructured finance data storage method according to claim 6, it is characterised in that: the document information
If not including the document information of inquiry in need, inquiry record is not present.
8. according to claim 1 or a kind of 4 any unstructured finance data storage methods, it is characterised in that: described
The index includes comment tag in step 5).
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810933306.0A CN109063167A (en) | 2018-08-16 | 2018-08-16 | A kind of unstructured finance data storage method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810933306.0A CN109063167A (en) | 2018-08-16 | 2018-08-16 | A kind of unstructured finance data storage method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN109063167A true CN109063167A (en) | 2018-12-21 |
Family
ID=64687018
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810933306.0A Pending CN109063167A (en) | 2018-08-16 | 2018-08-16 | A kind of unstructured finance data storage method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109063167A (en) |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102360367A (en) * | 2011-09-29 | 2012-02-22 | 广州中浩控制技术有限公司 | XBRL (Extensible Business Reporting Language) data search method and search engine |
| US20130246437A1 (en) * | 2010-11-05 | 2013-09-19 | Apple Inc. | Extended database search |
| CN104699748A (en) * | 2014-12-19 | 2015-06-10 | 深圳市燃气集团股份有限公司 | Method and system for showing report form with non-fixed column numbers |
| CN112115310A (en) * | 2020-08-28 | 2020-12-22 | 苏宁金融科技(南京)有限公司 | Map Schema storage method and device, computer equipment and storage medium |
-
2018
- 2018-08-16 CN CN201810933306.0A patent/CN109063167A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20130246437A1 (en) * | 2010-11-05 | 2013-09-19 | Apple Inc. | Extended database search |
| CN102360367A (en) * | 2011-09-29 | 2012-02-22 | 广州中浩控制技术有限公司 | XBRL (Extensible Business Reporting Language) data search method and search engine |
| CN104699748A (en) * | 2014-12-19 | 2015-06-10 | 深圳市燃气集团股份有限公司 | Method and system for showing report form with non-fixed column numbers |
| CN112115310A (en) * | 2020-08-28 | 2020-12-22 | 苏宁金融科技(南京)有限公司 | Map Schema storage method and device, computer equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20220342875A1 (en) | Data preparation context navigation | |
| JP7257068B2 (en) | Systems, methods, and data structures for fast searching or filtering of large datasets | |
| US11520760B2 (en) | System and method for providing bottom-up aggregation in a multidimensional database environment | |
| US6240418B1 (en) | Database apparatus | |
| CN101405728B (en) | Relational database architecture with dynamic load capability | |
| US20130268567A1 (en) | System And Method For Updating Slowly Changing Dimensions | |
| CN104933112A (en) | Distributed Internet transaction information storage and processing method | |
| CN103577440A (en) | Data processing method and device in non-relational database | |
| US7624117B2 (en) | Complex data assembly identifier thesaurus | |
| US9626401B1 (en) | Systems and methods for high-speed searching and filtering of large datasets | |
| US20090030880A1 (en) | Model-Based Analysis | |
| CN104750776A (en) | Accessing information content in a database platform using metadata | |
| JP2008537827A (en) | Hierarchical database relational queries | |
| CN112507067A (en) | Cache plug-in annotating device and annotation method | |
| US20200278980A1 (en) | Database processing apparatus, group map file generating method, and recording medium | |
| CN110457398A (en) | Block data storage method and device | |
| US20070282804A1 (en) | Apparatus and method for extracting database information from a report | |
| CN111680030A (en) | Data fusion method and device, and data processing method and device based on meta information | |
| CN101388018A (en) | Computer aided design document management method | |
| US20090012948A1 (en) | System and method for translating and executing queries | |
| CN109063167A (en) | A kind of unstructured finance data storage method | |
| CN113157692B (en) | Relational memory database system | |
| CN108241624A (en) | The generation method and device of a kind of query script | |
| CN114218277A (en) | Efficient query method and device for relational database | |
| CN110110024B (en) | Method for importing high-capacity VCT file into spatial database |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20181221 |
|
| RJ01 | Rejection of invention patent application after publication |