CN104809110A - Large data system - Google Patents
Large data system Download PDFInfo
- Publication number
- CN104809110A CN104809110A CN201410030762.6A CN201410030762A CN104809110A CN 104809110 A CN104809110 A CN 104809110A CN 201410030762 A CN201410030762 A CN 201410030762A CN 104809110 A CN104809110 A CN 104809110A
- Authority
- CN
- China
- Prior art keywords
- data
- dimension
- warehouse
- database
- cube
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention provides a large data system. The large data system comprises a more than one data base, a data extraction device and a data warehouse, wherein the data warehouse is used for storing business data of a user, the data warehouse is connected with more than one data base through the data extraction device, the data extraction device is used for extracting the data contents of more than one data base to obtain data meeting the format of the data warehouse, and transmitting the extracted and settled data into the data warehouse, and the data warehouse is used for analyzing and excavating the extracted and settled data, so as to obtain valuable data. A perfect large data system can be provided.
Description
Technical field
The present invention relates to a kind of computer realm, particularly the large data system of one.
Background technology
Large data (big data), or claim flood tide data, referring to huge the arriving of involved data quantity cannot through current main software instrument, reaching acquisition, management within reasonable time, processing and arrange the information becoming and help the more positive object of enterprise management decision-making, current, also there is no a kind of fairly perfect large data system framework.
Summary of the invention
For above-mentioned technical matters, the invention provides a kind of large data system, solve above-mentioned technical matters.
On the one hand, the invention provides a kind of large data system, comprise: more than 1 database, data pick-up device and data warehouse, wherein, the business datum of database stores user, data warehouse is connected with described more than 1 database by data pick-up device, data pick-up device is by the data content of more than 1 database, extract, obtain the data meeting data warehouse form, and extraction reduced data is sent in data warehouse, data warehouse will extract reduced data analysis and excavation, obtain valuable data.
Beneficial effect of the present invention: the invention provides a kind of perfect large data system.
Accompanying drawing explanation
Fig. 1 is the large data storage system architectures schematic diagram of one of the present invention.
In figure, 1, database, 2, data pick-up device, 3, data warehouse.
Embodiment
Below in conjunction with drawings and Examples, the invention will be further described, is to be noted that described embodiment is only intended to be convenient to the understanding of the present invention, and do not play any restriction effect to it.
Below in conjunction with the drawings and specific embodiments, the present invention is further detailed explanation.
As shown in Figure 1, the invention provides a kind of large data system, comprising: database 1, data pick-up device 2 and data warehouse 3.
Wherein, database 1 stores the business datum of user, different business can be stored on different databases, that is, database 1 can have multiple, data warehouse 3 is connected with multiple database 1 by data pick-up device 2, data pick-up device 2 is by the data content of multiple database 1, extract, obtain the data meeting data warehouse 3, and be sent in data warehouse 3 by extraction reduced data, data warehouse 3 is according to analysis and mining algorithm, reduced data analysis and excavation will be extracted, obtain valuable data.
This process is introduced below with concrete example.
Database 1 can comprise multiple database, as debtor's data source, includes the essential information of all debtor enterprises, financial information, supplier information and other relevant information, for the credit situation analyzing debtor provides Data support.Obligee's data source, includes the essential information of all debtor enterprises, financial information and other relevant information, for the credit situation analyzing obligee provides Data support.Fund provider data source, includes the essential information of all fund providers, organization establishes situation and other information, for the credit situation analyzing accounts receivable credits voucher provides Data support, for the liquidity risk of early warning accounts receivable credits voucher provides support.Accounts receivable source, include the essential information (date, the amount of money, obligee, debtor) of all accounts receivable, financing situation (funding ratio, combined ratio, mortgage situation) and make loans, the situation such as refund, for the credit risk of early warning accounts receivable credits voucher provides support.To make loans information database, include the situation of making loans of all accounts receivable vouchers, for supervision, ensure that the mobility of accounts receivable credits voucher provides support.Refund information database, includes the refund information of all accounts receivable vouchers, for the credit quality of supervision accounts receivable credits voucher provides support.Supplier's directory data source, includes all suppliers situation, for the comprehensive default risk analyzing a certain supplier provides Data support.Internal user data source, includes all internal user situations, for the operability risk comprehensively analyzing interior employee provides Data support.Cooperative bank's data source, includes all cooperative banks situation, for supervision bank of the Banking Supervision Commission provides Data support.Non-bank financial institution's data source, includes all guarantee agencies situation, does supervision Local finance mechanism provide Data support for Local finance.Insurance company's data source, includes all insurance companies situation, for supervision insurance company of Insurance Regulatory Commission provides Data support.Transaction details information database, includes the managing detailed catalogue of All Activity, for concrete analysis transaction risk provides Data support.External user data source, includes all external user Operation Logs, for analysis abnormal operation, operational risk provide support.Violation of agreement data source, includes the information of all promise breaking accounts receivable, for the overall risk analyzing accounts receivable credits circulation business provides support.
Data pick-up device 2 performs following process: data pick-up: the process of extracted data from data source, first time is extracted and extracts data all in database, afterwards can newly-increased in an extracted data storehouse, amendment, the data of deleting, by catching the delta data in operation system exactly, reduce the pressure that operation system is caused, and realize the real-time monitoring of business.Conversion: data changed by consolidation form, ensure the reasonable of database framework, and data deposits the unification of all forms.Cleaning: data cleansing refers to find and last one program of discernible mistake in correction of data file, comprises inspection data consistency, process invalid value and missing values etc.Because the data in data warehouse are set of the data towards a certain theme, these data extract and comprise historical data from multiple operation system, are misdata with regard to keeping away the data unavoidably had like this, the data that have have conflict each other, these mistakes or the data that have conflict be obviously that we are undesired, be called " dirty data ".We " will wash " dirty data " " off according to certain rule, data cleansing that Here it is.And the task of data cleansing filters those undesirable data, the result of filtration is given competent business department, be confirmed whether to filter out or extract again by after service unit correction.Undesirable data mainly have the data of incomplete data, mistake, the data three major types of repetition.Data cleansing audits different from questionnaire, and the data scrubbing after typing is generally by computing machine instead of manually completes.Load: data loading refers to the data changed to be saved in data warehouse and goes.Generally, data loading should carry out after system completes renewal.If first time loads whole data warehouse, index will be set up after shoveller completes, create the index time to reduce; To loading the data changed in origin system in the data warehouse run, (the record major key of input and the major key of existing record match to adopt the load mode of constructive merging, retain existing record, increase the record of input, and be labeled as substituting of old record), also destructive composite shipment mode can be adopted (if the major key of input data record and the major key of a record existed match, by new input Data Update target record data; If input record is a new record, just this record inputted is added in object table).
Data warehouse 3 proceeds as follows: formed and analyze cube.Concrete comprises: form user's cube by following dimension, dimension one: fund provider, dimension two: debtor, dimension three: obligee, dimension four: scope of the enterprise.Analyze data: initial amount, new opening mechanism number, end of term amount, more last amount of increase.Carry out comprehensively for dimension to data to participate in class of establishment (fund provider, debtor, obligee), form the analysis cube (comprising the situations such as the scale change of all kinds of mechanism) of a counting user statistics.
Customer service cube is formed, dimension one: Tianjin, dimension two: Hefei, dimension three: scope of the enterprise by following dimension.Analyze data: initial amount, new opening mechanism number, end of term amount, more last amount of increase.With region (Tianjin, Hefei etc.), scope of the enterprise (large enterprise, medium-sized enterprise, small business, microbusiness, other) carry out comprehensively to data for dimension, form the analysis cube (comprising the situations such as business stroke count, the amount of money) of a counting user business.
Accounts receivable cube is formed, dimension one: raw information is registered, dimension two: financing, dimension three: transaction, dimension four: make loans by following dimension.Analyze data: the number of services under various state, the amount of money.With type of service (raw information is registered, finances, concludes the business, made loans) for dimension is carried out comprehensively to data, form the analysis cube (comprising the situation such as quantity, the amount of money) of a statistics accounts receivable.
Transaction Information cube is formed, dimension one: treat quotation state by following dimension; Dimension two: state of having offered; Dimension three: failure state of bidding; Dimension four: conclusion of the business state.Analyze data: the number of services under various stateful transaction, the amount of money.Carry out comprehensively for dimension to data with stateful transaction (wait to offer, offer, bid inefficacy, conclusion of the business etc.), form the analysis cube (comprising situation such as financing quantity, the amount of money etc.) of a statistics Transaction Information.
Form year combined ratio cube by following dimension, dimension one: Tianjin, dimension two: Hefei, dimension three: scope of the enterprise, analyze data: the average rate of profit, on average other interest rates, average year combined ratio.With region (Tianjin, Hefei etc.), scope of the enterprise (large enterprise, medium-sized enterprise, small business, microbusiness, other) carry out comprehensively to data for dimension, form the analysis cube (comprising the average rate of profit, the on average situation such as other interest rates, average year combined ratio) of a statistics year combined ratio.
Formed by following dimension and increase letter cube, dimension one: plan as a whole guaranty money, dimension two: insurance, dimension three: guarantee, dimension four: ensure.Analyze data: increase letter quantity, the amount of money and value change.For dimension, carry out comprehensively to data to increase letter mode (planning as a whole guaranty money, insurance, guarantee, guarantee), form the analysis cube (increase letter approach, increase the letter amount of money and increase the variation believed and be worth) of a statistics increasing letter mode.
Real-time deal cube is formed, dimension one: send out voucher, dimension two: wait to buy back voucher, dimension three: buy back voucher by following dimension.Analyze data: voucher quantity, the amount of money.With credential status (provide, treat repurchase, buy back) for dimension, carry out comprehensively to data, form the analysis cube (sent out voucher quantity and the amount of money, bought back quantity and the amount of money, treated repurchase quantity and the amount of money) of a statistics real-time deal.
Credits circulation record cube is formed, dimension one: flowing mode, dimension two: circulation object, dimension three: assignee, dimension four: initial obligee by following dimension.Analyze data: circulation quantity, the amount of money.With flowing mode, circulation object, assignee, the initial artificial dimension of credits, carry out comprehensively to data, form the analysis cube (circulation quantity, the amount of money) of a statistics credits circulation record.
Data warehouse 4 analysis means can comprise: drill through, and drilling through is change to analyze cubical dimension hierarchy, the granularity of transform analysis.Be deep into detail data from combined data to carry out observing or increase new dimension.Such as, time customer analysis " sales situation in each department, city ", the sales volume in each year can be subdivided into the sales volume in some cities, to the sales volume in a certain year, the sales volume being subdivided into each season can be continued.By the function drilled through, user more can be understood in depth data, more easily pinpoint the problems, make correct decision-making.Upper volume, upper volume with drill through contrary, refer to, along a certain dimension layering upwards stipulations, the detail data of low level is summarized into high-level combined data by certain one dimension, or reduce dimension.The business cube such as comprising regional peacekeeping time dimension may delete area dimension in upper volume operation, is that business datum is only temporally assembled.Section and stripping and slicing, in multidimensional data structure, cut into slices by two dimension, carry out stripping and slicing by three-dimensional, can obtain required data.As carried out stripping and slicing and section in " lending bank, loan quality, time " three-dimensional cube, the statistical conditions of each lending bank, various loan can be obtained.Each is all carry out segmentation along wherein one dimension to be called burst, and the burst at every turn carried out along multidimensional is called piecemeal.Rotate, by rotating the data that can obtain different visual angles
Above-mentioned data warehouse 3 analysis personnels/show object comprises: BRC: for bank data source and relevant cube, utilize various analysis means, supervise bank.IRC: for insurance company's data source and relevant cube, utilize various analysis means, insurance company is supervised.Local finance supervision unit: for other non-bank financial institution's data sources and relevant cube, utilize various analysis means, local financial institution is supervised.
Describing above is only a specific embodiment of the present invention, and obviously anyone amendment done of this area or locally replacement under technical scheme of the present invention instructs, all belongs to the scope that claims of the present invention limits.
Claims (1)
1. a large data system, it is characterized in that, comprise: more than 1 database, data pick-up device and data warehouse, wherein, the business datum of database stores user, data warehouse is connected with described more than 1 database by data pick-up device, data pick-up device is by the data content of more than 1 database, extract, obtain the data meeting data warehouse form, and extraction reduced data is sent in data warehouse, data warehouse will extract reduced data analysis and excavation, obtain valuable data.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410030762.6A CN104809110A (en) | 2014-01-23 | 2014-01-23 | Large data system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410030762.6A CN104809110A (en) | 2014-01-23 | 2014-01-23 | Large data system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN104809110A true CN104809110A (en) | 2015-07-29 |
Family
ID=53693941
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201410030762.6A Pending CN104809110A (en) | 2014-01-23 | 2014-01-23 | Large data system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN104809110A (en) |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105354698A (en) * | 2015-10-28 | 2016-02-24 | 无锡澳优汇国际贸易有限公司 | Enterprise business data tracking and processing system |
| CN107317693A (en) * | 2016-04-27 | 2017-11-03 | 广州市动景计算机科技有限公司 | service management device, device and method |
| CN108153819A (en) * | 2017-12-01 | 2018-06-12 | 天津中发智能科技有限公司 | A kind of 3D hologram wisdom building managing and control system |
| CN110135987A (en) * | 2019-04-25 | 2019-08-16 | 安徽海汇金融投资集团有限公司 | A kind of accounts receivable credits plan as a whole the management method and system of guarantee fund |
| CN110941659A (en) * | 2019-11-26 | 2020-03-31 | 上海景域文化传播股份有限公司 | Data layered splitting method for composite revenue |
| CN111667353A (en) * | 2019-03-07 | 2020-09-15 | 安徽海汇金融投资集团有限公司 | Account and debt right transfer supervision system |
| TWI714262B (en) * | 2018-12-04 | 2020-12-21 | 開曼群島商創新先進技術有限公司 | Business risk prevention and control method and device |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20050044097A1 (en) * | 2003-08-19 | 2005-02-24 | Jaime Singson | Method and apparatus for facilitating data stewardship for metadata in an ETL and data warehouse system |
| CN101042746A (en) * | 2006-03-21 | 2007-09-26 | 上海浦东国际集装箱码头有限公司 | Container pier intellectualized report forms based on store house and method |
| CN101075304A (en) * | 2006-05-18 | 2007-11-21 | 河北全通通信有限公司 | Method for constructing decision supporting system of telecommunication industry based on database |
| US20080046506A1 (en) * | 2002-09-06 | 2008-02-21 | Tal Broda | Method and apparatus for a multiplexed active data window in a near real-time business intelligence system |
| CN101840563A (en) * | 2009-03-20 | 2010-09-22 | 上海佳锐信息科技有限公司 | Financial data service system |
| CN102508908A (en) * | 2011-11-11 | 2012-06-20 | 北京用友政务软件有限公司 | Method for acquiring subordinate financial business data and system for acquiring subordinate financial business data |
| US20120265726A1 (en) * | 2011-04-18 | 2012-10-18 | Infosys Limited | Automated data warehouse migration |
| US20130138601A1 (en) * | 2011-11-24 | 2013-05-30 | Tata Consultancy Services Limited | System and method for data aggregation, integration and analyses in a multi-dimensional database |
| CN103258047A (en) * | 2013-05-24 | 2013-08-21 | 杭州电子科技大学 | Data organization method of data warehouse for controlling operation cost of medicine enterprise |
-
2014
- 2014-01-23 CN CN201410030762.6A patent/CN104809110A/en active Pending
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20080046506A1 (en) * | 2002-09-06 | 2008-02-21 | Tal Broda | Method and apparatus for a multiplexed active data window in a near real-time business intelligence system |
| US20050044097A1 (en) * | 2003-08-19 | 2005-02-24 | Jaime Singson | Method and apparatus for facilitating data stewardship for metadata in an ETL and data warehouse system |
| CN101042746A (en) * | 2006-03-21 | 2007-09-26 | 上海浦东国际集装箱码头有限公司 | Container pier intellectualized report forms based on store house and method |
| CN101075304A (en) * | 2006-05-18 | 2007-11-21 | 河北全通通信有限公司 | Method for constructing decision supporting system of telecommunication industry based on database |
| CN101840563A (en) * | 2009-03-20 | 2010-09-22 | 上海佳锐信息科技有限公司 | Financial data service system |
| US20120265726A1 (en) * | 2011-04-18 | 2012-10-18 | Infosys Limited | Automated data warehouse migration |
| CN102508908A (en) * | 2011-11-11 | 2012-06-20 | 北京用友政务软件有限公司 | Method for acquiring subordinate financial business data and system for acquiring subordinate financial business data |
| US20130138601A1 (en) * | 2011-11-24 | 2013-05-30 | Tata Consultancy Services Limited | System and method for data aggregation, integration and analyses in a multi-dimensional database |
| CN103258047A (en) * | 2013-05-24 | 2013-08-21 | 杭州电子科技大学 | Data organization method of data warehouse for controlling operation cost of medicine enterprise |
Non-Patent Citations (1)
| Title |
|---|
| 韩宪宝: "商业智能技术在企业资源计划中的应用研究", 《中国优秀硕士学位论文全文数据库信息科技辑》 * |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105354698A (en) * | 2015-10-28 | 2016-02-24 | 无锡澳优汇国际贸易有限公司 | Enterprise business data tracking and processing system |
| CN107317693A (en) * | 2016-04-27 | 2017-11-03 | 广州市动景计算机科技有限公司 | service management device, device and method |
| CN107317693B (en) * | 2016-04-27 | 2021-06-22 | 阿里巴巴（中国）有限公司 | Service management equipment, device and method |
| CN108153819A (en) * | 2017-12-01 | 2018-06-12 | 天津中发智能科技有限公司 | A kind of 3D hologram wisdom building managing and control system |
| TWI714262B (en) * | 2018-12-04 | 2020-12-21 | 開曼群島商創新先進技術有限公司 | Business risk prevention and control method and device |
| CN111667353A (en) * | 2019-03-07 | 2020-09-15 | 安徽海汇金融投资集团有限公司 | Account and debt right transfer supervision system |
| CN110135987A (en) * | 2019-04-25 | 2019-08-16 | 安徽海汇金融投资集团有限公司 | A kind of accounts receivable credits plan as a whole the management method and system of guarantee fund |
| CN110941659A (en) * | 2019-11-26 | 2020-03-31 | 上海景域文化传播股份有限公司 | Data layered splitting method for composite revenue |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN104809110A (en) | Large data system | |
| Gilje | Does local access to finance matter? Evidence from US oil and natural gas shale booms | |
| US9251490B2 (en) | Aggregating business analytics architecture and configurator | |
| US8768809B1 (en) | Methods and systems for managing financial data | |
| An | Data analysis | |
| US7860781B1 (en) | Methods and systems for asset/loan management and processing | |
| CN111126955A (en) | Business data analysis method, processing server and storage medium | |
| US20070239587A1 (en) | System and Method For Dynamically Utilizing and Managing Financial, Operational, and Compliance Data | |
| CN105787661A (en) | Multilevel commission rapid settlement processing system and processing method | |
| CN107807953A (en) | A kind of information cloud platform management system based on block chain | |
| US11551230B2 (en) | Security attack detections for transactions in electronic payment processing networks | |
| Amaechi et al. | Application of computed financial ratios in fraud detection modelling: a study of selected banks in Nigeria | |
| CN107292733A (en) | A kind of household finance management information system | |
| Burdick et al. | Data science challenges in real estate asset and capital markets | |
| Agbo et al. | Effects of treasury single account on performance and survival of deposit money banks in Nigeria | |
| US20190347596A1 (en) | System for decommissioning information technology assets using solution data modelling | |
| US9529860B2 (en) | Keyword frequency analysis system | |
| US10936984B2 (en) | System for mitigating exposure associated with identified impacts of technological system changes based on solution data modelling | |
| Sun et al. | Cross‐border M&As and the performance of Chinese acquiring firms | |
| Wang et al. | A financial assets and liabilities management support system | |
| CN109299065A (en) | Generate processing method, system and the storage medium of asset-liabilities unified view | |
| CN110969444A (en) | Enterprise bidding qualification analysis method and device | |
| Kaur | Business intelligence and E-banking: a study of Bi importance in banking sector | |
| Liu | Financial indicator system based on data mining algorithm | |
| Mary et al. | A Contradiction of Corporate Social Responsibility in Moderating Tax Avoidance |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20150729 |
|
| RJ01 | Rejection of invention patent application after publication |