CN108573348A - Financial indicator distributed computing method and its system - Google Patents
Financial indicator distributed computing method and its system Download PDFInfo
- Publication number
- CN108573348A CN108573348A CN201810347176.2A CN201810347176A CN108573348A CN 108573348 A CN108573348 A CN 108573348A CN 201810347176 A CN201810347176 A CN 201810347176A CN 108573348 A CN108573348 A CN 108573348A
- Authority
- CN
- China
- Prior art keywords
- index
- user
- data source
- calculating
- calculates
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000004364 calculation method Methods 0.000 title claims abstract description 72
- 230000008859 change Effects 0.000 claims abstract description 57
- 238000000034 method Methods 0.000 claims abstract description 6
- 238000011144 upstream manufacturing Methods 0.000 claims description 7
- 230000005856 abnormality Effects 0.000 claims description 6
- 230000002776 aggregation Effects 0.000 claims description 2
- 238000004220 aggregation Methods 0.000 claims description 2
- 238000000205 computational method Methods 0.000 claims description 2
- 238000000151 deposition Methods 0.000 claims 1
- 230000004048 modification Effects 0.000 abstract description 3
- 238000012986 modification Methods 0.000 abstract description 3
- 238000012545 processing Methods 0.000 description 9
- 230000001960 triggered effect Effects 0.000 description 8
- 230000006399 behavior Effects 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 4
- 238000012544 monitoring process Methods 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 238000006116 polymerization reaction Methods 0.000 description 2
- 230000009471 action Effects 0.000 description 1
- 238000013475 authorization Methods 0.000 description 1
- 230000015556 catabolic process Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000006731 degradation reaction Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 230000006855 networking Effects 0.000 description 1
- 238000002203 pretreatment Methods 0.000 description 1
- 238000012797 qualification Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/06—Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
- G06Q10/063—Operations research, analysis or management
- G06Q10/0639—Performance analysis of employees; Performance analysis of enterprise or organisation operations
- G06Q10/06393—Score-carding, benchmarking or key performance indicator [KPI] analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/06—Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
- G06Q10/063—Operations research, analysis or management
- G06Q10/0635—Risk analysis of enterprise or organisation activities
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q40/00—Finance; Insurance; Tax strategies; Processing of corporate or income taxes
- G06Q40/03—Credit; Loans; Processing thereof
Landscapes
- Business, Economics & Management (AREA)
- Human Resources & Organizations (AREA)
- Engineering & Computer Science (AREA)
- Economics (AREA)
- Strategic Management (AREA)
- Entrepreneurship & Innovation (AREA)
- Development Economics (AREA)
- Educational Administration (AREA)
- Marketing (AREA)
- Physics & Mathematics (AREA)
- General Business, Economics & Management (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Accounting & Taxation (AREA)
- Finance (AREA)
- Game Theory and Decision Science (AREA)
- Operations Research (AREA)
- Quality & Reliability (AREA)
- Tourism & Hospitality (AREA)
- Technology Law (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
- Financial Or Insurance-Related Operations Such As Payment And Settlement (AREA)
Abstract
The present invention provides user's index distributed computing method and its system in a kind of finance, this method includes：The calculating of user's index is divided into three kinds of indexs and calculates type：Off-line calculation, business triggering calculate, data source change triggering calculates；User's index, including user's Criterion Attribute and index operator are defined, index operator includes defining the data source, index calculating logic, index used when index calculates to calculate type, and index calculating type is at least one in index operator；Definition triggers the condition that three kinds of indexs calculate type；Index belonging to user's index calculates type, and calculate type using index corresponding with trigger condition when trigger condition occurs calculates into row index；Store the user's index calculated, index computational efficiency of the present invention is high, can solve index computes repeatedly, newly-increased/modification index calculating logic is difficult, index lacks the problems such as serious, and it can effectively ensure that the high excellent execution for calculating fast operator, improve the stability and real-time throughput of system.
Description
Technical field
The invention belongs to financial information technology fields, and in particular to a kind of financial indicator distributed computing method and its be
System.
Background technology
With the development of Information and Communication Technology and internet, influence of the internet Financial Information to financial market is more next
More it can not be ignored.Current internet finance pattern, is made of conventional banking facilities and non-financial institution.Conventional banking facilities are main
Internet innovation for traditional financial business and electric businessization innovation, APP softwares etc., non-financial institution are then primarily referred to as using mutual
Networking technology carries out the electric business enterprise of financial operation, money borrows the network loan platform of (P2P) pattern, and pledging for P2I patterns is borrowed
Platform is borrowed, crowd raises the network investment platform of pattern, digs mobile phone the financing APP and Third-party payment platform etc. of wealth class.
When finance or non-financial institution are that user generates business, in order to avoid risk, user can all be assessed.It crosses
When going to calculate personal assessment's index in financial field, index calculating has that speed is slow, it is more to compute repeatedly, consuming computing resource is serious
The problems such as.And data source, which frequently becomes, but will cause operator to calculate demonstration, often have the feelings that 1,2 day calculating task is persistently accumulated
Condition.
Invention content
The present invention is to solve the above-mentioned problems, and it is an object of the present invention to provide a kind of computational efficiency is high, can solve
Previous index computes repeatedly, newly-increased/modification index calculating logic is difficult, index lacks the problems such as serious, can improve system
User's index distributed computing method and its system in stability and the finance of real-time throughput.
The present invention provides user's index distributed computing methods in a kind of finance, which is characterized in that includes the following steps：
Step 1, the calculating of user's index is divided into three kinds of indexs and calculates type：Off-line calculation, business triggering calculate and data source change
Triggering calculates；
Step 2, user's index is defined, including user's Criterion Attribute and index operator, the index operator of definition include
It defines the data source, index calculating logic, index used when index calculates and calculates type, index calculates class in the index operator
Type is at least one；
Step 3, definition triggers the condition that three kinds of indexs calculate type；
Step 4, the index belonging to user's index calculates type, is used and trigger condition pair when trigger condition occurs
The index answered calculates type and is calculated into row index；
Step 5, the user's index calculated is stored.
Further, in finance provided by the invention in user's index distributed computing method, can also have such
Feature：Wherein, the trigger condition of off-line calculation index is to meet following either condition：
When Add User index or more New Set calculating logic when；
When needing the newer user's index of timing, reach renewal time；
Business triggering parameter calculate in real time pressure it is overweight when, low priority index is downgraded to off-line calculation index automatically
After detecting in memory queue that task quantity is more than defined threshold value when processing or data source change triggering parameter, will in
Low priority index task drop power in queue is deposited to handle to off-line calculation index；
When upstream and downstream data source generation problem leads to corrupt data, loss or delay or business personnel has found certain fingers
When mark calculates problematic.
Further, in finance provided by the invention in user's index distributed computing method, can also have such
Feature：Wherein, the trigger condition of business triggering parameter is：Business side finds user's Indexes Abnormality, miscalculation or expired；Or it is new
After user's registration.
Further, in finance provided by the invention in user's index distributed computing method, can also have such
Feature：Wherein, the trigger condition of data source change triggering parameter is：Data source changes and/or index calculating logic changes.
Further, in finance provided by the invention in user's index distributed computing method, can also have such
Feature：Wherein, the computational methods of data source change triggering calculating are：
After data source and/or the change of index calculating logic, it is the finger that data source change triggering calculates to find out pointer type
It is marked on the data source needed when calculating, the index for changing data source and/or index calculating logic, which is put into data source change, to be touched
Hair calculates memory queue；
It is changed in triggering calculating memory queue from data source and obtains index in batches, pre-processed；
Pretreated index is calculated according to changed data source and/or index calculating logic；
User's index after calculating is stored.
Further, in finance provided by the invention in user's index distributed computing method, can also have such
Feature：Changing the pretreated method of index progress in triggering calculating memory queue to data source is：Duplicate removal, same is carried out to index
A data source change time aggregation operation.
The present invention also provides user's index distributed computing systems in a kind of finance, which is characterized in that includes：In registration
The heart, for storing user's index and index operator corresponding with user's index；
Data source changes index computing module, is used for being calculated in batches when data source or index operator calculating logic change
Family index；
Business triggers index computing module, for being calculated when business side finds user's Indexes Abnormality or after new user's registration
User's index；
Offline index computing module calculates use for being calculated when trigger condition occurs in offline index by the way of offline
Family index；
Trigger module, the index belonging to user's index calculate type, triggering and triggering item when trigger condition occurs
The corresponding data source change index computing module of part or business triggering index computing module or offline index computing module into
Row user's index calculates；And
Index memory module, for storing user's index after calculating,
Wherein, it is to meet following either condition that the offline index, which calculates trigger condition,：
When newly-increased index or more New Set calculating logic；
When needing the newer index of timing, reach renewal time；
Business triggering parameter calculate in real time pressure it is overweight when, low priority index is downgraded to off-line calculation index automatically
After detecting in memory queue that task quantity is more than defined threshold value when processing or data source change triggering parameter, will in
Low priority index task drop power in queue is deposited to handle to off-line calculation index；
When upstream and downstream data source generation problem leads to corrupt data, loss or delay or business personnel has found certain fingers
When mark calculates problematic.
Further, in finance provided by the invention in user's index distributed computing system, can also have such
Feature：The memory module includes：Index updates storage module and timing module, and the index updates storage module and is used for
Storage needs the update cycle of the newer index of timing and each index, the timing module to be used for each index
Carry out timing.
Further, in finance provided by the invention in user's index distributed computing system, can also have such
Feature：The registration center includes subscribing module, and the subscribing module subscribes to the data source needed when index calculates.
Advantages of the present invention is as follows：
According to user's index distributed computing method in finance according to the present invention, triggered according to different conditions different
Pointer type calculate, index computational efficiency is high, by flow complementation, solves that index computes repeatedly, newly-increased/modification index calculates
The problems such as logistical difficulties, serious index missing, and when business triggering calculates and data source change triggering is calculated to priority
The longer operator degradation processing of relatively low or calculating duration improves system to which the high excellent execution for calculating fast operator be effectively ensured
Stability and real-time throughput.
User's index distributed computing system index computational efficiency is high in finance provided by the invention, can solve index weight
It is multiple to calculate, is newly-increased/to change the problems such as index calculating logic is difficult, index missing is serious, and can effectively ensure that high excellent calculating
The execution of fast operator, improves the stability and real-time throughput of system.
Description of the drawings
Fig. 1 is the flow chart of user's index distributed computing method in finance in the present invention；
Fig. 2 is user's index calculating support composition in the present invention；
Fig. 3 is the structural schematic diagram of user's index distributed computing system in finance in the present invention.
Specific implementation mode
It is real below in order to make the technical means, the creative features, the aims and the efficiencies achieved by the present invention be easy to understand
Example combination attached drawing is applied to be specifically addressed user's index distributed computing method in finance of the invention and system.
Before describing, the technical term that this patent is related to is illustrated：
User's index：According to the data source of user, some feature of user is calculated.
Index calculates：It is calculated using the related data of user as parameter according to a certain specific operation rule or function
To user some feature as a result.
Data source：All types of data sources, including user submit data, user authorization data, user behavior data and outer
Portion mechanism data etc..
As shown in Figure 1, user's index distributed computing method comprises the steps of in finance：
Financial indicator calculating is divided into three kinds of indexs and calculates type by step S1：Off-line calculation, business triggering calculate and
Data source change triggering calculates.
Three kinds of indexs, which calculate type, to be studied according to the actual use situation problem, to avoid repeating big gauge
It calculates, efficiency is low, index lacks the problems such as serious.Such as：After a new user enters operation system, a series of award has been carried out
Power and operation.It is calculated if this when carries out data source change to the behavior of each step of this user, that is very possible very much
The index needs for mixing multiple data sources largely compute repeatedly, and result in the beneath of efficiency.If being classified as off-line calculation,
So next this user applies for that shortage of data can be encountered when product causes application to fail.This when is for this condition
User is calculated using business triggering, can not only be reduced and be computed repeatedly, and can calculate operation system when meeting professional qualification
The user's index needed.
Off-line calculation is specific to cross process：
Step 1, prepare calculating task output file, including the index operator after change and user filter (uses
The conditions such as id/ querying conditions/full dose are filtered).
Step 2, prepare the data needed for parameter, then calculated into row index.
Step 3, result of calculation file is returned.
Business triggering calculates detailed process：
Step 1, according to index, corresponding index operator is found.
Step 2, prepare the data needed for parameter, then calculated into row index.
Step 3, result of calculation file is returned.
Data source change triggering calculates detailed process：
Step 1, after data source and/or the change of index calculating logic, it is data source change triggering meter to find out pointer type
The data source that the index of calculation is needed when calculating, data source is put by the index for changing data source and/or index calculating logic
Change triggering calculates memory queue.
Step 2, it is changed in triggering calculating memory queue from data source and obtains index in batches, pre-processed.In this implementation
In example, carrying out pretreated mode is：The behaviour such as duplicate removal, the polymerization of same user data source altering event are carried out to the index of acquisition
Make.It is computed repeatedly to avoid subscribing to multi-data source index, reduces original 70% or more repeat operator calculating, such as：With
Family product clicks event, and triggering user's focus index calculates and crowd's index calculates, and user may click multiple in the short time
Product, the calculating of the two indexs can merge this multiple click behavior and once be calculated again, multiple without carrying out
It calculates.
Step 3, pretreated index is calculated according to changed data source and/or index calculating logic.
Step S2 defines user's index, including user's Criterion Attribute and index operator.The index operator of definition includes definition
Data source, index calculating logic, the index used when index calculating calculates type.Index calculates type extremely in each index operator
It is one less, determines that user's index belongs to which kind of index calculates type with specific reference to actual conditions.User's Criterion Attribute is user
Index name etc..
Step S3, definition trigger the condition that three kinds of indexs calculate type.
The trigger condition of off-line calculation is：1, when Add User index or more New Set calculating logic when, used when newly-increased
When the index of family, the user's index for needing definition newly-increased is identical as user's index defined in step S2, including the user that definition is newly-increased
The attribute and index operator of index.2, when index needs timing to update, reach renewal time.3, business triggering calculates calculates in real time
When pressure is overweight, low priority index detects interior when being downgraded to off-line calculation processing or data source change triggering calculating automatically
Task quantity in queue is deposited to arrive low priority index task drop power in memory queue at off-line calculation more than after defined threshold value
Reason.4, when upstream and downstream data source occurs when problem leads to corrupt data, loss or delay or business personnel has found certain index meters
When calculating problematic.When meeting the trigger condition of off-line calculation, off-line calculation is proceeded by.
Business triggers the condition calculated：Business side find user Indexes Abnormality, miscalculation or it is expired when, triggering pair
The index of correlation of the user recalculates, or after new user's registration, triggering calculates the index of the new user.When meeting industry
When the trigger condition that business triggering calculates, the business triggering of progress calculates.
The trigger condition that data source change triggering calculates：Data source changes and/or index calculating logic changes.That is data source
Change, index calculating logic change, data source and index calculating logic changes three kinds of situations can the change of trigger data source it is tactile
Hair calculates.Data source change further includes user and actively updates data or data source side's update user data.
Step S4, index belonging to user's index calculate type, are used when trigger condition occurs and trigger condition
Corresponding index calculates type and is calculated into row index.
Type only is calculated comprising the index when the index in index operator calculates in type, and meets index calculating
When the trigger condition of type, index could be calculated, for example, there was only off-line calculation when the index in index operator calculates type
When triggering calculating with business, when the condition that off-line calculation or business triggering calculate meets, which executes and triggers item
The corresponding index of part calculates type and is calculated into row index, that is, executes off-line calculation or business triggering calculates and carries out user's index meter
It calculates, and when data source changes the condition that triggering calculates and meets, which does not execute data source change triggering and calculates.
When meeting the trigger condition of off-line calculation, it includes off-line calculation, triggering that the index in index operator, which calculates type,
Off-line calculation carries out.Four kinds of trigger conditions of off-line calculation triggering off-line calculation is illustrated separately below.
When newly-increased index or more New Set calculating logic, off-line calculation is triggered, by New Set or more New Set meter
The batch data acquisition that the index of logic uses is calculated, hdfs is written, it is large quantities of using hadoop MR progress according to index calculating logic
Figureofmerit calculates, and is as a result stored in hdfs, the index calculated from hdfs, and hbase is written in batch.
Index updates storage module and safeguards that needs the newer index list of timing, in need comprising institute in index list
Periodically newer index stores title and the index newer period of newer index in need, for example, referring in index list
Marking the newer period is set as that update in every 1 hour is primary or the update of last day every month is primary.Expire when needing newer index
When sufficient time conditions, off-line calculation is triggered, the batch data used when which is calculated obtains, and hdfs is written, and uses
Hadoop MR carry out large quantities of figureofmerit calculating, are as a result stored in hdfs, the index calculated from hdfs, and hbase is written in batch.
When the real-time calculating pressure of business triggering calculating (API) is overweight, i.e., calculating task is excessive simultaneously for same computing unit, meeting
Low priority index computation requests are put into the troop of off-line calculation, off-line calculation processing is carried out.Data source change triggering meter
Detect that task quantity takes out low priority index task in memory queue more than after defined threshold value in memory queue when calculation
It is put into the troop of off-line calculation, carries out off-line calculation processing.The index being put into off-line calculation troop is calculated to the number needed
According to taking-up, hdfs is written, carries out large quantities of figureofmerit calculating using hadoop MR, is as a result stored in hdfs, is calculated from hdfs
Hbase is written in index, batch.
When detecting that upstream and downstream data source generation problem leads to corrupt data, loss or delay or business personnel has found
When certain indexs calculate problematic, off-line calculation is triggered, it would be desirable to which completion user's index is put into the troop of off-line calculation, then
Batch obtains index completion task, and hdfs is written in the data that completion task is needed, and large quantities of figureofmerits are carried out using hadoop MR
It calculates, is as a result stored in hdfs, hbase is written in the index calculated from hdfs, batch.
When meeting the trigger condition that business triggering calculates, it includes business triggering meter that the index in index operator, which calculates type,
It calculates, triggering business triggering, which calculates, to be carried out, and after obtaining data, is called and is calculated enforcement engine, merges with the same index of user, from registration
The heart finds index and executes worker, is calculated, storage result after calculating；If it is overweight that API calculates pressure in real time, can incite somebody to action
Low priority index computation requests are put into off-line calculation task queue and carry out processed offline, and when operation system being notified to cross one section
Between obtain.
When meeting the trigger condition that data source change triggering calculates, it includes business that the index in index operator, which calculates type,
Triggering calculates, and the change triggering of trigger data source, which calculates, to be carried out.For example the change triggering of data source change trigger data source is calculated and is carried out,
The index for the data source for having subscribed change is put into the memory queue that data source change triggering calculates, indication computing system work
Node computational threads (abbreviation computational threads) obtain index in batches from memory queue, carry out duplicate removal, the same user data source
The pretreatments such as altering event polymerization；After obtaining data, computational threads, which are called, calculates enforcement engine, merges with the same index of user, from
Registration center finds index and executes worker, is calculated, batch storage result after calculating；If monitoring thread finds memory
Task quantity is more than threshold value in queue, by low priority index task drop power in memory queue, is put into task team in off-line calculation
Row carry out processed offline.In the present embodiment, when it includes that data source change triggering calculates that the index of user's index, which calculates type,
It should mark which data source is user's index have subscribed.
Index computing architecture as shown in Fig. 2, when calculating index, according to different conditions be respectively adopted off-line calculation,
Business triggering calculates and data source changes triggering and calculates three types, is triggered in business and calculates (API) to the progress of user's index
When calculating, if to calculate pressure in real time overweight by API, low priority index computation requests can be put into off-line calculation troop into
The processing of row off-line calculation, and operation system is notified to extract after a certain period of time.Data source change triggering calculate to user's index into
When row calculates, if monitoring thread finds that task quantity is more than threshold value in memory queue, and low priority index in memory queue is appointed
Business drop power the case where being put into off-line calculation troop progress off-line calculation processing, calculating task is avoided persistently to accumulate, improves and is
The stability and real-time throughput of system.
Step S5 stores the user's index calculated.
As shown in figure 3, financial indicator distributed computing system 200 includes：Data source changes index computing module 210, industry
In business triggering index computing module 220, offline index computing module 230, trigger module 240, result memory module 250, registration
The heart 260.
Data source changes index computing module 210 for when data source or index operator calculating logic change, batch to be counted
Calculate user's index.
Business triggers index computing module 220 for calculating user's index after Indexes Abnormality or new user's registration.
Offline index computing module 230 is calculated in offline index when trigger condition occurs calculates user by the way of offline
Index.It is following any bar that offline index, which calculates trigger condition,：1, when newly-increased index or more New Set calculating logic；2, it needs
When wanting the newer index of timing, reach renewal time；3, business triggering parameter calculate in real time pressure it is overweight when, low priority
Index detects task in memory queue when being downgraded to the processing of off-line calculation index or data source change triggering parameter automatically
Quantity is handled low priority index task drop power in memory queue to off-line calculation index more than after defined threshold value；4, when
Upstream and downstream data source occurs when problem leads to corrupt data, loss or postpones or business personnel has found that certain indexs calculating are asked
When topic.
Index of the trigger module 240 belonging to user's index calculates type, triggering and triggering when trigger condition occurs
The corresponding data source change index computing module 210 of condition or 220 or offline index of business triggering index computing module calculate mould
Block 230 carries out user's index calculating.
Source change index computing module 210, business trigger index computing module to index memory module 250 for storing data
220, the index after offline index computing module 230 calculates.
Registration center 260 is for storing user's index and index operator corresponding with user's index.When the index that Adds User
It is also to be operated by registration center 260 when with change index calculating logic.Number is triggered according to trigger condition trigger module 240
Index computing module 220 or offline index computing module 230 are triggered into row index according to source change index computing module 210 or business
It calculates, in 220 or offline index computing module of data source change index computing module 210 or business triggering index computing module
When 230 parameter, data source is obtained according to the index operator of registration center 260, according to index calculating logic into row index meter
It calculates.
Index memory module 250 also updates storage module 251 and timing module 252 comprising index.Index updates storage mould
Block 251 is used to store the update cycle for needing timing newer index and each index.Timing module 252 is used to update index
The each index stored in memory module 251 carries out timing.After the timing time to index reaches the update cycle of the index,
Offline index is triggered to calculate.
Registration center 260 includes subscribing module 261, and subscribing module 261 subscribes to the data source needed when index calculates.Work as number
When changing according to source, the index trigger data source change triggering for subscribing to this data source calculates.
When data source change index computing module 210 when being calculated into row index, if monitoring thread find memory queue in
Task quantity is more than threshold value, by low priority index task drop power in memory queue, is put into offline index computing module 230 and calculates
Task queue in carry out processed offline.When business trigger index computing module 220 when being calculated into row index, if business trigger
Index computing module 220 (API) in real time calculate pressure it is overweight (i.e. with computing unit simultaneously calculating task it is excessive), can will it is low preferentially
Grade index computation requests, which are put into the task queue that offline index computing module 230 calculates, carries out processed offline, and notifies business
System obtains after a period of time.To effectively ensure the high excellent execution for calculating fast operator, the stability and reality of system are improved
When handling capacity.
The above embodiment is the preferred case of the present invention, is not intended to limit protection scope of the present invention.
Claims (9)
1. user's index distributed computing method in a kind of finance, which is characterized in that include the following steps：
Step 1, the calculating of user's index is divided into three kinds of indexs and calculates type：Off-line calculation, business triggering calculates and data source
Change triggering calculates；
Step 2, user's index, including user's Criterion Attribute and index operator are defined, the index operator of definition includes definition
Data source, index calculating logic, the index used when index calculating calculates type, and index calculates type extremely in the index operator
It is one less；
Step 3, definition triggers the condition that three kinds of indexs calculate type；
Step 4, the index belonging to user's index calculates type, when trigger condition occurs using corresponding with trigger condition
Index calculates type and is calculated into row index；
Step 5, the user's index calculated is stored.
2. user's index distributed computing method in finance according to claim 1, it is characterised in that：
Wherein, the trigger condition of off-line calculation index is to meet following either condition：
When Add User index or more New Set calculating logic when；
When needing the newer user's index of timing, reach renewal time；
Business triggering parameter calculate in real time pressure it is overweight when, low priority index is downgraded to automatically at off-line calculation index
After detecting in memory queue that task quantity is more than defined threshold value when reason or data source change triggering parameter, by memory
Low priority index task drop power is handled to off-line calculation index in queue；
When upstream and downstream data source generation problem leads to corrupt data, loss or delay or business personnel has found certain index meters
When calculating problematic.
3. user's index distributed computing method in finance according to claim 1, it is characterised in that：
Wherein, the trigger condition of business triggering parameter is：
Business side finds user's Indexes Abnormality, miscalculation or expired；Or
After new user's registration.
4. user's index distributed computing method in finance according to claim 1, it is characterised in that：
Wherein, the trigger condition of data source change triggering parameter is：Data source changes and/or index calculating logic changes.
5. user's index distributed computing method in finance according to claim 1, it is characterised in that：
Wherein, the computational methods of data source change triggering calculating are：
After data source and/or the change of index calculating logic, it is that the index that data source change triggering calculates exists to find out pointer type
The index for changing data source and/or index calculating logic is put into data source change triggering meter by the data source needed when calculating
Calculate memory queue；
It is changed in triggering calculating memory queue from data source and obtains index in batches, pre-processed；
Pretreated index is calculated according to changed data source and/or index calculating logic；
User's index after calculating is stored.
6. user's index distributed computing method in finance according to claim 5, it is characterised in that：
Wherein, changing the pretreated method of index progress in triggering calculating memory queue to data source is：
Duplicate removal, the change time aggregation operation of the same data source are carried out to user's index.
7. user's index distributed computing system in a kind of finance, which is characterized in that include：
Registration center, for storing user's index and index operator corresponding with user's index；
Data source changes index computing module, refers to for calculating user in batches in data source or the change of index operator calculating logic
Mark；
Business triggers index computing module, for calculating user when business side finds user's Indexes Abnormality or after new user's registration
Index；
Offline index computing module refers to for calculating user by the way of offline when offline index calculates trigger condition generation
Mark；
Trigger module, the index belonging to user's index calculate type, triggering and trigger condition pair when trigger condition occurs
The data source change index computing module or business triggering index computing module answered or offline index computing module are used
Family index calculates；And
Index memory module, for storing user's index after calculating,
Wherein, it is to meet following either condition that the offline index, which calculates trigger condition,：
When newly-increased index or more New Set calculating logic；
When needing the newer index of timing, reach renewal time；
Business triggering parameter calculate in real time pressure it is overweight when, low priority index is downgraded to automatically at off-line calculation index
After detecting in memory queue that task quantity is more than defined threshold value when reason or data source change triggering parameter, by memory
Low priority index task drop power is handled to off-line calculation index in queue；
When upstream and downstream data source generation problem leads to corrupt data, loss or delay or business personnel has found certain index meters
When calculating problematic.
8. user's index distributed computing system in finance according to claim 7, it is characterised in that：
The memory module includes：Index updates storage module and timing module, and the index updates storage module for depositing
Storage needs the update cycle of the newer index of timing and each index, the timing module be used for each index into
Row timing.
9. user's index distributed computing system in finance according to claim 7, it is characterised in that：
The registration center includes subscribing module, and the subscribing module subscribes to the data source needed when index calculates.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810347176.2A CN108573348B (en) | 2018-04-18 | 2018-04-18 | Financial index distributed computing method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810347176.2A CN108573348B (en) | 2018-04-18 | 2018-04-18 | Financial index distributed computing method and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108573348A true CN108573348A (en) | 2018-09-25 |
| CN108573348B CN108573348B (en) | 2021-01-01 |
Family
ID=63574965
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810347176.2A Active CN108573348B (en) | 2018-04-18 | 2018-04-18 | Financial index distributed computing method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108573348B (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109447443A (en) * | 2018-10-18 | 2019-03-08 | 阳光人寿保险股份有限公司 | Index calculating method and device |
| CN109960560A (en) * | 2019-03-29 | 2019-07-02 | 北京九章云极科技有限公司 | A kind of index processing method and system |
| CN111680048A (en) * | 2020-08-14 | 2020-09-18 | 支付宝(杭州)信息技术有限公司 | Aggregation task processing method, device, server and medium |
| CN111813871A (en) * | 2020-06-30 | 2020-10-23 | 北京九章云极科技有限公司 | Real-time data processing system and method |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP2927861A1 (en) * | 2014-03-31 | 2015-10-07 | Sap Se | In-memory database system for the management of variant configuration of a product |
| CN105677836A (en) * | 2016-01-05 | 2016-06-15 | 北京汇商融通信息技术有限公司 | Big data processing and solving system simultaneously supporting offline data and real-time online data |
| CN107122431A (en) * | 2017-04-14 | 2017-09-01 | 浙江数链科技有限公司 | A kind of real-time calculating platform and the method for computing data based on real-time calculating platform |
| CN107169070A (en) * | 2017-05-08 | 2017-09-15 | 山大地纬软件股份有限公司 | The constructing system and its method in a kind of social security index warehouse based on big data |
| CN107358354A (en) * | 2017-07-10 | 2017-11-17 | 深圳市分期乐网络科技有限公司 | User property computational methods and device for decision in the face of risk |
-
2018
- 2018-04-18 CN CN201810347176.2A patent/CN108573348B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP2927861A1 (en) * | 2014-03-31 | 2015-10-07 | Sap Se | In-memory database system for the management of variant configuration of a product |
| CN105677836A (en) * | 2016-01-05 | 2016-06-15 | 北京汇商融通信息技术有限公司 | Big data processing and solving system simultaneously supporting offline data and real-time online data |
| CN107122431A (en) * | 2017-04-14 | 2017-09-01 | 浙江数链科技有限公司 | A kind of real-time calculating platform and the method for computing data based on real-time calculating platform |
| CN107169070A (en) * | 2017-05-08 | 2017-09-15 | 山大地纬软件股份有限公司 | The constructing system and its method in a kind of social security index warehouse based on big data |
| CN107358354A (en) * | 2017-07-10 | 2017-11-17 | 深圳市分期乐网络科技有限公司 | User property computational methods and device for decision in the face of risk |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109447443A (en) * | 2018-10-18 | 2019-03-08 | 阳光人寿保险股份有限公司 | Index calculating method and device |
| CN109960560A (en) * | 2019-03-29 | 2019-07-02 | 北京九章云极科技有限公司 | A kind of index processing method and system |
| CN109960560B (en) * | 2019-03-29 | 2019-12-10 | 北京九章云极科技有限公司 | Index processing method and system |
| CN111813871A (en) * | 2020-06-30 | 2020-10-23 | 北京九章云极科技有限公司 | Real-time data processing system and method |
| CN111680048A (en) * | 2020-08-14 | 2020-09-18 | 支付宝(杭州)信息技术有限公司 | Aggregation task processing method, device, server and medium |
| CN111680048B (en) * | 2020-08-14 | 2021-06-29 | 支付宝(杭州)信息技术有限公司 | Aggregation task processing method, device, server and medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108573348B (en) | 2021-01-01 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108573348A (en) | Financial indicator distributed computing method and its system | |
| JP6707564B2 (en) | Data quality analysis | |
| CN108090225A (en) | Operation method, device, system and the computer readable storage medium of database instance | |
| CN107851106A (en) | It is the resource scaling of the automatic requirement drive serviced for relational database | |
| CN104915846A (en) | Electronic commerce time sequence data anomaly detection method and system | |
| CN110135711A (en) | A kind of information management method and device | |
| CN106251178A (en) | Data digging method and device | |
| CN109191133A (en) | The selection method and terminal device of channel of disbursement | |
| CN109472693A (en) | A kind of ATM cash concocting method and device | |
| CN112785057A (en) | Component prediction method, device, equipment and storage medium based on exponential smoothing | |
| CN109934700A (en) | A kind of method and device of arbitrage detecting | |
| CN115829714A (en) | Financial product data dynamic monitoring method and system | |
| US20240265463A1 (en) | Digital system comprising finite state machine engines for automated processing of flexible configurable individual policies segmented by life-cycle states, inter alia, using a sliced admission control triggered by unscheduled sum cover increase, and method thereof | |
| CN107688901A (en) | Data adjustment method and device | |
| CN105005575A (en) | Quick developing interface method for enterprise intelligent prediction | |
| CN111415261B (en) | Control method, system and device for dynamically updating flow control threshold of bank system | |
| CN104321800A (en) | Price target builder | |
| CN108549981A (en) | Method for improving service quality of large-batch parallel business process | |
| Li et al. | Fluctuation-driven price dynamics and investment strategies | |
| EP2056240A1 (en) | Method for efficient product assessment | |
| CN109360118A (en) | A kind of plant states monitoring method, device, system, equipment and storage medium | |
| CN112199359B (en) | Data checking method, device, electronic equipment and storage medium | |
| CN110472827B (en) | System policy early warning method, device, server and readable storage medium | |
| CN112633936A (en) | Supplier hidden cost prediction method, supplier hidden cost prediction device, server and storage medium | |
| CN106681967A (en) | Meter changing operation method based on data backtracking mechanism |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20240104 Address after: 200000 Shanghai Yangpu District National Road 127 16 Floor (Centralized Registration Place) Patentee after: Xinyi Search (Shanghai) Digital Technology Co.,Ltd. Address before: 200433 room 8, 14 building, 18 Government Road, Yangpu District, Shanghai. Patentee before: XINYONG COMPUTING INFORMATION TECHNOLOGY (SHANGHAI) Co.,Ltd. |