CN109697567A - A kind of real-time method for prewarning risk of big data and system - Google Patents
A kind of real-time method for prewarning risk of big data and system Download PDFInfo
- Publication number
- CN109697567A CN109697567A CN201811607509.7A CN201811607509A CN109697567A CN 109697567 A CN109697567 A CN 109697567A CN 201811607509 A CN201811607509 A CN 201811607509A CN 109697567 A CN109697567 A CN 109697567A
- Authority
- CN
- China
- Prior art keywords
- real
- time
- data
- transaction
- computing engines
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/06—Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
- G06Q10/063—Operations research, analysis or management
- G06Q10/0635—Risk analysis of enterprise or organisation activities
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q40/00—Finance; Insurance; Tax strategies; Processing of corporate or income taxes
- G06Q40/02—Banking, e.g. interest calculation or account maintenance
Landscapes
- Business, Economics & Management (AREA)
- Engineering & Computer Science (AREA)
- Human Resources & Organizations (AREA)
- Strategic Management (AREA)
- Economics (AREA)
- Entrepreneurship & Innovation (AREA)
- General Business, Economics & Management (AREA)
- Finance (AREA)
- Development Economics (AREA)
- Marketing (AREA)
- Accounting & Taxation (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Physics & Mathematics (AREA)
- Educational Administration (AREA)
- Tourism & Hospitality (AREA)
- Quality & Reliability (AREA)
- Operations Research (AREA)
- Game Theory and Decision Science (AREA)
- Technology Law (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a kind of real-time method for prewarning risk of big data and systems, are related to real-time Risk-warning field, this method comprises: including: to turn over to write real-time transaction data to large-scale data computing engines from raw data base；It carries out adding up to summarize according to account information by real-time transaction data of the large-scale data computing engines to access, the summarized results of real-time update corresponding account information；The summarized results of update is analyzed respectively by large-scale data computing engines, obtains early warning analysis result.The present invention will calculate and raw data base is isolated, and while guaranteeing real-time early warning, does not influence the normal operation of regular traffic, improves the overall treatment efficiency and operation stability of whole system.
Description
Technical field
The present invention relates to real-time Risk-warning field more particularly to a kind of real-time method for prewarning risk of big data and systems.
Background technique
In recent years, as banking is constantly brought forth new ideas development, industry confusion also numerous and complicated, bank is as risk management master
Body, it is further urgent to real-time Risk-warning demand.
With being continuously increased for early warning project, Early-warning Model is increasingly complicated, traditional Risk-warning be call directly it is original
Real-time transaction data in database is analyzed, and the data of raw data base can be all called when because of business transaction, is read and write non-
Often frequently, it can only realize the Risk-warning of T+1, just can guarantee the operation for not influencing regular traffic transaction.Such as: it trades to today
The analysis and early warning of information can only be completed tomorrow, that is to say, that the same day can only inquire yesterday and pervious Risk-warning knot
Fruit.The Risk-warning of this T+1 has been unable to satisfy the requirement of present early warning timeliness.
Summary of the invention
The object of the present invention is to provide a kind of real-time method for prewarning risk of big data and systems, provide real-time Risk-warning
Meanwhile it will not the operation traded to regular traffic of image.
Technical solution provided by the invention is as follows:
A kind of real-time method for prewarning risk of big data, comprising: turned over from raw data base and write real-time transaction data to big rule
Modulus is according to computing engines；Added up by real-time transaction data of the large-scale data computing engines to access according to account information
Summarize, the summarized results of real-time update corresponding account information；Knot is summarized to update respectively by large-scale data computing engines
Fruit is analyzed, and early warning analysis result is obtained.
In the above-mentioned technical solutions, when carrying out real-time Risk-warning, first the real-time transaction data in raw data base is turned over
It writes, calculating process is isolated with raw data base, the calling of data will not influence the property of raw data base when calculating
Can, it more will not influence the operation of regular traffic transaction system.
Further, described to turn over that write real-time transaction data specific to large-scale data computing engines from raw data base
Are as follows: it is turned over from raw data base and writes real-time transaction data to real-time Message Passing system；Pass through the real-time Message Passing system
The real-time transaction data is accessed in large-scale data computing engines.
Further, the summarized results of update is analyzed respectively by large-scale data computing engines, obtains early warning point
Analysing result includes: respectively to be compared the summarized results of update with default early warning rule by large-scale data computing engines,
If meeting the trigger condition of default early warning rule, the corresponding warning information of summarized results of the update is generated.
In the above-mentioned technical solutions, the summarized results updated every time can be all compared with default early warning rule, ensure that
The real-time of warning information.
Further, the default early warning rule includes any of the following or a variety of: the single card odd-numbered day is accumulative to occur amount
Limitation, the accumulative transaction stroke count limitation of single card odd-numbered day, high risk zone transaction limits.
In the above-mentioned technical solutions, the set-up mode of default early warning rule is given, the setting of diversification is realized to a variety of
The monitoring of trading situation.
Further, further includes: store the warning information to presetting database.
Further, further includes: every real-time transaction data of access is stored in by large-scale data computing engines and is preset
In database.
In the above-mentioned technical solutions, warning information and real-time transaction data are stored to presetting database, with extensive number
It is isolated according to computing engines, makes to calculate and store isolation, the runnability both guaranteed is independent of each other, and when use has good sound
Answer speed.
Further, further includes: when receiving transaction detail query instruction, the friendship is transferred from the presetting database
The easily corresponding transaction details table of detail inquiry instruction.
In the above-mentioned technical solutions, it is directly inquired from presetting database, there is good response speed.
The present invention also provides a kind of real-time Warning Systems of big data, comprising: data turn over writing module, are used for from original number
Real-time transaction data is write to large-scale data computing engines according to turning in library；The large-scale data computing engines, to the reality of access
When transaction data according to account information carry out it is accumulative summarize, the summarized results of real-time update corresponding account information；And it is right respectively
The summarized results of update is analyzed, and early warning analysis result is obtained.
In the above-mentioned technical solutions, when carrying out real-time Risk-warning, first the real-time transaction data in raw data base is turned over
It writes, calculating process is isolated with raw data base, the calling of data will not influence the property of raw data base when calculating
Can, it more will not influence the operation of regular traffic transaction system.
Further, the large-scale data computing engines, respectively analyze the summarized results of update, obtain early warning point
Analysing result includes: the large-scale data computing engines, by large-scale data computing engines respectively by the summarized results of update
It is compared with default early warning rule, if meeting the trigger condition of default early warning rule, generates the summarized results of the update
Corresponding warning information.
Further, further includes: presetting database, for storing every real-time transaction data；Enquiry module, for working as
When receiving transaction detail query instruction, the transaction detail query is transferred from the presetting database and instructs corresponding transaction
Detail list.
Compared with prior art, the real-time method for prewarning risk of big data of the invention and system beneficial effect are:
The present invention will calculate and raw data base is isolated, and while guaranteeing real-time early warning, does not influence regular traffic
It operates normally, and when calculating and storage is isolated again, ensure that response speed when calculating and inquiry, improve whole system
Overall treatment efficiency and operation stability.
Detailed description of the invention
Below by clearly understandable mode, preferred embodiment is described with reference to the drawings, to a kind of real-time risk of big data
Method for early warning and above-mentioned characteristic, technical characteristic, advantage and its implementation of system are further described.
Fig. 1 is the flow chart of the real-time method for prewarning risk one embodiment of big data of the present invention；
Fig. 2 is the flow chart of real-time another embodiment of method for prewarning risk of big data of the present invention；
Fig. 3 is the structural schematic diagram of the real-time Warning System one embodiment of this hair big data；
Fig. 4 is the practical architecture diagram of the real-time Warning System one embodiment of big data of the present invention.
Drawing reference numeral explanation:
10. data turn over writing module, 20. real-time Message Passing systems, 30. large-scale data computing engines, 40. preset datas
Library, 50. enquiry modules.
Specific embodiment
In order to more clearly explain the embodiment of the invention or the technical proposal in the existing technology, Detailed description of the invention will be compareed below
A specific embodiment of the invention.It should be evident that drawings in the following description are only some embodiments of the invention, for
For those of ordinary skill in the art, without creative efforts, it can also be obtained according to these attached drawings other
Attached drawing, and obtain other embodiments.
To make simplified form, part related to the present invention is only schematically shown in each figure, they are not represented
Its practical structures as product.In addition, there is identical structure or function in some figures so that simplified form is easy to understand
Component only symbolically depicts one of those, or has only marked one of those.Herein, "one" is not only indicated
" only this ", can also indicate the situation of " more than one ".
The present invention is the real-time Risk-warning realized based on big data technology, by multiple technologies, by some business scenarios
Under the Risk-warning timeliness that (is set as needed) improved by T+1 to real-time.
Fig. 1 shows the embodiment of a real-time method for prewarning risk of big data of the invention, comprising:
S101 is turned over from raw data base writes real-time transaction data to large-scale data computing engines.
Specifically, raw data base refers to database used by existing business transaction system, mainly record each
The real-time transaction data (i.e. real-time flowing water information) of bank card, core accounting data etc..Traditional method for prewarning risk is direct
It calls the real-time transaction data in raw data base to be analyzed, the data of raw data base can be all called when because of business transaction,
Its read-write is very frequent, can only realize the Risk-warning of T+1, just can guarantee the operation for not influencing regular traffic transaction.
The real-time transaction data of raw data base is turned over and is written to large-scale data computing engines by the present embodiment, is equivalent to turn over and be write
To another place, called for subsequent Risk-warning.
Optionally, S101 is turned over from raw data base writes real-time transaction data to large-scale data computing engines specifically:
It is turned over from raw data base and writes real-time transaction data to real-time Message Passing system；It will be handed in real time by real-time Message Passing system
Easy data access is into large-scale data computing engines.
It is realized specifically, turning over and writing using IBM CDC (IBM InfoSphere Change Data Capture) technology, energy
Enough near-real-times replicate isomeric data, to support Data Migration, application program integration, data synchronization, dynamic storage, master data
(MDM), business diagnosis and data quality process are managed, the data for being capable of providing DB2 to Kafka are synchronous.
Real-time Message Passing system is that a kind of distributed post of high-throughput subscribes to message system using Kafka,
It can handle the everything flow data in the website of consumer's scale.
The setting of real-time Message Passing system be in order to by real-time transaction data be transferred to large-scale data computing engines into
The analysis of row Risk-warning.
Large-scale data computing engines, can be used Spark, aim at large-scale data processing and the Universal-purpose quick that designs
Computing engines carry out the real-time analysis of Risk-warning using stream calculation ability in Spark.Certainly, other to be counted on a large scale
It can also be used according to the system or engine of calculating, this is not restricted.
It should be noted that writing Kafka because IBM CDC can only be turned over, therefore, real-time Message Passing system is increased, led to
It crosses it to access to real-time transaction data in large-scale data computing engines, if other technologies can directly turn over real-time transaction data
It is written in large-scale data computing engines, can skip this step of real-time Message Passing system.
S102 is by large-scale data computing engines to the real-time transaction data of access according to account information (account information packet
Include: the account that card number and/or client open is arranged according to actual needs) carry out it is accumulative summarize, real-time update corresponding account information
Summarized results.
Specifically, Risk-warning can generally implement to specific card number or specific account, therefore, adding up to summarize is to same
Real-time transaction data under one card number or account is summarized.
Such as: the debit card that card number is 3102564823256, on October 20th, 2018,18:30:20 have occurred one and turn
Account transaction, the amount of money are 500 yuan, and turning single account is 62263024485200, and traction equipment 0056, here it is a real-time deals
Data can be added up to be summarised in 3102564823256 card number under one's name；A withdrawal has occurred again and hands over for this card of 19:20:23
Easily, then it is summarised in 3102564823256 card number again under one's name.It is no longer superfluous herein similarly to carry out adding up to summarize according to account
It states.
Preferably, add up when summarizing, according to the type of transaction of real-time transaction data (such as: transferring accounts, remit money, withdraw the money) into
Row summarizes respectively, and subsequent Risk-warning is facilitated to analyze.
S103 respectively analyzes the summarized results of update by large-scale data computing engines, obtains early warning analysis knot
Fruit.
Specifically, illustrating that it produces new transaction under one's name whenever there is the summarized results an of account or card number to be updated
Information, the summarized results after combining accumulative summarize in real time are analyzed, convenient for generating early warning analysis result in real time.
Optionally, the real-time method for prewarning risk of big data, further includes: by large-scale data computing engines by the every of access
In real-time transaction data deposit presetting database (optional, to be stored in the card detail list of presetting database).
Specifically, every real-time transaction data other than being called for risk early warning analysis, they can also be stored to
In presetting database, for the inquiry of subsequent applications layer.
Optionally, the real-time method for prewarning risk of big data, further includes: when receiving transaction detail query instruction, from pre-
If (card detail list) transfers transaction detail query and instructs corresponding transaction details table in database.
Specifically, it is bright to transfer transaction in client input transaction detail query instruction from presetting database by user
The corresponding transaction details table of thin inquiry instruction, is checked for user.Transaction detail query instruction can be by selecting specific card number, account
The information such as family, equipment are inquired.
Transaction details table includes: card transaction flow water meter, account flowing water, equipment essential information etc.；It is traded by inquirying card bright
Thin table obtains.
Presetting database is independently of the database of large-scale data computing engines, real-time Message Passing system, realizes
Storage and the isolation calculated call data when application layer inquiry from presetting database, will not influence large-scale data calculating
The calculating of engine, ensure that between calculating, inquiry will not mutual image, improve response speed.Presetting database can be used
Hbase, Hive distributed database.
In the present embodiment, when carrying out real-time Risk-warning, first the real-time transaction data in raw data base is turned over and is write,
Calculating process is isolated with raw data base, the calling of data will not influence the performance of raw data base when calculating, less
It will affect the operation of regular traffic transaction system.
In addition, every real-time transaction data is stored to presetting database, and it is independent with large-scale data computing engines, work as user
When inquiring on the client, application layer carries out the calling of data directly from presetting database, will not influence large-scale data meter
The calculated performance of engine is calculated, realizes the isolation for calculating and inquiring, the process performance of both guarantees, realizes real-time respectively.
Fig. 2 shows the real-time method for prewarning risk of another big data of the invention, comprising:
S201 is turned over from raw data base writes real-time transaction data to large-scale data computing engines specifically: S211 is from original
It is turned in beginning database and writes real-time transaction data to real-time Message Passing system；S221 will be handed in real time by real-time Message Passing system
Easy data access is into large-scale data computing engines.
S202 carries out accumulative remittance according to account information by real-time transaction data of the large-scale data computing engines to access
Always, the summarized results of real-time update corresponding account information.
The present embodiment explanation part same as the previously described embodiments is not repeated to describe, and refers to above-described embodiment.
S203 respectively analyzes the summarized results of update by large-scale data computing engines, obtains early warning analysis knot
Fruit includes:
The summarized results of update is compared by S213 with default early warning rule respectively by large-scale data computing engines,
If meeting the trigger condition of default early warning rule, the corresponding warning information of summarized results of update is generated, if not meeting default
The trigger condition of early warning rule does not generate the corresponding warning information of summarized results of update then.
Specifically, analytic process is to compare the summarized results of update and a rule in default early warning rule
Compared with judging whether to meet wherein one or more trigger conditions, if yes, just generate corresponding warning information.
Default early warning rule be arranged according to actual demand, such as: it presets early warning rule and includes any of the following or more
Kind: the accumulative generation amount limitation of single card odd-numbered day, the accumulative transaction stroke count limitation of single card odd-numbered day, high risk zone transaction limits.
The accumulative generation amount limitation of single card odd-numbered day includes: that individual (debit/credit) card odd-numbered day cumulative consumption amount of money is greater than
The odd-numbered day highest cumulative consumption amount of money of (such as: 6 months, 3 months etc.) in nearest certain time；Individual (debit/credit) card is single
The odd-numbered day highest that day accumulative withdrawal amount is greater than in nearest certain time (such as: 6 months, 8 months etc.) adds up withdrawal amount；It is single
Debit card odd-numbered day add up transfer amounts be greater than in nearest certain time (such as: 6 months, 4 months etc.) odd-numbered day highest it is accumulative
Transfer amounts etc..
The accumulative transaction stroke count limitation of single card odd-numbered day includes: that individual (debit/credit) card odd-numbered day cumulative consumption stroke count is greater than
Certain odd-numbered day highest cumulative consumption stroke count (such as: 10,8 etc.)；It is big that individual (debit/credit) blocks odd-numbered day accumulative withdrawal stroke count
Add up withdrawal stroke count (such as: 12,8 etc.) in certain odd-numbered day highest；Individual debit card odd-numbered day, accumulative stroke count of transferring accounts was greater than one
The accumulative stroke count of transferring accounts of order day highest (such as: 10,5 etc.)；Individual credit card odd-numbered day same transaction type adds up identical gold
Volume authorization stroke count is greater than the accumulative authorization stroke count of certain odd-numbered day highest (such as: 3).
High risk zone transaction limits include: in day trade transaction data acquirer be the associated mechanisms rule such as the Banking Supervision Commission, Unionpay
Fixed high-risk country, bad holder's standard.
It is then to generate one when the summarized results of update meets the wherein trigger condition of any one default early warning rule
If corresponding warning information generates the corresponding corresponding warning information of N item trigger N simultaneously, and N is the integer greater than 1.
If trigger N simultaneously, corresponding 1 warning information is also produced, contains the early warning of all triggerings in this warning information
Situation.
Optionally, the real-time method for prewarning risk of big data, further includes: S205 will be accessed by large-scale data computing engines
Every real-time transaction data deposit presetting database in (in card detail list)；When receiving transaction detail query instruction,
Transaction detail query is transferred from presetting database (card detail list) instructs corresponding transaction details table.
Preferably, the real-time method for prewarning risk of big data, further includes: S204 stores warning information to presetting database
(early warning rules results table).
Specifically, warning information is stored in presetting database when generating warning information, for calling when subsequent query.
Warning information and account information, are set early warning type (if each item is preset if early warning rule has one's own early warning type)
The information associations storage such as standby, facilitates the inquiry of subsequent user.
Preferably, the real-time method for prewarning risk of big data, further includes: when receiving the instruction of early warning result queries, from pre-
If transferring early warning result queries in database instructs corresponding warning information and transaction details table.
Specifically, user in client by inputting, the instruction of early warning result queries carries out corresponding warning information and transaction is bright
Thin table is checked.The display of transaction details table is that user understands specific transaction details for convenience, convenient for comparing.
Presetting database can be individually made of Hbase, Hive distributed database, can also be by Hbase, Hive equal distribution
Formula database and Solr component collectively constitute.Solr component is a high-performance, the full-text search technology based on Lucene, mainly
Realize value to key retrieval, supplement provides the index function of Hbase, Hive distributed database, improves search response
Speed.
The present embodiment has updated summarized results every time and will be compared with default early warning rule, judges whether there is early warning letter
Breath generates, and realizes the real-time of Risk-warning.And warning information is stored in the present count independently of large-scale data computing engines
According to response speed when in library, realizing the isolation of calculating and storage, while guaranteeing calculating and inquiring, whole system is improved
Overall treatment efficiency and operation stability.
It should be noted that, sequence that process of the invention illustrates and with no restriction, as long as it is real-time to complete big data
Risk-warning.
Fig. 3 shows one embodiment of the real-time Warning System of big data of the present invention, comprising:
Data turn over writing module 10, draw for turning over to write real-time transaction data and calculate to large-scale data from raw data base
It holds up.
Specifically, raw data base refers to database used by existing business transaction system, mainly record each
The real-time transaction data (i.e. real-time flowing water information) of bank card, core accounting data etc..Traditional method for prewarning risk is direct
It calls the real-time transaction data in raw data base to be analyzed, the data of raw data base can be all called when because of business transaction,
Its read-write is very frequent, can only realize the Risk-warning of T+1, just can guarantee the operation for not influencing regular traffic transaction.
The real-time transaction data of raw data base is turned over and is written to large-scale data computing engines by the present embodiment, is equivalent to turn over and be write
To another place, called for subsequent Risk-warning.
Optionally, data turn over writing module 10, write real-time transaction data to large-scale data for turning over from raw data base
Computing engines specifically: data turn over writing module 10, turn over from raw data base and write real-time transaction data to real-time Message Passing system
System 20；Real-time Message Passing system 20 accesses to real-time transaction data in large-scale data computing engines 30.
Specifically, turn over write using IBM CDC technology realize, can near-real-time replicate isomeric data, to support data
Migration, application program integration, data synchronization, dynamic storage, master data management (MDM), business diagnosis and data quality process, energy
The data for enough providing DB2 to Kafka are synchronous.
Real-time Message Passing system is that a kind of distributed post of high-throughput subscribes to message system using Kafka,
It can handle the everything flow data in the website of consumer's scale.
The setting of real-time Message Passing system be in order to by real-time transaction data be transferred to large-scale data computing engines into
The analysis of row Risk-warning.
Large-scale data computing engines, can be used Spark, aim at large-scale data processing and the Universal-purpose quick that designs
Computing engines carry out the real-time analysis of Risk-warning using stream calculation ability in Spark.Certainly, other to be counted on a large scale
It can also be used according to the system or engine of calculating, this is not restricted.
It should be noted that writing Kafka because IBM CDC can only be turned over, therefore, real-time Message Passing system is increased, led to
It crosses it to access to real-time transaction data in large-scale data computing engines, if other technologies can directly turn over real-time transaction data
It is written in large-scale data computing engines, can skip this step of real-time Message Passing system.
Large-scale data computing engines 30, to the real-time transaction data of access, according to account information, (account information includes: card
Number and/or the account opened of client, be arranged according to actual needs) carry out it is accumulative summarize, real-time update corresponding account information summarizes
As a result；And the summarized results of update is analyzed respectively, obtain early warning analysis result.
Specifically, Risk-warning can generally implement to specific card number or specific account, therefore, adding up to summarize is to same
Real-time transaction data under one card number or account is summarized.Specific example refers to corresponding embodiment of the method, herein
It repeats no more.
Preferably, add up when summarizing, according to the type of transaction of real-time transaction data (such as: transferring accounts, remit money, withdraw the money) into
Row summarizes respectively, and subsequent Risk-warning is facilitated to analyze.
Whenever there is the summarized results an of account or card number to be updated, illustrate that it produces new Transaction Information under one's name, it is real
When analyzed in conjunction with the summarized results after accumulative summarize, convenient for generating early warning analysis result in real time.
Optionally, the real-time Warning System of big data, further includes:
Presetting database 40 (optionally, is stored in the card detail of presetting database for storing every real-time transaction data
In table).
Specifically, every real-time transaction data other than being called for risk early warning analysis, they can also be stored to
In presetting database, for the inquiry of subsequent applications layer.
Enquiry module 50, for when receive transaction detail query instruction when, from presetting database (card detail list)
It transfers transaction detail query and instructs corresponding transaction details table.
Specifically, it is bright to transfer transaction in client input transaction detail query instruction from presetting database by user
The corresponding transaction details table of thin inquiry instruction, is checked for user.Transaction detail query instruction can be by selecting specific card number, account
The information such as family, equipment are inquired.
Transaction details table includes: card transaction flow water meter, account flowing water, equipment essential information etc.；It is traded by inquirying card bright
Thin table obtains.
Presetting database is independently of the database of large-scale data computing engines, real-time Message Passing system, realizes
Storage and the isolation calculated call data when application layer inquiry from presetting database, will not influence large-scale data calculating
The calculating of engine, ensure that between calculating, inquiry will not mutual image, improve response speed.Presetting database can be used
Hbase, Hive distributed database.
In the present embodiment, when carrying out real-time Risk-warning, first the real-time transaction data in raw data base is turned over and is write,
Calculating process is isolated with raw data base, the calling of data will not influence the performance of raw data base when calculating, less
It will affect the operation of regular traffic transaction system.
In addition, every real-time transaction data is stored to presetting database, and it is independent with large-scale data computing engines, work as user
When inquiring on the client, application layer carries out the calling of data directly from presetting database, will not influence large-scale data meter
The calculated performance of engine is calculated, realizes the isolation for calculating and inquiring, the process performance of both guarantees, realizes real-time respectively.
In the embodiment of the real-time Warning System of another big data of the invention, comprising:
Data turn over writing module 10, write real-time transaction data to large-scale data computing engines for turning over from raw data base
Specifically: data turn over writing module 10, turn over from raw data base and write real-time transaction data to real-time Message Passing system；Disappear in real time
Transmission system 20 is ceased, real-time transaction data is accessed in large-scale data computing engines 30.
Large-scale data computing engines 30 carry out adding up to summarize to the real-time transaction data of access according to account information, real
The summarized results of Shi Gengxin corresponding account information；And the summarized results of update is analyzed respectively, obtain early warning analysis knot
Fruit.
The present embodiment explanation part same as the previously described embodiments is not repeated to describe, and refers to above-described embodiment.
Large-scale data computing engines respectively analyze the summarized results of update, obtain early warning analysis result and include:
Large-scale data computing engines by the summarized results of update and are preset pre- respectively by large-scale data computing engines
Police regulations are then compared, if meeting the trigger condition of default early warning rule, generate the corresponding early warning letter of summarized results of update
Breath, if not meeting the trigger condition of default early warning rule, does not generate the corresponding warning information of summarized results of update.
Specifically, analytic process is to compare the summarized results of update and a rule in default early warning rule
Compared with judging whether to meet wherein one or more trigger conditions, if yes, just generate corresponding warning information.
Default early warning rule be arranged according to actual demand, such as: it presets early warning rule and includes any of the following or more
Kind: the accumulative generation amount limitation of single card odd-numbered day, the accumulative transaction stroke count limitation of single card odd-numbered day, high risk zone transaction limits.Respectively
The explanation of default early warning rule refers to corresponding embodiment of the method, and details are not described herein.
It is then to generate one when the summarized results of update meets the wherein trigger condition of any one default early warning rule
If corresponding warning information generates the corresponding corresponding warning information of N item trigger N simultaneously, and N is the integer greater than 1.
If trigger N simultaneously, corresponding 1 warning information is also produced, contains the early warning of all triggerings in this warning information
Situation.
Optionally, the real-time Warning System of big data, further includes: presetting database 40 is handed in real time for storing every
Easy data；Enquiry module 50, for transferring transaction details from presetting database and looking into when receiving transaction detail query instruction
It askes and instructs corresponding transaction details table.
Optionally, presetting database 40 are further used for storing warning information (early warning rules results table)；Enquiry module
50, it is further used for transferring the instruction pair of early warning result queries from presetting database when receiving the instruction of early warning result queries
The warning information and card detail list answered.
Specifically, warning information is stored in presetting database when generating warning information, for calling when subsequent query.
Warning information and account information, are set early warning type (if each item is preset if early warning rule has one's own early warning type)
The information associations storage such as standby, facilitates the inquiry of subsequent user.
User carries out corresponding warning information and transaction details table by inputting the instruction of early warning result queries in client
It checks.The display of transaction details table is that user understands specific transaction details for convenience, convenient for comparing.
Presetting database can be individually made of Hbase, Hive distributed database, can also be by Hbase, Hive equal distribution
Formula database and Solr component collectively constitute.Solr component is a high-performance, the full-text search technology based on Lucene, mainly
Realize value to key retrieval, supplement provides the index function of Hbase, Hive distributed database, improves search response
Speed.
The present embodiment has updated summarized results every time and will be compared with default early warning rule, judges whether there is early warning letter
Breath generates, and realizes the real-time of Risk-warning.And warning information is stored in the present count independently of large-scale data computing engines
According to response speed when in library, realizing the isolation of calculating and storage, while guaranteeing calculating and inquiring, whole system is improved
Overall treatment efficiency and operation stability.
Practical operation example is as follows:
The system architecture diagram of use is as shown in figure 4, real-time transaction data is turned over from raw data base and is written to by IBM CDC
Kafka accesses Spark, and Spark calls default early warning rule to be calculated from Risk-warning management module, by warning information
It stores with every real-time transaction data to the Hbase of presetting database and Solr；Enquiry module is looked into from presetting database
It askes.
When the card transaction details (i.e. real-time transaction data) that client occurs enters Kafka by turning over writing technology, by this
Flowing water adds up the result after summarizing in Spark and compares with default early warning rule.If meeting trigger condition, a note is generated
It records in Hbase triggering early warning rules results table, while whether triggering early warning, every transaction detail of inflow is stored in
Into card detail list, handled to subsequent verification.
Business personnel selects the information such as code, mechanism or card number of early warning type to carry out transaction details/pre- by client
Alert result queries, generate corresponding inquiry instruction, relevant query result are transferred from presetting database and is shown to business personnel.
It should be noted that above-described embodiment can be freely combined as needed.The above is only of the invention preferred
Embodiment, it is noted that for those skilled in the art, in the premise for not departing from the principle of the invention
Under, several improvements and modifications can also be made, these modifications and embellishments should also be considered as the scope of protection of the present invention.
Claims (10)
1. a kind of real-time method for prewarning risk of big data characterized by comprising
It is turned over from raw data base and writes real-time transaction data to large-scale data computing engines；
It carries out adding up to summarize according to account information by real-time transaction data of the large-scale data computing engines to access, in real time more
The summarized results of new corresponding account information；
The summarized results of update is analyzed respectively by large-scale data computing engines, obtains early warning analysis result.
2. the real-time method for prewarning risk of big data as described in claim 1, which is characterized in that described from raw data base
It turns over and writes real-time transaction data to large-scale data computing engines specifically:
It is turned over from raw data base and writes real-time transaction data to real-time Message Passing system；
The real-time transaction data is accessed in large-scale data computing engines by the real-time Message Passing system.
3. the real-time method for prewarning risk of big data as described in claim 1, which is characterized in that drawn by large-scale data calculating
It holds up and the summarized results of update is analyzed respectively, obtaining early warning analysis result includes:
The summarized results of update is compared with default early warning rule respectively by large-scale data computing engines, if meeting pre-
If the trigger condition of early warning rule, then the corresponding warning information of summarized results of the update is generated.
4. the real-time method for prewarning risk of big data as claimed in claim 3, which is characterized in that the default early warning rule includes
Below any one or more:
The accumulative generation amount limitation of single card odd-numbered day, the accumulative transaction stroke count limitation of single card odd-numbered day, high risk zone transaction limits.
5. the real-time method for prewarning risk of big data as claimed in claim 3, which is characterized in that further include: the early warning is believed
Breath is stored to presetting database.
6. the real-time method for prewarning risk of big data as described in claim 1, which is characterized in that further include: by counting on a large scale
It will be in every real-time transaction data deposit presetting database of access according to computing engines.
7. the real-time method for prewarning risk of big data as claimed in claim 6, which is characterized in that further include: it trades when receiving
When detail inquiry instruction, the transaction detail query is transferred from the presetting database and instructs corresponding transaction details table.
8. a kind of real-time Warning System of big data characterized by comprising
Data turn over writing module, write real-time transaction data to large-scale data computing engines for turning over from raw data base；
The large-scale data computing engines carry out adding up to summarize, in real time to the real-time transaction data of access according to account information
Update the summarized results of corresponding account information；And the summarized results of update is analyzed respectively, obtain early warning analysis knot
Fruit.
9. the real-time Warning System of big data as claimed in claim 8, which is characterized in that the large-scale data calculating is drawn
It holds up, the summarized results of update is analyzed respectively, obtaining early warning analysis result includes:
The large-scale data computing engines by the summarized results of update and are preset pre- respectively by large-scale data computing engines
Police regulations are then compared, if meeting the trigger condition of default early warning rule, the summarized results for generating the update is corresponding pre-
Alert information.
10. the real-time Warning System of big data as claimed in claim 8, which is characterized in that further include:
Presetting database, for storing every real-time transaction data；
Enquiry module, for it is bright to transfer the transaction from the presetting database when receiving transaction detail query instruction
The corresponding transaction details table of thin inquiry instruction.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811607509.7A CN109697567A (en) | 2018-12-27 | 2018-12-27 | A kind of real-time method for prewarning risk of big data and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811607509.7A CN109697567A (en) | 2018-12-27 | 2018-12-27 | A kind of real-time method for prewarning risk of big data and system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN109697567A true CN109697567A (en) | 2019-04-30 |
Family
ID=66232156
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811607509.7A Pending CN109697567A (en) | 2018-12-27 | 2018-12-27 | A kind of real-time method for prewarning risk of big data and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109697567A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110570097A (en) * | 2019-08-15 | 2019-12-13 | 中国平安人寿保险股份有限公司 | business personnel risk identification method and device based on big data and storage medium |
| CN111382150A (en) * | 2020-03-19 | 2020-07-07 | 交通银行股份有限公司 | Real-time computing method and system based on Flink |
| CN111930746A (en) * | 2020-07-31 | 2020-11-13 | 银盛支付服务股份有限公司 | Risk transaction control method, device and system based on offline data processing |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103714479A (en) * | 2012-10-09 | 2014-04-09 | 四川欧润特软件科技有限公司 | Intelligent centralized monitor method and system for bank personal business fraudulent conducts |
| CN104376397A (en) * | 2014-10-15 | 2015-02-25 | 中国农业银行股份有限公司 | Data real-time analysis method and device |
| CN104392381A (en) * | 2014-10-29 | 2015-03-04 | 中国建设银行股份有限公司 | Risk monitoring method of transaction data and system thereof |
| CN105205715A (en) * | 2015-09-24 | 2015-12-30 | 北京银丰新融科技开发有限公司 | On-line bank exception business monitoring system |
| CN106530074A (en) * | 2016-10-25 | 2017-03-22 | 宁波银行股份有限公司 | Large amount charge-off control method and system, and cash management information system |
| CN107833131A (en) * | 2017-10-31 | 2018-03-23 | 中国银行股份有限公司 | Inter-bank pays reconciliation method for early warning, device, computer equipment and storage medium |
-
2018
- 2018-12-27 CN CN201811607509.7A patent/CN109697567A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103714479A (en) * | 2012-10-09 | 2014-04-09 | 四川欧润特软件科技有限公司 | Intelligent centralized monitor method and system for bank personal business fraudulent conducts |
| CN104376397A (en) * | 2014-10-15 | 2015-02-25 | 中国农业银行股份有限公司 | Data real-time analysis method and device |
| CN104392381A (en) * | 2014-10-29 | 2015-03-04 | 中国建设银行股份有限公司 | Risk monitoring method of transaction data and system thereof |
| CN105205715A (en) * | 2015-09-24 | 2015-12-30 | 北京银丰新融科技开发有限公司 | On-line bank exception business monitoring system |
| CN106530074A (en) * | 2016-10-25 | 2017-03-22 | 宁波银行股份有限公司 | Large amount charge-off control method and system, and cash management information system |
| CN107833131A (en) * | 2017-10-31 | 2018-03-23 | 中国银行股份有限公司 | Inter-bank pays reconciliation method for early warning, device, computer equipment and storage medium |
Non-Patent Citations (3)
| Title |
|---|
| 刘丁发，葛学锋，邓春华等: "《Oracle数据库应用与开发实战 双色》", 31 August 2017 * |
| 医院信息系统数据库技术与应用》: "《医院信息系统数据库技术与应用》", 31 October 2015 * |
| 顾立平，马景源等: "《数据馆员的SPARK简明手册》", 31 October 2017 * |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110570097A (en) * | 2019-08-15 | 2019-12-13 | 中国平安人寿保险股份有限公司 | business personnel risk identification method and device based on big data and storage medium |
| CN111382150A (en) * | 2020-03-19 | 2020-07-07 | 交通银行股份有限公司 | Real-time computing method and system based on Flink |
| CN111382150B (en) * | 2020-03-19 | 2023-08-18 | 交通银行股份有限公司 | Real-time computing method and system based on Flink |
| CN111930746A (en) * | 2020-07-31 | 2020-11-13 | 银盛支付服务股份有限公司 | Risk transaction control method, device and system based on offline data processing |
| CN111930746B (en) * | 2020-07-31 | 2024-05-03 | 银盛支付服务股份有限公司 | Risk transaction management and control method, device and system based on offline data processing |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN104915450B (en) | A kind of big data storage and retrieval method and system based on HBase | |
| US10572296B2 (en) | System and method for a data processing architecture | |
| CN108595621B (en) | Early warning analysis method and system for false value-added tax invoice | |
| CN109697567A (en) | A kind of real-time method for prewarning risk of big data and system | |
| EP3039574A1 (en) | Queries involving multiple databases and execution engines | |
| CN110264328A (en) | Account checking method, device, equipment, system and computer readable storage medium | |
| CN104679646B (en) | A kind of method and apparatus for detecting SQL code defect | |
| CN109034988A (en) | A kind of accounting entry generation method and device | |
| CN102708525A (en) | Vacant position intelligent recommendation method based on GPU (graphics processing unit) acceleration | |
| CN112365355B (en) | Method, device and readable medium for calculating foundation valuation and risk index in real time | |
| CN103258034A (en) | Economic and financial behavior analysis system model based on social media | |
| CN110175919A (en) | Transaction data processing method, device, equipment and computer readable storage medium | |
| CN109325055A (en) | The screening of business association tables of data and checking method, device, electronic equipment | |
| CN112667612A (en) | Data quality checking method and device, electronic equipment and storage medium | |
| CN110457320A (en) | Real-time storage method and apparatus, storage medium and the computer equipment of data | |
| CN110266555A (en) | Method for analyzing web site service request | |
| CN110162521A (en) | A kind of payment system transaction data processing method and system | |
| CN107844874A (en) | Enterprise operation problem analysis system and its method | |
| CN113407587B (en) | Data processing method, device and equipment for online analysis processing engine | |
| Li | Research on the Social Security and Elderly Care System under the Background of Big Data | |
| CN111143328A (en) | Agile business intelligent data construction method, system, equipment and storage medium | |
| CN110659867A (en) | Comprehensive personal management service system for fund flow, travel management and financial management | |
| CN109934622A (en) | A kind of method and apparatus of processing electricity pin data | |
| TWM595272U (en) | Microfinance credit system | |
| CN107463576A (en) | Data processing method and equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20190430 |