CN108681590A - Incremental data processing method and processing device, computer equipment, computer storage media - Google Patents
Incremental data processing method and processing device, computer equipment, computer storage media Download PDFInfo
- Publication number
- CN108681590A CN108681590A CN201810462762.1A CN201810462762A CN108681590A CN 108681590 A CN108681590 A CN 108681590A CN 201810462762 A CN201810462762 A CN 201810462762A CN 108681590 A CN108681590 A CN 108681590A
- Authority
- CN
- China
- Prior art keywords
- data
- incremental
- incremental data
- transformation rule
- source database
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
This application provides a kind of incremental data processing method and processing device, computer equipment, computer storage medias, wherein this method includes：It is obtained and the relevant incremental data of target service system from source database；From each transformation rule built in advance, transformation rule corresponding with target service system is searched；According to the transformation rule found, the incremental data of acquisition is converted；Transformed incremental data is sent to target service system.Pass through incremental data processing method and processing device provided by the present application, computer equipment, computer storage media, transformation rule corresponding with target service system can be automatically determined according to the transformation rule built in advance, and the conversion of incremental data is carried out based on matched transformation rule, eliminate the isomerism between each operation system, and the requirement to developer is relatively low, it is time saving and energy saving.
Description
Technical field
This application involves computer processing technical fields, in particular to a kind of incremental data processing method and dress
It sets, computer equipment, computer storage media.
Background technology
With the deep application of Internet technology, internet data is either in data traffic, data processing amount or number
According to all becoming increasing in amount of storage.Wherein, internet system data are typically stored at relevant database (i.e. source data
Library) in, such as Mysql, Oracle and Sqlserver.In practical applications, it generally requires from above-mentioned relation type database
It is middle to extract corresponding data to meet the business demand of different business systems.
Common data pick-up method has full dose extraction and increment extraction, wherein full dose extraction refers to will be in data source
The data such as table or view are intactly extracted from database；And increment extraction refers to from being drawn from database
The data for having increased newly, changing and/or deleting since secondary extraction.In extraction-conversion-load (Extract- of database
Transform-Load, ETL) during, increment extraction is more extensive compared with the application that full dose extracts.
In the related art, when extracting corresponding incremental data from source database to meet the need of multiple operation systems
When asking, mainly operated for each operation system progress hard coded to realize by developer.However, it is contemplated that each industry
The Service Control Logic of business system is simultaneously different, and, to the more demanding of developer, exploitation is complex, time-consuming and laborious for this.
Invention content
In view of this, the embodiment of the present application is designed to provide a kind of incremental data processing method and processing device, computer
Equipment, computer storage media, to eliminate the isomerism between each operation system, the requirement to developer is relatively low, time saving province
Power.
In order to solve foregoing problems, the embodiment of the present application provides a kind of incremental data processing method, the method packet
It includes：
It is obtained and the relevant incremental data of target service system from source database；
From each transformation rule built in advance, transformation rule corresponding with the target service system is searched；
According to the transformation rule found, the incremental data of acquisition is converted；
Transformed incremental data is sent to the target service system.
Optionally, the transformation rule includes that the first field is converted to the second field, the conversion that the basis is found
Rule converts the incremental data of acquisition, including：
From the incremental data of acquisition, first field is searched, the first field found out is converted into the second word
Section, obtains transformed incremental data.
Optionally, the transformation rule includes that at least two third fields are carried out logical operation, and the basis is found
Transformation rule, the incremental data of acquisition is converted, including：
From the incremental data of acquisition, each third field and its corresponding field value are searched；
The field value found is subjected to logical operation according to the transformation rule, obtains transformed incremental data.
Optionally, the transformation rule includes conditional expression, and the conditional expression includes that condition and condition correspond to
As a result, the transformation rule that the basis is found, the incremental data of acquisition is converted, including：
Judge whether the incremental data obtained meets the condition, if so, using the corresponding result of the condition as described in
Transformed incremental data.
Optionally, described after source database acquisition with the relevant incremental data of target service system, the method
Further include：
The incremental data based on acquisition determines the major key for corresponding to the incremental data in the source database；
Based on determining major key, associated data is searched from the source database；Wherein, the associated data includes going through
History associated data and increment associated data.
Optionally, the transformation rule that the basis is found converts the incremental data of acquisition, including：
According to the transformation rule found, by the incremental data of acquisition, the historical context data and the increment
Associated data is converted respectively.
Optionally, described from source database acquisition and the relevant incremental data of target service system, including：
It monitors source database and whether there is data variation；
If so, parsing changed data by daily record analysis mode, the incremental data is obtained.
Optionally, the monitoring source database whether there is data variation, including：
Data are dumped to from source database in database Incremental Log resolution server；
Incremental Log resolution server in library judges that the source database whether there is data variation based on the data；
It is described that changed data are parsed by daily record analysis mode, including：
Judging the source database there are when data variation, by the database Incremental Log resolution server,
Changed data are parsed.
Optionally, the Incremental Log resolution server of library based on the data judges that the source database whether there is
Data variation, including：
Itself is modeled as backup database by the database Incremental Log resolution server；
According to the communication protocol between the backup database and the source database, pushed away to source database transmission
Instruction is sent to ask, so that described in data change event is back to according to push instruction request by the source database one by one
Backup database；
Message queue is generated based on the data change event that the source database returns；
The message queue is analyzed, determines that the source database whether there is data variation according to analysis result.
Optionally, described that the message queue is analyzed, determine whether the source database is deposited according to analysis result
In data variation, including：
In the message queue when the corresponding data of the additional data change event, obtain in the message queue
First offset of additional data recently；
The second offset is set for data currently additional in the message queue, second offset is equal to described
First offset adds 1；
Obtain the offset of the data last read from the message queue；
If the offset of the data last read is less than second offset, it is determined that the source data inventory
In data variation.
The embodiment of the present application also provides a kind of incremental data processing unit, described device includes：
Incremental data acquisition module, for being obtained and the relevant incremental data of target service system from source database；
Transformation rule searching module, for from each transformation rule built in advance, searching and the target service system
It unites corresponding transformation rule；
Incremental data conversion module, for according to the transformation rule found, the incremental data of acquisition to be turned
It changes；
Incremental data sending module, for transformed incremental data to be sent to the target service system.
The embodiment of the present application also provides a kind of computer equipments, including processor, memory and bus, wherein described
Memory is stored with the executable machine readable instructions of the processor, when the computer equipment is run, the processor
By the bus communication between the memory, when the machine readable instructions are executed by the processor, realize as follows
Step：
It is obtained and the relevant incremental data of target service system from source database；
From each transformation rule built in advance, transformation rule corresponding with the target service system is searched；
According to the transformation rule found, the incremental data of acquisition is converted；
Transformed incremental data is sent to the target service system.
The embodiment of the present application also provides a kind of computer storage media, the computer storage media is stored with calculating
Machine executable instruction when the computer executable instructions are executed by computer equipment, realizes above-mentioned incremental data processing side
Method.
Incremental data processing method provided by the embodiments of the present application obtains and target service system from source database first
Relevant incremental data；Secondly it from each transformation rule built in advance, searches corresponding with the target service system
Transformation rule；Then according to the transformation rule found, the incremental data of acquisition is converted；After finally converting
Incremental data be sent to the target service system.Pass through incremental data processing method provided by the embodiments of the present application and dress
It sets, computer equipment, computer storage media, can be automatically determined according to the transformation rule built in advance and target service system
Unite corresponding transformation rule, and carry out the conversion of incremental data based on matched transformation rule, eliminate each operation system it
Between isomerism, and the requirement to developer is relatively low, time saving and energy saving.
To keep the above objects, features, and advantages of the application more obvious and easy to understand, preferred embodiment cited below particularly, and match
Attached drawing is closed, is described in detail as follows.
Description of the drawings
It, below will be to needed in the embodiment in order to illustrate more clearly of the technical solution of the embodiment of the present application
Attached drawing is briefly described, it should be understood that the following drawings illustrates only some embodiments of the application, therefore is not construed as
Restriction to protection domain without creative efforts, may be used also for those of ordinary skill in the art
To obtain other relevant attached drawings according to these attached drawings.
Fig. 1 shows a kind of flow chart for incremental data processing method that the embodiment of the present application is provided；
Fig. 2 shows the flow charts for another incremental data processing method that the embodiment of the present application is provided；
Fig. 3 shows the flow chart for another incremental data processing method that the embodiment of the present application is provided；
Fig. 4 shows the flow chart for another incremental data processing method that the embodiment of the present application is provided；
Fig. 5 shows the flow chart for another incremental data processing method that the embodiment of the present application is provided；
Fig. 6 shows a kind of structural schematic diagram for incremental data processing unit that the embodiment of the present application is provided；
Fig. 7 shows incremental data acquisition module in a kind of incremental data processing unit that the embodiment of the present application is provided
Structural schematic diagram；
Fig. 8 shows a kind of structural schematic diagram for computer equipment that the embodiment of the present application is provided.
Specific implementation mode
To keep the purpose, technical scheme and advantage of the embodiment of the present application clearer, below in conjunction with attached drawing, to this Shen
Technical solution that please be in embodiment carries out clear, complete description.Obviously, described embodiment is only the one of the application
Section Example, instead of all the embodiments.In general, the group of the embodiment of the present application being described and illustrated herein in the accompanying drawings
Part can be arranged and be designed with a variety of different configurations.Therefore, below in conjunction with the detailed of the embodiment of the present application of attached drawing offer
Description be not intended to limit this application claims range, and be merely representative of the selected embodiment of the application.Based on this Shen
Embodiment please, the every other embodiment that those skilled in the art are obtained without making creative work,
Belong to this application claims range.
In the related art, it is mainly operated for each operation system progress hard coded by developer and is increased to realize
Measure data processing.However, it is contemplated that the Service Control Logic of each operation system and differing, therefore developer is wanted
Ask higher, exploitation is complex, time-consuming and laborious.Based on this, the embodiment of the present application provides a kind of incremental data processing method,
To eliminate the isomerism between each operation system, the requirement to developer is relatively low, time saving and energy saving, embodiment as described below.
It is the flow chart of incremental data processing method provided by the embodiments of the present application, incremental data processing referring to Fig. 1
The executive agent of method can be computer equipment, and the above method includes the following steps：
S101, it is obtained and the relevant incremental data of target service system from source database.
Here, source database can refer to that can directly provide firsthand information or the self-sufficiency database of specific data, user
Other information source need not be consulted again.Wherein, each operation system can correspond to a source database, can also be multiple business
System shares a source database, and the embodiment of the present application does not do this specific limitation.
Wherein, the embodiment of the present application can be captured and be pushed incremental number by daily record analysis program, daily record monitoring programme etc.
According to then by stream process program reception incremental data, to obtain and the relevant incremental data of target service system.
It is worth noting that the operation system in the embodiment of the present application can be personal loan platform, corporation information query
The Implementing Educational Information Platform etc. of system, Management System on Public Rooms, macroeconomic data platform, storage educational information and information, the application
Embodiment does not also do this specific limitation.
S102, from each transformation rule built in advance, search corresponding with target service system transformation rule.
Here, the embodiment of the present application can be previously stored with the mapping relations between transformation rule and operation system, such as：
Mapping table or other forms map information.The mapping table or other map informations reflect at least one transformation rule at least
Correspondence between one operation system, that is, each operation system is corresponding with corresponding one or more conversion rule
Then.In this way, computer equipment after getting incremental data, can be deposited by inquiring the computer equipment or other equipment
The mapping relations of storage obtain the transformation rule corresponded to corresponding to above-mentioned target service system.Wherein, it is reflected for the ease of realizing
Relationship is penetrated, the embodiment of the present application can be established based on the service identification information of operation system between operation system and transformation rule
Relationship.
Wherein, the embodiment of the present application can form modulus of conversion by each transfer process of specified grammer configuration data
Then formula compiles above-mentioned translative mode by compiler, to form executable resolver, that is, having obtained advance structure
Each transformation rule.
The transformation rule that S103, basis are found, the incremental data of acquisition is converted.
It here, can be according to the transformation rule to obtaining after finding the transformation rule corresponding to target service system
Incremental data converted.
Wherein, incremental data can be input to resolver by the embodiment of the present application, by resolver a step by a step into line number
According to conversion.
S104, transformed incremental data is sent to target service system.
Here, for transformed incremental data, which can be sent to target service system.Wherein, turn
Incremental data after changing disclosure satisfy that target service systematic difference demand.In addition, after being sent to the conversion of target service system
Incremental data can also be written in knowledge mapping, and cover existing legacy data, after write-in, knowledge mapping
Relevant interface and visual page synchronized update, displaying are newest as a result, improving at incremental data provided by the embodiments of the present application
The applicability of reason method.
For different operation systems, corresponding transformation rule may and differ.In the embodiment of the present application, it is based on
The transformation rule found can convert the incremental data of acquisition, and above-mentioned transfer process includes but not limited to as next
Kind or a variety of conversion regimes：Replace conversion, logical operation conversion, condition conversion.Next point three aspects carry out above-mentioned conversion
The detailed description of mode.
In a first aspect, for conversion regime is replaced, the transformation rule in the embodiment of the present application is converted to including the first field
Second field matches the incremental data lookup based on acquisition with the first field in this way, after getting incremental data
The first field of target, and first field of target is converted into corresponding second field, using transformed second field as turning
Incremental data after changing.
For example, the json formats of the incremental data obtained are " { " phoneNumber ":" 131****3463 " ", it searches
The transformation rule arrived is " { " mobile ":" $ phoneNumber " ", after data conversion, exporting new json formats is
“{“mobile”:" 131****3463 " } ", wherein i.e. corresponding first fields of phoneNumber, i.e. corresponding second words of mobile
Section, according to the transformational relation between the second field mobile and the first field phoneNumber, by the first field
PhoneNumber is converted to the second field mobile, and conversion is replaced to realize.
Second aspect, for logical operation conversion regime, the transformation rule in the embodiment of the present application includes by least two
Third field carries out logical operation, in this way, after getting incremental data, the incremental data based on acquisition is searched each the
Three fields and the corresponding field value of each third field, and the field value found is subjected to logical operation, to be converted
Incremental data afterwards.
Wherein, above-mentioned logical operation includes but not limited to summation operation, seeks difference operation, quadrature operation, division arithmetic, asks
Maximum value transport, operation of minimizing.
The third aspect, for condition conversion regime, the transformation rule in the embodiment of the present application includes conditional expression, item
Part expression formula includes that condition and condition are corresponding as a result, in this way, after getting incremental data, will determine that the increment of acquisition
Whether data meet condition, if so, using the corresponding result of condition as transformed incremental data.
For example, the json formats of the incremental data obtained are " { " systemSourceName "：" YRD " ", it finds
Transformation rule is
Wherein, [[" $ systemSourceName ", "==", " YRD "], " | | ", [" $
IsYrdOfflineProduct ", "==", true]] i.e. corresponding to the condition of loanProductLine, " first man is borrowed
Second people's loan platform of platform " with " is borrowed " to correspond to two of above-mentioned condition respectively as a result, according to transformation rule, will meet
{“systemSourceName”:" YRD " this condition result " first man loan platform " conduct
LoanProductLine's as a result, so that realize condition conversion.
In view of the regular query source client database of polling routine that generally use outside starts in the related technology is in poll week
Change data in phase, to obtain corresponding incremental data, such as SQL polls.Wherein, the change data in above-mentioned polling cycle
Refer to the incremental data inquired at interval of preset time, and in order to ensure not missing data, it will usually take a period of time number more
According to.For example program is primary every operation in 10 minutes, can take nearest 20 minutes data every time, i.e., it is to have taken 10 minute datas more.
However, the incremental data method of above-mentioned acquisition is due to the data structure that is limited in source database, indefinite
When some ad hoc inquiry field (such as update_time fields), newer data can not be obtained, and ask in the presence of what is computed repeatedly
Topic, it is less efficient.To solve the above-mentioned problems, the embodiment of the present application also provides one kind is increased based on daily record analysis mode
The method for measuring data acquisition, as shown in Fig. 2, the process of above-mentioned acquisition incremental data is realized especially by following steps：
S201, monitoring source database whether there is data variation；
S202, if so, by daily record analysis mode parse changed data, obtain incremental data.
Here, the embodiment of the present application can be based on daily record monitoring programme and monitor source database, and source data one have variation,
Its variation daily record just can be captured by daily record monitoring programme, and the daily record after capture is by parsing and cleaning treatment, by treated
Data-pushing is to intermediate storage；In this way, the stream process program to be run without interruption again by 7*24 hours, (compares at regular intervals
Such as 1 second) check an intermediate storage.If intermediate storage has new data, the reading of incremental data is carried out by stream process program
It takes.In addition, stream process program can also record the rearmost position that every batch of reads data, next batch is since this position
Continue to read, to ensure not repeat to read.
For the ease of parsing changed data, the embodiment of the present application provides a kind of utilization database Incremental Log
The method that resolution server carries out daily record parsing, as shown in figure 3, above-mentioned resolving specifically comprises the following steps：
S301, data are dumped to from source database in database Incremental Log resolution server；
S302, judge that source database whether there is data variation based on database Incremental Log resolution server；
S303, judging source database there are when data variation, it is right by database Incremental Log resolution server
Changed data are parsed.
Here, the embodiment of the present application can judge whether source database is deposited based on database Incremental Log resolution server
In data variation, if so, directly being parsed to changed data.Wherein, database Incremental Log resolution server
Operation is above-mentioned daily record parsing operating method.
Wherein, as shown in figure 4, above-mentioned judge that source database whether there is based on database Incremental Log resolution server
Data variation can specifically be implemented by the following steps：
Itself is modeled as backup database by S401, database Incremental Log resolution server；
S402, according to the communication protocol between backup database and source database, send push instruction to source database and ask
It asks, so that data change event is back to backup database by source database one by one according to push instruction request；
S403, message queue is generated based on the data change event that source database returns；
S404, message queue is analyzed, determines that source database whether there is data variation according to analysis result.
Here, the database Incremental Log resolution server in the embodiment of the present application, itself will be modeled as backup number first
According to library, secondly by the communication protocol between backup database and source database, push instruction request is sent to source database,
So that data change event is back to backup database by source database one by one according to push instruction request, it is then based on return
Data change event generate message queue, finally the message queue of generation is analyzed, to determine whether source database is deposited
In data variation.
Wherein, as shown in figure 5, above-mentioned S404 can specifically be implemented by the following steps：
S501, it when the corresponding data of supplemental data change events, obtains and is added recently in message queue in message queue
Data the first offset；
S502, the second offset is set for data currently additional in message queue, the second offset is equal to the first offset
Amount plus 1；
The offset for the data that S503, acquisition are last read from message queue；
If the offset of S504, the data last read is less than the second offset, it is determined that there are data changes for source database
Change.
Here, incremental data processing method supplemental data change events in message queue provided by the embodiments of the present application
When corresponding data, obtain the first offset of data additional recently in message queue first, secondly in message queue when
The second offset is arranged in the data of preceding addition, be then based on the offsets of the data last read from message queue with it is above-mentioned
Magnitude relationship between second offset, to determine source database with the presence or absence of variation.
Incremental data processing method provided by the embodiments of the present application is also based on the increasing after getting incremental data
It measures the corresponding major key of data and searches corresponding associated data.Specifically, in the embodiment of the present application, the incremental data based on acquisition,
The major key for corresponding to the incremental data in source database is determined first；It is then based on determining major key, is searched from source database
Associated data；Wherein, associated data includes historical context data and increment associated data.
Wherein, for every incremental data, when being stored in source database, it will be corresponding with corresponding major key,
In this way, can be based on the major key searches corresponding historical context data and increment associated data.In addition, the embodiment of the present application
Not only can to incremental data carry out data conversion, additionally it is possible to included by associated data historical context data and increment close
Connection data are converted respectively, and specific transformation rule is similar to the transformation rule of above-mentioned incremental data, and details are not described herein.
Incremental data processing method provided by the embodiments of the present application obtains and target service system from source database first
Relevant incremental data；Secondly from each transformation rule built in advance, conversion corresponding with target service system is searched
Rule；Then according to the transformation rule found, the incremental data of acquisition is converted；Finally by transformed incremental number
According to being sent to target service system.This method can automatically determine and target service system according to the transformation rule built in advance
Corresponding transformation rule, and based on matched transformation rule carry out incremental data conversion, to eliminate each operation system it
Between isomerism, and the requirement to developer is relatively low, time saving and energy saving.
It is next special to lift one for the ease of further understanding incremental data processing method provided by the embodiments of the present application
Finance debt-credit scene is illustrated as sample.
(1) incremental data obtains the stage：It is captured and is pushed by daily record analysis software, daily record monitoring software etc. first and increased
Measure data, secondly by stream process software receive incremental data, then to incremental data duplicate removal, be normalized into part major key, finally lead to
Cross into part major key find it is all with into other relevant increments of part and historical data.
Above-mentioned " increment and historical data " includes borrowing preceding application information and performance information of refunding after borrowing.Application before wherein borrowing
Information includes the information such as personal identity card, address, phone, contact person, qq, Taobao's account, and refund performance information includes also after loan
The information such as money state, refund issue, repayment amount.
Above-mentioned " being normalized into part major key " can be：Portion may include completely bank card data, connection into number of packages evidence
It is personal data, address date etc..If a certain moment, certain portion only has updated address date into number of packages evidence, then passing through address
Data find the major key (such as into part application numbers) into part, and this completes major key mappings.
(2) the data conversion stage：Data carry out renaming, logical operation, condition judgment etc. after being borrowed before borrowing, final to turn
Be changed to one comprising borrow before, borrow after various details complete json.Specifically, for a new product, number is defined first
According to the pattern of conversion, data after borrowing then are inputted before the loan obtained on last stage, and then carrying out parsing to the data of acquisition turns
It changes, generates standard, complete, writable knowledge mapping complete json data.Transfer process therein includes but unlimited
In：Renaming has field, increases constant field, calls custom function calculating, IF ELSE condition judgments, single layer json
Nesting is multilayer, multilayer json tilings are one layer etc..
(3) phase data output：By the previous step data that have merged of conversion by knowledge mapping bottom storage organization into traveling
One step is converted, and knowledge mapping is written in transformed incremental data, to realize the visual presentation of knowledge mapping.Wherein, in knowledge
It can be shown in collection of illustrative plates into the details of part and into the relationship between part, and can then pass through electricity into the relationship between part
The information such as words, address are attached.
Based on same inventive concept, the embodiment of the present application also provides incremental numbers corresponding with incremental data processing method
According to processing unit, since the principle that incremental data processing unit provided by the embodiments of the present application solves the problems, such as is implemented with the application
The incremental data processing method that example provides is similar, therefore the implementation of device may refer to the implementation of method, and it is no longer superfluous to repeat place
It states.As shown in fig. 6, the structural schematic diagram of the incremental data processing unit provided by the embodiment of the present application, at the incremental data
Managing device includes：
Incremental data acquisition module 601, for being obtained and the relevant incremental data of target service system from source database；
Transformation rule searching module 602, for from each transformation rule built in advance, searching and target service system
It unites corresponding transformation rule；
Incremental data conversion module 603, for according to the transformation rule found, the incremental data of acquisition to be turned
It changes；
Incremental data sending module 604, for transformed incremental data to be sent to target service system.
In one embodiment, transformation rule includes that the first field is converted to the second field, and above-mentioned incremental data turns
Block 603 is changed the mold, is specifically used for from the incremental data of acquisition, the first field is searched, the first field found out is converted to the
Two fields obtain transformed incremental data.
In another embodiment, transformation rule includes that at least two third fields are carried out logical operation, above-mentioned increasing
Data conversion module 603 is measured, is specifically used for from the incremental data of acquisition, searches each third field and its corresponding field
Value；The field value found is subjected to logical operation according to transformation rule, obtains transformed incremental data.
In another embodiment, transformation rule includes conditional expression, and conditional expression includes condition and condition
It is corresponding as a result, above-mentioned incremental data conversion module 603, specifically for judging whether the incremental data obtained meets condition, if
It is, using the corresponding result of condition as transformed incremental data.
In specific implementation, above-mentioned incremental data processing unit further includes：
Associated data searching module 605 is used for the incremental data based on acquisition, determines and corresponds to the increasing in source database
Measure the major key of data；Based on determining major key, associated data is searched from source database；Wherein, associated data includes that history is closed
Join data and increment associated data.
Above-mentioned incremental data conversion module 603, specifically for the transformation rule that basis is found, by the incremental number of acquisition
It is converted respectively according to, historical context data and increment associated data.
In specific implementation, as shown in fig. 7, above-mentioned incremental data acquisition module 601 includes：
Monitoring unit 701 whether there is data variation for monitoring source database；
Incremental data acquiring unit 702 is used to, if so, by daily record analysis mode parsing changed data, obtain
To incremental data.
In another embodiment, above-mentioned monitoring unit 701, specifically for data are dumped to number from source database
According in the Incremental Log resolution server of library；Judge source database with the presence or absence of number based on database Incremental Log resolution server
According to variation；
Above-mentioned incremental data acquiring unit 702, specifically for judging that source database there are when data variation, passes through
Database Incremental Log resolution server, parses changed data.
In another embodiment, above-mentioned monitoring unit 701 is specifically used for：
Itself is modeled as backup database by database Incremental Log resolution server；
According to the communication protocol between backup database and source database, push instruction request is sent to source database, with
Make source database that data change event are back to backup database one by one according to push instruction request；
Message queue is generated based on the data change event that source database returns；
Message queue is analyzed, determines that source database whether there is data variation according to analysis result.
In another embodiment, above-mentioned monitoring unit 701 is specifically used for：
In message queue when the corresponding data of supplemental data change events, number additional recently in message queue is obtained
According to the first offset；
The second offset is set for data currently additional in message queue, the second offset adds equal to the first offset
1；
Obtain the offset of the data last read from message queue；
If the offset of the data last read is less than the second offset, it is determined that there are data variations for source database.
Incremental data processing method and processing device provided by the embodiments of the present application can also bring following technique effect：
(1) real-time that incremental data obtains is promoted, synchronization delay foreshortens to second grade by minute grade.The embodiment of the present application
Used synchronization program runs without interruption always.Source database is once update, and active push to synchronization program carries out same at once
Step has the difference of essence with the passive mode periodically inquired such as sql polls, faster, more in real time.
(2) simplify the development time of new business data conversion fusion, workload was reduced to for 1 man day by about 10 man days.By
The mode that relevant data conversion fusion process is improved to configuration file in the embodiment of the present application from the mode of hard coded is real
It is existing, increase a configuration in new business data, you can operation, almost without code development quantity, the development time is short, development efficiency
It is high.
The embodiment of the present application also provides a kind of computer equipments, as shown in figure 8, the computer equipment includes processor
801, memory 802 and bus 803, wherein memory 802 is stored with the executable machine readable instructions of processor 801, when
When computer equipment is run, communicated by bus 803 between processor 801 and memory 802, machine readable instructions are handled
When device 801 executes, following steps are realized：
It is obtained and the relevant incremental data of target service system from source database；
From each transformation rule built in advance, transformation rule corresponding with target service system is searched；
According to the transformation rule found, the incremental data of acquisition is converted；
Transformed incremental data is sent to target service system.
In one embodiment, transformation rule includes that the first field is converted to the second field, above-mentioned processor 801
In the step of realization, according to the transformation rule found, the incremental data of acquisition is converted, including：
From the incremental data of acquisition, the first field is searched, the first field found out is converted into the second field, is obtained
To transformed incremental data.
In another embodiment, transformation rule includes that at least two third fields are carried out logical operation, above-mentioned place
It manages in the step of device 801 is realized, according to the transformation rule found, the incremental data of acquisition is converted, including：
From the incremental data of acquisition, each third field and its corresponding field value are searched；The field value that will be found
Logical operation is carried out according to transformation rule, obtains transformed incremental data.
In another embodiment, transformation rule includes conditional expression, and conditional expression includes condition and condition
It is corresponding as a result, in the step of above-mentioned processor 801 is realized, according to the transformation rule found, by the incremental data of acquisition into
Row conversion, including：
Judge whether the incremental data obtained meets condition, if so, using the corresponding result of condition as transformed increment
Data.
In the step of above-mentioned processor 801 is realized, obtained and the relevant incremental number of target service system from source database
According to later, further include：
Incremental data based on acquisition determines the major key for corresponding to the incremental data in source database；
Based on determining major key, associated data is searched from source database；Wherein, associated data includes historical context number
According to increment associated data.
In the step of above-mentioned processor 801 is realized, according to the transformation rule found, the incremental data of acquisition is carried out
Conversion, including：
According to the transformation rule found, the incremental data of acquisition, historical context data and increment associated data are distinguished
It is converted.
In the step of above-mentioned processor 801 is realized, obtained and the relevant incremental number of target service system from source database
According to, including：
It monitors source database and whether there is data variation；
If so, parsing changed data by daily record analysis mode, incremental data is obtained.
In another embodiment, above-mentioned processor 801 realize the step of in, monitoring source database with the presence or absence of number
According to variation, including：Data are dumped to from source database in database Incremental Log resolution server；Based on database increment
Daily record resolution server judges that source database whether there is data variation；
Changed data are parsed by daily record analysis mode, including：Judging source database, there are data variations
When, by database Incremental Log resolution server, changed data are parsed.
In another embodiment, above-mentioned processor 801 realize the step of in, based on database Incremental Log parse
Server judges that source database whether there is data variation, including：
Itself is modeled as backup database by database Incremental Log resolution server；
According to the communication protocol between backup database and source database, push instruction request is sent to source database, with
Make source database that data change event are back to backup database one by one according to push instruction request；
Message queue is generated based on the data change event that source database returns；
Message queue is analyzed, determines that source database whether there is data variation according to analysis result.
In another embodiment, above-mentioned processor 801 realize the step of in, message queue is analyzed, according to
Analysis result determines that source database whether there is data variation, including：
In message queue when the corresponding data of supplemental data change events, number additional recently in message queue is obtained
According to the first offset；
The second offset is set for data currently additional in message queue, the second offset adds equal to the first offset
1；
Obtain the offset of the data last read from message queue；
If the offset of the data last read is less than the second offset, it is determined that there are data variations for source database.
The embodiment of the present application also provides a kind of computer storage media, which is stored with computer
Executable instruction when the computer executable instructions are executed by computer equipment, realizes above-mentioned incremental data processing method.
Specifically, which can be universal storage medium, such as mobile disk, hard disk, the computer
When computer program on storage medium is run, above-mentioned incremental data processing method can be realized, to solve to be directed at present
Each operation system carries out complex, the time-consuming and laborious problem of the existing exploitation of hard coded operation respectively, and then it is each to reach elimination
Isomerism between operation system, the effect that the requirement to developer is relatively low, time saving and energy saving.
The computer program product for the incremental data processing method that the embodiment of the present application is provided, including store program
The computer readable storage medium of code, the instruction that program code includes can be used for executing the method in preceding method embodiment,
Specific implementation can be found in embodiment of the method, and details are not described herein.
It is apparent to those skilled in the art that for convenience and simplicity of description, above system and device
Specific work process, can refer to corresponding processes in the foregoing method embodiment, details are not described herein.
If function is realized in the form of SFU software functional unit and when sold or used as an independent product, Ke Yicun
Storage is in a computer read/write memory medium.Based on this understanding, the technical solution of the application is substantially right in other words
The part of part or the technical solution that the prior art contributes can be expressed in the form of software products, the meter
Calculation machine software product is stored in a storage medium, including some instructions are used so that a computer equipment (can be
People's computer, server or network equipment etc.) execute each embodiment of the application method all or part of step.
And aforementioned storage medium includes：USB flash disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory
The various media that can store program code such as device (RAM, Random Access Memory), magnetic disc or CD.
It these are only the specific implementation mode of the application, but the protection domain of the application is not limited thereto, it is any to be familiar with
In the technical scope that the application discloses, the change or replacement that can be readily occurred in should all be covered those skilled in the art
Within the protection domain of the application.Therefore, the protection domain of the application should be with the protection domain that claims are recorded
It is accurate.
Claims (13)
1. a kind of incremental data processing method, which is characterized in that the method includes：
It is obtained and the relevant incremental data of target service system from source database；
From each transformation rule built in advance, transformation rule corresponding with the target service system is searched；
According to the transformation rule found, the incremental data of acquisition is converted；
Transformed incremental data is sent to the target service system.
2. according to the method described in claim 1, it is characterized in that, the transformation rule includes that the first field is converted to second
Field, the transformation rule that the basis is found convert the incremental data of acquisition, including：
From the incremental data of acquisition, first field is searched, the first field found out is converted into the second field, is obtained
Transformed incremental data.
3. according to the method described in claim 1, it is characterized in that, the transformation rule include by least two third fields into
Row logical operation, the transformation rule that the basis is found convert the incremental data of acquisition, including：
From the incremental data of acquisition, each third field and its corresponding field value are searched；
The field value found is subjected to logical operation according to the transformation rule, obtains transformed incremental data.
4. according to the method described in claim 1, it is characterized in that, the transformation rule includes conditional expression, the condition
Expression formula includes that condition and condition are corresponding as a result, the transformation rule that the basis is found, by the incremental number of acquisition
According to being converted, including：
Judge whether the incremental data obtained meets the condition, if so, using the corresponding result of the condition as the conversion
Incremental data afterwards.
5. according to the method described in claim 1, it is characterized in that, being obtained and target service system phase from source database described
After the incremental data of pass, the method further includes：
The incremental data based on acquisition determines the major key for corresponding to the incremental data in the source database；
Based on determining major key, associated data is searched from the source database；Wherein, the associated data includes historical context
Data and increment associated data.
6. according to the method described in claim 5, it is characterized in that, the transformation rule that the basis is found, by the institute of acquisition
Incremental data is stated to be converted, including：
According to the transformation rule found, the incremental data of acquisition, the historical context data are associated with the increment
Data are converted respectively.
7. according to the method described in claim 1, it is characterized in that, described related to target service system from source database acquisition
Incremental data, including：
It monitors source database and whether there is data variation；
If so, parsing changed data by daily record analysis mode, the incremental data is obtained.
8. the method according to the description of claim 7 is characterized in that the monitoring source database whether there is data variation, packet
It includes：
Data are dumped to from source database in database Incremental Log resolution server；
Incremental Log resolution server in library judges that the source database whether there is data variation based on the data；
It is described that changed data are parsed by daily record analysis mode, including：
The source database is being judged there are when data variation, by the database Incremental Log resolution server, to hair
The data for changing are parsed.
9. according to the method described in claim 8, it is characterized in that, the Incremental Log resolution server of library based on the data
Judge that the source database whether there is data variation, including：
Itself is modeled as backup database by the database Incremental Log resolution server；
According to the communication protocol between the backup database and the source database, push instruction is sent to the source database
Request, so that data change event is back to the Backup Data by the source database one by one according to push instruction request
Library；
Message queue is generated based on the data change event that the source database returns；
The message queue is analyzed, determines that the source database whether there is data variation according to analysis result.
10. according to the method described in claim 9, it is characterized in that, described analyze the message queue, according to analysis
As a result determine that the source database whether there is data variation, including：
In the message queue when the corresponding data of the additional data change event, obtains and chased after recently in the message queue
First offset of the data added；
The second offset is set for data currently additional in the message queue, second offset is equal to described first partially
Shifting amount adds 1；
Obtain the offset of the data last read from the message queue；
If the offset of the data last read is less than second offset, it is determined that there are data for the source database
Variation.
11. a kind of incremental data processing unit, which is characterized in that described device includes：
Incremental data acquisition module, for being obtained and the relevant incremental data of target service system from source database；
Transformation rule searching module, for from each transformation rule built in advance, searching and the target service system phase
Corresponding transformation rule；
Incremental data conversion module, for according to the transformation rule found, the incremental data of acquisition to be converted；
Incremental data sending module, for transformed incremental data to be sent to the target service system.
12. a kind of computer equipment, which is characterized in that including processor, memory and bus, wherein the memory storage
The machine readable instructions for having the processor executable, when the computer equipment is run, the processor and the storage
By the bus communication between device, when the machine readable instructions are executed by the processor, following steps are realized：
It is obtained and the relevant incremental data of target service system from source database；
From each transformation rule built in advance, transformation rule corresponding with the target service system is searched；
According to the transformation rule found, the incremental data of acquisition is converted；
Transformed incremental data is sent to the target service system.
13. a kind of computer storage media, the computer storage media is stored with computer executable instructions, the computer
When executable instruction is executed by computer equipment, the incremental data processing method described in any one of claims 1 to 10 is realized.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810462762.1A CN108681590A (en) | 2018-05-15 | 2018-05-15 | Incremental data processing method and processing device, computer equipment, computer storage media |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810462762.1A CN108681590A (en) | 2018-05-15 | 2018-05-15 | Incremental data processing method and processing device, computer equipment, computer storage media |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN108681590A true CN108681590A (en) | 2018-10-19 |
Family
ID=63805364
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810462762.1A Pending CN108681590A (en) | 2018-05-15 | 2018-05-15 | Incremental data processing method and processing device, computer equipment, computer storage media |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108681590A (en) |
Cited By (20)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109634937A (en) * | 2018-12-20 | 2019-04-16 | 成都四方伟业软件股份有限公司 | Incremental data acquisition method, apparatus and system |
| CN109857524A (en) * | 2019-01-25 | 2019-06-07 | 深圳前海微众银行股份有限公司 | Streaming computing method, apparatus, equipment and computer readable storage medium |
| CN110413953A (en) * | 2019-07-30 | 2019-11-05 | 深圳市智物联网络有限公司 | Amount of material determines method, system, device and computer readable storage medium |
| CN110543478A (en) * | 2019-07-17 | 2019-12-06 | 阿里巴巴集团控股有限公司 | Public layer width table construction method and device and server |
| CN110781243A (en) * | 2019-11-06 | 2020-02-11 | 杭州安恒信息技术股份有限公司 | Incremental synchronization method and system for double main data of relational database |
| CN111061787A (en) * | 2019-11-21 | 2020-04-24 | 新浪网技术(中国)有限公司 | Method and system for processing object relation in relational database |
| CN111241182A (en) * | 2020-01-19 | 2020-06-05 | 北京奇艺世纪科技有限公司 | Data processing method and apparatus, storage medium, and electronic apparatus |
| CN111342933A (en) * | 2020-02-25 | 2020-06-26 | 卓望数码技术（深圳）有限公司 | Data transmission method, device and medium |
| CN111506556A (en) * | 2020-04-09 | 2020-08-07 | 北京市测绘设计研究院 | Multi-source heterogeneous structured data synchronization method |
| CN111639138A (en) * | 2020-06-03 | 2020-09-08 | 中国联合网络通信集团有限公司 | Data processing method, device, equipment and storage medium |
| CN111813845A (en) * | 2020-06-29 | 2020-10-23 | 平安国际智慧城市科技股份有限公司 | ETL task-based incremental data extraction method, device, equipment and medium |
| CN111897808A (en) * | 2020-07-15 | 2020-11-06 | 苏宁金融科技(南京)有限公司 | Data processing method and device, computer equipment and storage medium |
| CN112434811A (en) * | 2019-08-26 | 2021-03-02 | 华为技术有限公司 | Knowledge graph construction method and device, computing equipment and storage medium |
| CN112632191A (en) * | 2020-12-29 | 2021-04-09 | 中国农业银行股份有限公司 | Data processing method and system |
| CN112882974A (en) * | 2021-02-09 | 2021-06-01 | 深圳市云网万店科技有限公司 | JSON data conversion method and device, computer equipment and storage medium |
| CN113239057A (en) * | 2021-05-24 | 2021-08-10 | 量子数聚（北京）科技有限公司 | Enterprise data updating method and device |
| CN114124878A (en) * | 2022-01-27 | 2022-03-01 | 深圳华锐金融技术股份有限公司 | Market data issuing method, device, system, equipment and medium |
| CN114416703A (en) * | 2020-10-28 | 2022-04-29 | 北京中祥英科技有限公司 | Method, device, equipment and medium for automatically monitoring data integrity |
| CN115600560A (en) * | 2022-09-28 | 2023-01-13 | 中电金信软件有限公司（Cn） | Data conversion method, device, system, electronic equipment and storage medium |
| CN116431688A (en) * | 2022-11-14 | 2023-07-14 | 北京远舢智能科技有限公司 | Data processing method and device, electronic equipment and storage medium |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102841897A (en) * | 2011-06-23 | 2012-12-26 | 阿里巴巴集团控股有限公司 | Incremental data extracting method, device and system |
| CN104506496A (en) * | 2014-12-10 | 2015-04-08 | 山大地纬软件股份有限公司 | Quasi-real-time data increment distribution middleware based on Oracle Streams technology and method |
| CN105069142A (en) * | 2015-08-18 | 2015-11-18 | 山大地纬软件股份有限公司 | System and method for extraction, transformation and distribution of data increments |
| CN105488187A (en) * | 2015-12-02 | 2016-04-13 | 北京四达时代软件技术股份有限公司 | Method and device for extracting multi-source heterogeneous data increment |
| CN107180116A (en) * | 2017-06-28 | 2017-09-19 | 努比亚技术有限公司 | A kind of data synchronizing processing method, mobile terminal and computer-readable recording medium |
| US20170337314A1 (en) * | 2016-05-20 | 2017-11-23 | International Business Machines Corporation | Enabling an incremental sign-off process using design data |
-
2018
- 2018-05-15 CN CN201810462762.1A patent/CN108681590A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102841897A (en) * | 2011-06-23 | 2012-12-26 | 阿里巴巴集团控股有限公司 | Incremental data extracting method, device and system |
| CN104506496A (en) * | 2014-12-10 | 2015-04-08 | 山大地纬软件股份有限公司 | Quasi-real-time data increment distribution middleware based on Oracle Streams technology and method |
| CN105069142A (en) * | 2015-08-18 | 2015-11-18 | 山大地纬软件股份有限公司 | System and method for extraction, transformation and distribution of data increments |
| CN105488187A (en) * | 2015-12-02 | 2016-04-13 | 北京四达时代软件技术股份有限公司 | Method and device for extracting multi-source heterogeneous data increment |
| US20170337314A1 (en) * | 2016-05-20 | 2017-11-23 | International Business Machines Corporation | Enabling an incremental sign-off process using design data |
| CN107180116A (en) * | 2017-06-28 | 2017-09-19 | 努比亚技术有限公司 | A kind of data synchronizing processing method, mobile terminal and computer-readable recording medium |
Cited By (29)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109634937A (en) * | 2018-12-20 | 2019-04-16 | 成都四方伟业软件股份有限公司 | Incremental data acquisition method, apparatus and system |
| CN109857524A (en) * | 2019-01-25 | 2019-06-07 | 深圳前海微众银行股份有限公司 | Streaming computing method, apparatus, equipment and computer readable storage medium |
| CN109857524B (en) * | 2019-01-25 | 2024-02-27 | 深圳前海微众银行股份有限公司 | Stream computing method, device, equipment and computer readable storage medium |
| CN110543478A (en) * | 2019-07-17 | 2019-12-06 | 阿里巴巴集团控股有限公司 | Public layer width table construction method and device and server |
| CN110543478B (en) * | 2019-07-17 | 2023-02-03 | 创新先进技术有限公司 | Public layer width table construction method and device and server |
| CN110413953A (en) * | 2019-07-30 | 2019-11-05 | 深圳市智物联网络有限公司 | Amount of material determines method, system, device and computer readable storage medium |
| CN112434811A (en) * | 2019-08-26 | 2021-03-02 | 华为技术有限公司 | Knowledge graph construction method and device, computing equipment and storage medium |
| CN110781243A (en) * | 2019-11-06 | 2020-02-11 | 杭州安恒信息技术股份有限公司 | Incremental synchronization method and system for double main data of relational database |
| CN111061787A (en) * | 2019-11-21 | 2020-04-24 | 新浪网技术(中国)有限公司 | Method and system for processing object relation in relational database |
| CN111241182A (en) * | 2020-01-19 | 2020-06-05 | 北京奇艺世纪科技有限公司 | Data processing method and apparatus, storage medium, and electronic apparatus |
| CN111342933A (en) * | 2020-02-25 | 2020-06-26 | 卓望数码技术（深圳）有限公司 | Data transmission method, device and medium |
| CN111342933B (en) * | 2020-02-25 | 2022-06-07 | 卓望数码技术（深圳）有限公司 | Data transmission method, device and medium |
| CN111506556A (en) * | 2020-04-09 | 2020-08-07 | 北京市测绘设计研究院 | Multi-source heterogeneous structured data synchronization method |
| CN111639138B (en) * | 2020-06-03 | 2023-04-25 | 中国联合网络通信集团有限公司 | Data processing method, device, equipment and storage medium |
| CN111639138A (en) * | 2020-06-03 | 2020-09-08 | 中国联合网络通信集团有限公司 | Data processing method, device, equipment and storage medium |
| CN111813845A (en) * | 2020-06-29 | 2020-10-23 | 平安国际智慧城市科技股份有限公司 | ETL task-based incremental data extraction method, device, equipment and medium |
| CN111897808B (en) * | 2020-07-15 | 2023-04-11 | 苏宁金融科技(南京)有限公司 | Data processing method and device, computer equipment and storage medium |
| CN111897808A (en) * | 2020-07-15 | 2020-11-06 | 苏宁金融科技(南京)有限公司 | Data processing method and device, computer equipment and storage medium |
| CN114416703A (en) * | 2020-10-28 | 2022-04-29 | 北京中祥英科技有限公司 | Method, device, equipment and medium for automatically monitoring data integrity |
| CN112632191B (en) * | 2020-12-29 | 2024-06-11 | 中国农业银行股份有限公司 | Data processing method and system |
| CN112632191A (en) * | 2020-12-29 | 2021-04-09 | 中国农业银行股份有限公司 | Data processing method and system |
| CN112882974B (en) * | 2021-02-09 | 2024-01-05 | 深圳市云网万店科技有限公司 | JSON data conversion method and device, computer equipment and storage medium |
| CN112882974A (en) * | 2021-02-09 | 2021-06-01 | 深圳市云网万店科技有限公司 | JSON data conversion method and device, computer equipment and storage medium |
| CN113239057A (en) * | 2021-05-24 | 2021-08-10 | 量子数聚（北京）科技有限公司 | Enterprise data updating method and device |
| CN114124878B (en) * | 2022-01-27 | 2022-04-22 | 深圳华锐金融技术股份有限公司 | Market data issuing method, device, system, equipment and medium |
| CN114124878A (en) * | 2022-01-27 | 2022-03-01 | 深圳华锐金融技术股份有限公司 | Market data issuing method, device, system, equipment and medium |
| CN115600560A (en) * | 2022-09-28 | 2023-01-13 | 中电金信软件有限公司（Cn） | Data conversion method, device, system, electronic equipment and storage medium |
| CN116431688A (en) * | 2022-11-14 | 2023-07-14 | 北京远舢智能科技有限公司 | Data processing method and device, electronic equipment and storage medium |
| CN116431688B (en) * | 2022-11-14 | 2024-05-03 | 北京远舢智能科技有限公司 | Data processing method and device, electronic equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108681590A (en) | Incremental data processing method and processing device, computer equipment, computer storage media | |
| CA3087309C (en) | Blockchain-based data processing method and device | |
| CN105446991B (en) | Date storage method, querying method and equipment | |
| US20100293163A1 (en) | Operational-related data computation engine | |
| KR101735880B1 (en) | Method and system for centralized reservation context management on multi-server reservation system | |
| CN105808653B (en) | A kind of data processing method and device based on user tag system | |
| CN105164674A (en) | Queries involving multiple databases and execution engines | |
| US11689428B1 (en) | Systems and methods for visualization based on historical network traffic and future projection of infrastructure assets | |
| US20210157821A1 (en) | Blockchain-based data processing methods and apparatuses and computer devices | |
| CN109903050A (en) | Transaction De-weight method, transaction building method, equipment and storage medium | |
| CN117350880B (en) | Full-link account checking method, device, equipment and medium based on snowflake algorithm | |
| CN101452556A (en) | Customer information processing system and method | |
| CN112364102A (en) | Block chain-based big data transaction method, device, medium and equipment | |
| CN111242779B (en) | Financial data characteristic selection and prediction method, device, equipment and storage medium | |
| CN106156904A (en) | A kind of cross-platform fictitious assets source tracing method based on eID | |
| CN110275798A (en) | Block chain data processing method, device, server and storage medium | |
| CN108038225A (en) | A kind of data processing method and system | |
| CN114116908A (en) | Data management method and device and electronic equipment | |
| CN114168595A (en) | Data analysis method and device | |
| CN113159926A (en) | Loan transaction repayment date determination method and device | |
| CN112052282A (en) | Automatic modeling method and device, electronic equipment and readable storage medium | |
| CN113392114B (en) | Intelligent relationship management and intelligent data fusion method based on business object | |
| CN113609130B (en) | Method, device, electronic equipment and storage medium for acquiring gateway access data | |
| CN114820079B (en) | Crowd determination method, device, equipment and medium | |
| CN116186058A (en) | Batch data real-time processing method and device, computer equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20181019 |