CN108628931A - A kind of method, apparatus and equipment of data-driven business - Google Patents
A kind of method, apparatus and equipment of data-driven business Download PDFInfo
- Publication number
- CN108628931A CN108628931A CN201810213068.6A CN201810213068A CN108628931A CN 108628931 A CN108628931 A CN 108628931A CN 201810213068 A CN201810213068 A CN 201810213068A CN 108628931 A CN108628931 A CN 108628931A
- Authority
- CN
- China
- Prior art keywords
- data
- business
- view model
- universal
- valid
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/25—Integrating or interfacing systems involving database management systems
- G06F16/254—Extract, transform and load [ETL] procedures, e.g. ETL data flows in data warehouses
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
- Stored Programmes (AREA)
Abstract
This specification embodiment discloses a kind of method, apparatus and equipment of data-driven business.Scheme includes：Obtain the initial data of the first business, according to initial data, by ETL, universal data view model is generated, regulation engine is utilized, screen out the invalid data in the required data including at least universal data view model, valid data are obtained, according to the valid data, drive the second business, wherein, required data such as further include the characteristic of universal data view model interaction, are copied to local or other data etc. by remotely capturing.
Description
Technical field
This specification be related to computer software technical field more particularly to a kind of method, apparatus of data-driven business with
And equipment.
Background technology
It in business scenario, often needs to drive this business scenario using other business conducts, to reach the mesh of scene connection
's.
User use certain business function when, trigger another business scenario, be normally based on service application level message or
Interface docking series connection.When docking scene is excessive, R＆D costs etc. are than improving, and usually scene connection is in the presence of rule, amount etc.
Business demand, service application are difficult to provide flexible expansion.When the business side for docking a scene gradually increases, using message
The message cost of reproduction of pattern is also promoted at double.
Based on this, the business drive scheme of high efficient and flexible is needed.
Invention content
This specification embodiment provides a kind of method, apparatus and equipment of data-driven business, to solve following skill
Art problem：Need the business drive scheme of high efficient and flexible.
In order to solve the above technical problems, what this specification embodiment was realized in：
A kind of method for data-driven business that this specification embodiment provides, including：
Obtain the initial data of the first business；
According to the initial data, (Extract-Transform-Load, ETL) is loaded by extracting to convert, generates system
One Data View model；
Using regulation engine, the invalid data in the required data including at least the universal data view model is screened out,
Obtain valid data；
According to the valid data, the second business is driven.
A kind of device for data-driven business that this specification embodiment provides, including：
Acquisition module obtains the initial data of the first business；
Generation module generates universal data view model according to the initial data by ETL；
Module is screened out, using regulation engine, is screened out in the required data including at least the universal data view model
Invalid data obtains valid data；
Drive module drives the second business according to the valid data.
A kind of equipment for data-driven business that this specification embodiment provides, including：
At least one processor；And
The memory being connect at least one processor communication；Wherein,
The memory is stored with the instruction that can be executed by least one processor, and described instruction is by described at least one
A processor executes, so that at least one processor can：
Obtain the initial data of the first business；
Universal data view model is generated by ETL according to the initial data；
Using regulation engine, the invalid data in the required data including at least the universal data view model is screened out,
Obtain valid data；
According to the valid data, the second business is driven.
Above-mentioned at least one technical solution that this specification embodiment uses can reach following advantageous effect：It can be with industry
The data such as online real-time stream of business are as driving source, by ETL and rules engines processes, layout drive scheme, into
And it can drive to high efficient and flexible other business.
Description of the drawings
In order to illustrate more clearly of this specification embodiment or technical solution in the prior art, below will to embodiment or
Attached drawing needed to be used in the description of the prior art is briefly described, it should be apparent that, the accompanying drawings in the following description is only
Some embodiments described in this specification, for those of ordinary skill in the art, in not making the creative labor property
Under the premise of, other drawings may also be obtained based on these drawings.
Fig. 1 is a kind of overall architecture schematic diagram that the scheme of this specification is related under a kind of practical application scene；
Fig. 2 is a kind of flow diagram of the method for data-driven business that this specification embodiment provides；
Fig. 3 is a kind of implementation of the method for the data-driven business that this specification embodiment provides under practical application scene
Principle schematic；
Fig. 4 is a kind of structural representation of the device for data-driven business corresponding to Fig. 2 that this specification embodiment provides
Figure；
Fig. 5 is a kind of structural representation of the equipment for data-driven business corresponding to Fig. 2 that this specification embodiment provides
Figure.
Specific implementation mode
This specification embodiment provides a kind of method, apparatus and equipment of data-driven business.
In order to make those skilled in the art more fully understand the technical solution in this specification, below in conjunction with this explanation
Attached drawing in book embodiment is clearly and completely described the technical solution in this specification embodiment, it is clear that described
Embodiment be merely a part but not all of the embodiments of the present application.Based on this specification embodiment, this field
The every other embodiment that those of ordinary skill is obtained without creative efforts, should all belong to the application
The range of protection.
Data technique (Data Technology, DT) epoch Internet architecture layered mode, can be divided into number by overall architecture
According to layer and operation layer, operation layer more concern business itself, and data Layer can be used as basic platform hinge by each independent business
Scene is attached.The scheme of this specification is mostly in data Layer, using the data-driven thought of innovation, completes data service
The process of change quickly establishes scene connection between help business, is data application framework by the improvement of traditional application architecture, no matter from frame
Structure stability or aspect of performance can get a promotion, and be also beneficial to reduce hardware cost.
Fig. 1 is a kind of overall architecture schematic diagram that the scheme of this specification is related under a kind of practical application scene.This is whole
It in body framework, is carried into execution a plan by the data-driven system positioned at data Layer, data-driven system is to be in the correlation of data Layer together
Business datum is driving source, after handling driving source, to drive the business for being located at operation layer.Business includes multiple business
Scene can be specifically the corresponding business scenario of driving.
The exemplary architecture being based primarily upon below in Fig. 1 illustrates the scheme of this specification.
Fig. 2 is a kind of flow diagram of the method for data-driven business that this specification embodiment provides.Stream in Fig. 2
Journey may comprise steps of：
S202：Obtain the initial data of the first business.
In this specification embodiment, after handling a kind of initial data of business, the business itself may be driven
Or another business, " driving " here can be regarded as the business action in triggering business scenario and the business scenario.For
Convenient for statement, the first business will be known as the affiliated business of the body data of driving source, driven business is known as second
Business.
In this specification embodiment, initial data such as includes behavioral data and/or pipelined data etc..With electric business business
For：Behavioral data is able to record the browsing commodity of user, addition commodity to collection, places an order, evaluates the behaviors such as trade company, moreover it is possible to
Enough behaviors such as the upper undercarriage commodity of record trade company, evaluation user, addition logistics information；Pipelined data can include the order of user
The information such as detail, detail of transferring accounts, more directly have recorded the trading activity between user and trade company.
Further, for inline system, requirement of real-time is higher, correspondingly, can also be obtained in step S202
The real-time initial data of line, to be conducive to subsequently drive business in time.For example, can be with online data stream with initial data
Form flow into, message queue, which may be used, in inflow mode, database synchronization, daily record synchronize etc. is lost in data access side
Formula.
S204：Universal data view model is generated by ETL according to the initial data.
In this specification embodiment, it can be realized data from source terminal by ETL by extracting, converting, be loaded onto
The process of destination usually can be got a promotion by the ease for use of the processed data of ETL.Wherein, transfer process is such as wrapped
It includes：Processing empty value, standardization data format split data, verify data correctness, data replacement, data query, establish outside main
The actions such as key constraint.
Initial data is the data of inclined bottom, not readily understood, and may be belonged in different ranges (for example, different
Class, different structures, different business scenarios etc.), be unfavorable for subsequently being uniformly processed.Based on this, by ETL to original number
According to unitized processing and viewization processing, more easy-to-use universal data view model is generated, Data View model is in unified model
Middle expression initial data is enclosed, and for not intelligible data, additionally it is possible to show with view.
S206：Using regulation engine, it is invalid in the required data including at least the universal data view model to screen out
Data obtain valid data.
In this specification embodiment, required data refer to the data needed for subsequent drive business to be realized, in addition to including system
Other than one Data View model, it is also possible to include following at least one data：The characteristic of universal data view model interaction,
It is copied to local or other data by remotely capturing.
The characteristic can be generated according to universal data view model, be generally comprised in universal data view model
Some statistical data.Other described data are usually directly related with the business drive condition of setting, need to combine other data,
Whether decision carries out business driving, and specifically how to drive, other data such as include environment number when the first business carries out
According to the data etc. of the associated services of, the first business.
In this specification embodiment, in regulation engine, rule can be screened out according to business drive condition setting data,
Data screen out rule and screen out condition including one or more, according to condition is screened out, screen out invalid data.
For example it is assumed that required data are denoted as q, data screen out rule include condition 1 and condition 2 the two screen out condition.
Condition 1：Assuming that there are attribute a, b by q, it is desirable that a>5 and b is not belonging to set m；
Condition 2：Assuming that there are attribute c by q, it is desirable that c%3=0, this condition ignore condition 1 when being true.
Then according to condition 1 and condition, after screening out the invalid data in q, q meets：
“(q.a>5＆＆q.b not in m) | | (c%3=0) ".
In this specification embodiment, valid data can be used as by screening out the required data after invalid data, alternatively, can
It can need to screen out after the required data after invalid data are further processed, be re-used as valid data.
S208：According to the valid data, the second business is driven.
In this specification embodiment, in the presence of valid data, the second business is driven, alternatively, existing in valid data
And when meeting other business drive conditions, the second business is driven.
By the method for Fig. 2, ETL can be passed through using data such as the online real-time streams of business as driving source
And rules engines processes, layout drive scheme, and then can drive to high efficient and flexible other business.
Method based on Fig. 2, this specification embodiment additionally provide some specific embodiments of this method, and extension
Scheme is illustrated below.
It is described to utilize regulation engine for step S206 in this specification embodiment, it screens out including at least the unification
Invalid data in the required data of Data View model, obtains valid data, can specifically include：Obtain the uniform data
Required data other than view model, the required data include following at least one：The universal data view model interaction
Characteristic is copied to local or other data by remotely capturing；According to the universal data view model and
Required data other than it, merging obtain calculating data；Using regulation engine, the invalid data in the calculating data is screened out,
Obtain valid data.
In this specification embodiment, characteristic can be extracted by Feature Engineering or the modes such as artificial.
In Feature Engineering mode as an example.Specifically, Feature Engineering processing can be carried out to the universal data view model,
Obtain the characteristic of the universal data view model interaction, wherein the characteristic include using specified main body as dimension,
Following at least one statistical result of its specified attribute data at the appointed time window：Tired note value, most value, mean value, duplicate removal statistics
Value.
For electric business business, it is such as user, trade company or commodity etc. to specify main body.Aggregate-value is such as user one
The accumulation consumption number of times etc. of the progressive payment amount of money or user in one week in week；It is maximum value or most to be most worth such as
Small value；Duplicate removal statistical value is such as the trademark quantity that user buys in one week, if user has purchased the two pieces of same brand or more
Commodity, then belong to repetition, only primary for brand note, i.e. duplicate removal.
In this specification embodiment, certain business within a certain period of time can driven number be conditional, if
Through reaching the driving number upper limit (being known as amount), then corresponding calculating data can be used as excess data to abandon, and be not used in follow-up
Processing procedure.It should be noted that excess situation is not limited to this one kind, if for example, multiple business scenarios are to select one to be driven
, after one of them is driven, other are no longer driven, then after one of them is driven, subsequently there is this multiple business scenario again
Corresponding calculating data can be screened out as excess data.
According to the analysis of the preceding paragraph, the invalid data screened out in the calculating data obtains valid data, specifically may be used
To include：Screen out the invalid data in the calculating data；According to the calculating data after the invalid data are screened out, amount is carried out
It calculates, to screen out excess data, obtains valid data.This process is typically based on the progress of certain main body dimension, for example, for single
The main bodys such as user, shop or commodity analyze excess situation, and then carry out excess calculating.
For example, according to the consumption in certain retail shop on the day of single user, when consumption reaches 500 yuan, and when consumption
When reaching 1000 yuan, a red packet can be rewarded to user respectively.The consumption data of the user is driving source, and reward red packet is to be wanted
The business of driving is currently at most awarded two red packets for the user, and the consumption data within 1000 yuan is enough accordingly
Ground drives business twice, and the consumption data more than 1000 yuan may belong to excess data.
In this specification embodiment, under some scenes, the second business can be directly driven using valid data, but
It is that under other scenes, may not can directly drive the second business using valid data, but may need to valid data
Further processing calculates, and adapts it to the scene of the second business, then, the valid data after recycling processing to calculate,
Drive the second business.Such as, it is assumed that the first business is payment transaction, customer consumption data that valid data are that treated, second
Business is public good game, and there is ideal money in public good game the inside, then it is such as by customer consumption number that above-mentioned processing, which calculates specifically,
According to according to predetermined policy, the ideal money being converted to, and then trigger some scenes in public good game, to realize that business is driven
It is dynamic.
In this specification embodiment, the type of drive of the second business of driving is various, for example, type of drive can prop up
Hold more middle agreements such as message queue, far call, HTTP2, dynamic script.In addition, for on-line system, can utilize a variety of same
Step mode, various data needed for synchronous service driving process, for above-mentioned flow.
According to explanation above, the method for the above-mentioned data-driven business that this specification embodiment also provides is in a kind of reality
Implementation principle schematic diagram under application scenarios, as shown in Figure 3.
In figure 3, by corresponding data-driven system, the method for executing above-mentioned data-driven business.The system is with certain
(step 1) in corresponding diagram 3, layout drive scheme is (in corresponding diagram 3 the step of p1, p2, p3 as driving source for the data flow of business
2~5), and then another business is driven.By taking wherein p1 as an example, business driving flow mainly includes the following steps that：
1, online real time data d is flowed into；
2, d passes through ETL processing, generates universal data view model d '；
2.1, d ' is inputted into real-time computing engines, carries out Feature Engineering processing, the associated characteristic f1, f2 of output d ',
f3、…、fn；
2.2, it polymerize other required data e, d ', f1~fn, e is merged into and calculate data q；
3, by q input rule engines, invalid data is screened out；
4, excess calculating is carried out to screening out the q after invalid data, screens out excess data；
5, it is processed calculating to screening out the q after excess data, generates q '；
6, another business is driven using q '.
Said program has the following advantages：Data-driven system is in overall architecture data Layer, and bottom-up driving can
Each business of application layer is decoupled, framework stability is improved；The connection between scene is completed in a manner of operation data, it is whole
The layout of a drive scheme is completed by configuration mode, is developed without application layer；Data-driven system can be used and quasi real time be driven
Dynamic scheme, ensures higher business timeliness；The data statistics of supported feature engineering calculates, extending real-time data attribute.
Based on same thinking, this specification embodiment additionally provides the corresponding device of the above method and equipment, referring to such as
Shown in Fig. 4, Fig. 5.
Fig. 4 is a kind of structural representation of the device for data-driven business corresponding to Fig. 2 that this specification embodiment provides
Figure, described device include：
Acquisition module 401 obtains the initial data of the first business；
Generation module 402 loads ETL by extracting to convert, generates universal data view mould according to the initial data
Type；
Module 403 is screened out, using regulation engine, is screened out in the required data including at least the universal data view model
Invalid data, obtain valid data；
Drive module 404 drives the second business according to the valid data.
Optionally, the initial data includes online behavioral data and/or pipelined data in real time.
Optionally, the module 403 that screens out utilizes regulation engine, screens out including at least the universal data view model
Invalid data in required data, obtains valid data, specifically includes：
The module 403 that screens out obtains required data other than the universal data view model, which includes
Following at least one：The characteristic of the universal data view model interaction is copied to locally or by remotely capturing
Other data arrived；
Required data according to the universal data view model and its in addition, merging obtain calculating data；
Using regulation engine, the invalid data in the calculating data is screened out, valid data are obtained.
Optionally, the characteristic for screening out module 403 and obtaining the universal data view model interaction is specific to wrap
It includes：
The module 403 that screens out carries out Feature Engineering processing to the universal data view model, obtains the unified number
According to the associated characteristic of view model；
Wherein, the characteristic includes the specified attribute data at the appointed time window using specified main body as dimension
Following at least one statistical result：Tired note value, most value, mean value, duplicate removal statistical value.
Optionally, the module 403 that screens out screens out the invalid data calculated in data, obtains valid data, also wraps
It includes：
The module 403 that screens out screens out the invalid data calculated in data；
According to the calculating data after the invalid data are screened out, amount calculating is carried out, to screen out excess data, is obtained effectively
Data.
Optionally, the drive module 404 drives the second business, specifically includes according to the valid data：
The drive module 404 drives the second business according to the valid data in the way of following at least one：Disappear
Cease queue, far call, HTTP2, dynamic script.
Optionally, the drive module 404 drives the second business, specifically includes according to the valid data：
The drive module 404 is processed calculating to the valid data, adapts it to the scene of the second business；
Valid data after being calculated using the processing, drive second business.
Fig. 5 is a kind of structural representation of the equipment for data-driven business corresponding to Fig. 2 that this specification embodiment provides
Figure, the equipment include：
At least one processor；And
The memory being connect at least one processor communication；Wherein,
The memory is stored with the instruction that can be executed by least one processor, and described instruction is by described at least one
A processor executes, so that at least one processor can：
Obtain the initial data of the first business；
Universal data view model is generated by ETL according to the initial data；
Using regulation engine, the invalid data in the required data including at least the universal data view model is screened out,
Obtain valid data；
According to the valid data, the second business is driven.
Based on same thinking, this specification embodiment additionally provides a kind of non-volatile computer corresponding to Fig. 2 and deposits
Storage media, is stored with computer executable instructions, and the computer executable instructions are set as：
Obtain the initial data of the first business；
Universal data view model is generated by ETL according to the initial data；
Using regulation engine, the invalid data in the required data including at least the universal data view model is screened out,
Obtain valid data；
According to the valid data, the second business is driven.
It is above-mentioned that this specification specific embodiment is described.Other embodiments are in the scope of the appended claims
It is interior.In some cases, the action recorded in detail in the claims or step can be come according to different from the sequence in embodiment
It executes and desired result still may be implemented.In addition, the process described in the accompanying drawings not necessarily require show it is specific suitable
Sequence or consecutive order could realize desired result.In some embodiments, multitasking and parallel processing be also can
With or it may be advantageous.
Each embodiment in this specification is described in a progressive manner, identical similar portion between each embodiment
Point just to refer each other, and each embodiment focuses on the differences from other embodiments.Especially for device,
For equipment, nonvolatile computer storage media embodiment, since it is substantially similar to the method embodiment, so the ratio of description
Relatively simple, the relevent part can refer to the partial explaination of embodiments of method.
Device that this specification embodiment provides, equipment, nonvolatile computer storage media with method be it is corresponding, because
This, device, equipment, nonvolatile computer storage media also have the advantageous effects similar with corresponding method, due to upper
Face is described in detail the advantageous effects of method, therefore, which is not described herein again corresponding intrument, equipment, it is non-easily
The advantageous effects of the property lost computer storage media.
In the 1990s, the improvement of a technology can be distinguished clearly be on hardware improvement (for example,
Improvement to circuit structures such as diode, transistor, switches) or software on improvement (improvement for method flow).So
And with the development of technology, the improvement of current many method flows can be considered as directly improving for hardware circuit.
Designer nearly all obtains corresponding hardware circuit by the way that improved method flow to be programmed into hardware circuit.Cause
This, it cannot be said that the improvement of a method flow cannot be realized with hardware entities module.For example, programmable logic device
(Programmable Logic Device, PLD) (such as field programmable gate array (Field Programmable Gate
Array, FPGA)) it is exactly such a integrated circuit, logic function determines device programming by user.By designer
Voluntarily programming comes a digital display circuit " integrated " on a piece of PLD, designs and makes without asking chip maker
Dedicated IC chip.Moreover, nowadays, substitution manually makes IC chip, this programming is also used instead mostly " patrols
Volume compiler (logic compiler) " software realizes that software compiler used is similar when it writes with program development,
And the source code before compiling also write by handy specific programming language, this is referred to as hardware description language
(Hardware Description Language, HDL), and HDL is also not only a kind of, but there are many kind, such as ABEL
(Advanced Boolean Expression Language)、AHDL(Altera Hardware Description
Language)、Confluence、CUPL(Cornell University Programming Language)、HDCal、JHDL
(Java Hardware Description Language)、Lava、Lola、MyHDL、PALASM、RHDL(Ruby
Hardware Description Language) etc., VHDL (Very-High-Speed are most generally used at present
Integrated Circuit Hardware Description Language) and Verilog.Those skilled in the art also answer
This understands, it is only necessary to method flow slightly programming in logic and is programmed into integrated circuit with above-mentioned several hardware description languages,
The hardware circuit for realizing the logical method flow can be readily available.
Controller can be implemented in any suitable manner, for example, controller can take such as microprocessor or processing
The computer for the computer readable program code (such as software or firmware) that device and storage can be executed by (micro-) processor can
Read medium, logic gate, switch, application-specific integrated circuit (Application Specific Integrated Circuit,
ASIC), the form of programmable logic controller (PLC) and embedded microcontroller, the example of controller includes but not limited to following microcontroller
Device：ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20 and Silicone Labs C8051F320, are deposited
Memory controller is also implemented as a part for the control logic of memory.It is also known in the art that in addition to
Pure computer readable program code mode is realized other than controller, can be made completely by the way that method and step is carried out programming in logic
Controller is obtained in the form of logic gate, switch, application-specific integrated circuit, programmable logic controller (PLC) and embedded microcontroller etc. to come in fact
Existing identical function.Therefore this controller is considered a kind of hardware component, and to including for realizing various in it
The device of function can also be considered as the structure in hardware component.Or even, it can will be regarded for realizing the device of various functions
For either the software module of implementation method can be the structure in hardware component again.
System, device, module or the unit that above-described embodiment illustrates can specifically realize by computer chip or entity,
Or it is realized by the product with certain function.It is a kind of typically to realize that equipment is computer.Specifically, computer for example may be used
Think personal computer, laptop computer, cellular phone, camera phone, smart phone, personal digital assistant, media play
It is any in device, navigation equipment, electronic mail equipment, game console, tablet computer, wearable device or these equipment
The combination of equipment.
For convenience of description, it is divided into various units when description apparatus above with function to describe respectively.Certainly, implementing this
The function of each unit is realized can in the same or multiple software and or hardware when specification.
It should be understood by those skilled in the art that, this specification embodiment can be provided as method, system or computer program
Product.Therefore, this specification embodiment can be used complete hardware embodiment, complete software embodiment or combine software and hardware
The form of the embodiment of aspect.Moreover, it wherein includes that computer is available that this specification embodiment, which can be used in one or more,
It is real in the computer-usable storage medium (including but not limited to magnetic disk storage, CD-ROM, optical memory etc.) of program code
The form for the computer program product applied.
This specification is with reference to the method, equipment (system) and computer program product according to this specification embodiment
Flowchart and/or the block diagram describes.It should be understood that can be realized by computer program instructions every in flowchart and/or the block diagram
The combination of flow and/or box in one flow and/or box and flowchart and/or the block diagram.These computers can be provided
Processor of the program instruction to all-purpose computer, special purpose computer, Embedded Processor or other programmable data processing devices
To generate a machine so that the instruction executed by computer or the processor of other programmable data processing devices generates use
In the dress for realizing the function of being specified in one flow of flow chart or multiple flows and/or one box of block diagram or multiple boxes
It sets.
These computer program instructions, which may also be stored in, can guide computer or other programmable data processing devices with spy
Determine in the computer-readable memory that mode works so that instruction generation stored in the computer readable memory includes referring to
Enable the manufacture of device, the command device realize in one flow of flow chart or multiple flows and/or one box of block diagram or
The function of being specified in multiple boxes.
These computer program instructions also can be loaded onto a computer or other programmable data processing device so that count
Series of operation steps are executed on calculation machine or other programmable devices to generate computer implemented processing, in computer or
The instruction executed on other programmable devices is provided for realizing in one flow of flow chart or multiple flows and/or block diagram one
The step of function of being specified in a box or multiple boxes.
In a typical configuration, computing device includes one or more processors (CPU), input/output interface, net
Network interface and memory.
Memory may include computer-readable medium in volatile memory, random access memory (RAM) and/or
The forms such as Nonvolatile memory, such as read-only memory (ROM) or flash memory (flash RAM).Memory is computer-readable medium
Example.
Computer-readable medium includes permanent and non-permanent, removable and non-removable media can be by any method
Or technology realizes information storage.Information can be computer-readable instruction, data structure, the module of program or other data.
The example of the storage medium of computer includes, but are not limited to phase transition internal memory (PRAM), static RAM (SRAM), moves
State random access memory (DRAM), other kinds of random access memory (RAM), read-only memory (ROM), electric erasable
Programmable read only memory (EEPROM), fast flash memory bank or other memory techniques, read-only disc read only memory (CD-ROM) (CD-ROM),
Digital versatile disc (DVD) or other optical storages, magnetic tape cassette, tape magnetic disk storage or other magnetic storage apparatus
Or any other non-transmission medium, it can be used for storage and can be accessed by a computing device information.As defined in this article, it calculates
Machine readable medium does not include temporary computer readable media (transitory media), such as data-signal and carrier wave of modulation.
It should also be noted that, the terms "include", "comprise" or its any other variant are intended to nonexcludability
Including so that process, method, commodity or equipment including a series of elements include not only those elements, but also wrap
Include other elements that are not explicitly listed, or further include for this process, method, commodity or equipment intrinsic want
Element.In the absence of more restrictions, the element limited by sentence "including a ...", it is not excluded that wanted including described
There is also other identical elements in the process of element, method, commodity or equipment.
It will be understood by those skilled in the art that this specification embodiment can be provided as method, system or computer program product.
Therefore, complete hardware embodiment, complete software embodiment or embodiment combining software and hardware aspects can be used in this specification
Form.Moreover, can be used can in the computer that one or more wherein includes computer usable program code for this specification
With the computer program product implemented on storage medium (including but not limited to magnetic disk storage, CD-ROM, optical memory etc.)
Form.
This specification can describe in the general context of computer-executable instructions executed by a computer, such as journey
Sequence module.Usually, program module include routines performing specific tasks or implementing specific abstract data types, program, object,
Component, data structure etc..This specification can also be put into practice in a distributed computing environment, in these distributed computing environment
In, by executing task by the connected remote processing devices of communication network.In a distributed computing environment, program module
It can be located in the local and remote computer storage media including storage device.
Each embodiment in this specification is described in a progressive manner, identical similar portion between each embodiment
Point just to refer each other, and each embodiment focuses on the differences from other embodiments.Especially for system reality
For applying example, since it is substantially similar to the method embodiment, so description is fairly simple, related place is referring to embodiment of the method
Part explanation.
The foregoing is merely this specification embodiments, are not intended to limit this application.For those skilled in the art
For, the application can have various modifications and variations.It is all within spirit herein and principle made by any modification, equivalent
Replace, improve etc., it should be included within the scope of claims hereof.
Claims (15)
1. a kind of method of data-driven business, including：
Obtain the initial data of the first business；
According to the initial data, ETL is loaded by extracting to convert, generates universal data view model；
Using regulation engine, the invalid data in the required data including at least the universal data view model is screened out, is obtained
Valid data；
According to the valid data, the second business is driven.
2. the method as described in claim 1, the initial data includes online behavioral data and/or pipelined data in real time.
3. the method as described in claim 1, described to utilize regulation engine, screen out including at least the universal data view model
Required data in invalid data, obtain valid data, specifically include：
The required data other than the universal data view model are obtained, which includes following at least one：The system
The characteristic of one Data View model interaction is copied to local or other data by remotely capturing；
Required data according to the universal data view model and its in addition, merging obtain calculating data；
Using regulation engine, the invalid data in the calculating data is screened out, valid data are obtained.
4. method as claimed in claim 3 obtains the characteristic of the universal data view model interaction, specifically includes：
Feature Engineering processing is carried out to the universal data view model, obtains the feature of the universal data view model interaction
Data；
Wherein, the characteristic include using specified main body as dimension, specified attribute data at the appointed time window it is following
At least one statistical result：Tired note value, most value, mean value, duplicate removal statistical value.
5. method as claimed in claim 3, the invalid data screened out in the calculating data, obtain valid data, have
Body includes：
Screen out the invalid data in the calculating data；
According to the calculating data after the invalid data are screened out, carries out amount calculating and obtain significant figure to screen out excess data
According to.
6. the method as described in claim 1, described according to the valid data, the second business is driven, is specifically included：
According to the valid data the second business is driven in the way of following at least one：Message queue, far call,
HTTP2, dynamic script.
7. the method as described in claim 1, described according to the valid data, the second business is driven, is specifically included：
Calculating is processed to the valid data, adapts it to the scene of the second business；
Valid data after being calculated using the processing, drive second business.
8. a kind of device of data-driven business, including：
Acquisition module obtains the initial data of the first business；
Generation module loads ETL by extracting to convert, generates universal data view model according to the initial data；
Module is screened out, using regulation engine, it is invalid in the required data including at least the universal data view model to screen out
Data obtain valid data；
Drive module drives the second business according to the valid data.
9. device as claimed in claim 8, the initial data includes online behavioral data and/or pipelined data in real time.
10. device as claimed in claim 8, the module that screens out utilizes regulation engine, screens out including at least the unified number
According to the invalid data in the required data of view model, valid data are obtained, are specifically included：
The module that screens out obtains required data other than the universal data view model, the required data include it is following at least
It is a kind of：The characteristic of the universal data view model interaction is copied to local or other by remotely capturing
Data；
Required data according to the universal data view model and its in addition, merging obtain calculating data；
Using regulation engine, the invalid data in the calculating data is screened out, valid data are obtained.
11. device as claimed in claim 10, the feature for screening out module and obtaining the universal data view model interaction
Data specifically include：
The module that screens out carries out Feature Engineering processing to the universal data view model, obtains the universal data view mould
The associated characteristic of type；
Wherein, the characteristic include using specified main body as dimension, specified attribute data at the appointed time window it is following
At least one statistical result：Tired note value, most value, mean value, duplicate removal statistical value.
12. device as claimed in claim 10, the module that screens out screens out the invalid data calculated in data, is had
Data are imitated, are specifically included：
The module that screens out screens out the invalid data calculated in data；
According to the calculating data after the invalid data are screened out, carries out amount calculating and obtain significant figure to screen out excess data
According to.
13. device as claimed in claim 8, the drive module drives the second business according to the valid data, specific to wrap
It includes：
The drive module drives the second business according to the valid data in the way of following at least one：Message queue,
Far call, HTTP2, dynamic script.
14. device as claimed in claim 8, the drive module drives the second business according to the valid data, specific to wrap
It includes：
The drive module is processed calculating to the valid data, adapts it to the scene of the second business；
Valid data after being calculated using the processing, drive second business.
15. a kind of equipment of data-driven business, including：
At least one processor；And
The memory being connect at least one processor communication；Wherein,
The memory is stored with the instruction that can be executed by least one processor, and described instruction is by least one place
It manages device to execute, so that at least one processor can：
Obtain the initial data of the first business；
According to the initial data, ETL is loaded by extracting to convert, generates universal data view model；
Using regulation engine, the invalid data in the required data including at least the universal data view model is screened out, is obtained
Valid data；
According to the valid data, the second business is driven.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810213068.6A CN108628931B (en) | 2018-03-15 | 2018-03-15 | Method, device and equipment for data driving service |
| CN202211110282.1A CN115563197A (en) | 2018-03-15 | 2018-03-15 | Method, device and equipment for data driving service |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810213068.6A CN108628931B (en) | 2018-03-15 | 2018-03-15 | Method, device and equipment for data driving service |
Related Child Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211110282.1A Division CN115563197A (en) | 2018-03-15 | 2018-03-15 | Method, device and equipment for data driving service |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108628931A true CN108628931A (en) | 2018-10-09 |
| CN108628931B CN108628931B (en) | 2022-08-30 |
Family
ID=63706206
Family Applications (2)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211110282.1A Pending CN115563197A (en) | 2018-03-15 | 2018-03-15 | Method, device and equipment for data driving service |
| CN201810213068.6A Active CN108628931B (en) | 2018-03-15 | 2018-03-15 | Method, device and equipment for data driving service |
Family Applications Before (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211110282.1A Pending CN115563197A (en) | 2018-03-15 | 2018-03-15 | Method, device and equipment for data driving service |
Country Status (1)
| Country | Link |
|---|---|
| CN (2) | CN115563197A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109614205A (en) * | 2018-10-18 | 2019-04-12 | 阿里巴巴集团控股有限公司 | A kind of method for processing business, device, equipment and system |
| CN111144950A (en) * | 2019-12-30 | 2020-05-12 | 北京顺丰同城科技有限公司 | Model screening method and device, electronic equipment and storage medium |
| CN112084179A (en) * | 2020-09-02 | 2020-12-15 | 北京锐安科技有限公司 | Data processing method, device, equipment and storage medium |
Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102567796A (en) * | 2010-12-20 | 2012-07-11 | 北大方正集团有限公司 | Method and device for importing data |
| CN102722582A (en) * | 2012-06-07 | 2012-10-10 | 陈浩 | System and method for integrating data on basis of reverse clearing |
| US20130205252A1 (en) * | 2012-02-06 | 2013-08-08 | International Business Machines Corporation | Conveying hierarchical elements of a user interface |
| US20150081619A1 (en) * | 2013-09-19 | 2015-03-19 | Platfora, Inc. | Systems and Methods for Interest-Driven Business Intelligence Systems Including Geo-Spatial Data |
| US20160162825A1 (en) * | 2014-12-05 | 2016-06-09 | International Business Machines Corporation | Monitoring the impact of information quality on business application components through an impact map to data sources |
| CN106339222A (en) * | 2016-08-23 | 2017-01-18 | 联动优势电子商务有限公司 | Service implementing method and device |
| CN106874482A (en) * | 2017-02-20 | 2017-06-20 | 山东鲁能软件技术有限公司 | A kind of device and method of the patterned data prediction based on big data technology |
| CN107103548A (en) * | 2011-11-17 | 2017-08-29 | 阿里巴巴集团控股有限公司 | The monitoring method and system and risk monitoring and control method and system of network behavior data |
| CN107526600A (en) * | 2017-09-05 | 2017-12-29 | 成都优易数据有限公司 | A kind of visual numeric simulation analysis platform and its data cleaning method based on hadoop and spark |
| WO2018027226A1 (en) * | 2016-08-05 | 2018-02-08 | Fractal Industries, Inc. | Detection mitigation and remediation of cyberattacks employing an advanced cyber-decision platform |
| CN107766493A (en) * | 2017-10-19 | 2018-03-06 | 国网辽宁省电力有限公司 | A kind of patch source power network resources Model analysis system and method |
-
2018
- 2018-03-15 CN CN202211110282.1A patent/CN115563197A/en active Pending
- 2018-03-15 CN CN201810213068.6A patent/CN108628931B/en active Active
Patent Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102567796A (en) * | 2010-12-20 | 2012-07-11 | 北大方正集团有限公司 | Method and device for importing data |
| CN107103548A (en) * | 2011-11-17 | 2017-08-29 | 阿里巴巴集团控股有限公司 | The monitoring method and system and risk monitoring and control method and system of network behavior data |
| US20130205252A1 (en) * | 2012-02-06 | 2013-08-08 | International Business Machines Corporation | Conveying hierarchical elements of a user interface |
| CN102722582A (en) * | 2012-06-07 | 2012-10-10 | 陈浩 | System and method for integrating data on basis of reverse clearing |
| US20150081619A1 (en) * | 2013-09-19 | 2015-03-19 | Platfora, Inc. | Systems and Methods for Interest-Driven Business Intelligence Systems Including Geo-Spatial Data |
| US20160162825A1 (en) * | 2014-12-05 | 2016-06-09 | International Business Machines Corporation | Monitoring the impact of information quality on business application components through an impact map to data sources |
| WO2018027226A1 (en) * | 2016-08-05 | 2018-02-08 | Fractal Industries, Inc. | Detection mitigation and remediation of cyberattacks employing an advanced cyber-decision platform |
| CN106339222A (en) * | 2016-08-23 | 2017-01-18 | 联动优势电子商务有限公司 | Service implementing method and device |
| CN106874482A (en) * | 2017-02-20 | 2017-06-20 | 山东鲁能软件技术有限公司 | A kind of device and method of the patterned data prediction based on big data technology |
| CN107526600A (en) * | 2017-09-05 | 2017-12-29 | 成都优易数据有限公司 | A kind of visual numeric simulation analysis platform and its data cleaning method based on hadoop and spark |
| CN107766493A (en) * | 2017-10-19 | 2018-03-06 | 国网辽宁省电力有限公司 | A kind of patch source power network resources Model analysis system and method |
Non-Patent Citations (2)
| Title |
|---|
| HU ZHI-NAN等: "Metadata-driven middleware for customizable active SMS service", 《COMPUTER ENGINEERING AND DESIGN》 * |
| 胡细平: "业务驱动型信息分析平台研究", 《信息化研究》 * |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109614205A (en) * | 2018-10-18 | 2019-04-12 | 阿里巴巴集团控股有限公司 | A kind of method for processing business, device, equipment and system |
| CN111144950A (en) * | 2019-12-30 | 2020-05-12 | 北京顺丰同城科技有限公司 | Model screening method and device, electronic equipment and storage medium |
| CN111144950B (en) * | 2019-12-30 | 2023-06-30 | 北京顺丰同城科技有限公司 | Model screening method and device, electronic equipment and storage medium |
| CN112084179A (en) * | 2020-09-02 | 2020-12-15 | 北京锐安科技有限公司 | Data processing method, device, equipment and storage medium |
| CN112084179B (en) * | 2020-09-02 | 2023-11-07 | 北京锐安科技有限公司 | Data processing method, device, equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115563197A (en) | 2023-01-03 |
| CN108628931B (en) | 2022-08-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107391526A (en) | A kind of data processing method and equipment based on block chain | |
| CN107424069A (en) | A kind of generation method of air control feature, risk monitoring and control method and apparatus | |
| CN107644286A (en) | Workflow processing method and device | |
| CN107450972A (en) | A kind of dispatching method, device and electronic equipment | |
| CN107679700A (en) | Business flow processing method, apparatus and server | |
| CN109934697A (en) | A kind of credit risk control method, device and equipment based on graph structure model | |
| CN108154211A (en) | Quick Response Code generation, method for processing business, device and equipment and Quick Response Code | |
| CN110134668A (en) | Data migration method, device and equipment applied to block chain | |
| CN107748989A (en) | A kind of virtual card opens chucking method, system and payment system, card sending system | |
| CN105335389B (en) | A kind of business method for visualizing and system | |
| CN108628931A (en) | A kind of method, apparatus and equipment of data-driven business | |
| CN108173706A (en) | Service marker method, apparatus and equipment under a kind of multiservice system | |
| CN107015871A (en) | A kind of data processing method and device | |
| CN109271587A (en) | A kind of page generation method and device | |
| CN108647065A (en) | A kind of information displaying method, device and equipment | |
| CN110309034A (en) | Work flow link view generation method and device | |
| CN108021610A (en) | Random walk, random walk method, apparatus and equipment based on distributed system | |
| CN110019277A (en) | A kind of method, the method, device and equipment of data query of data accumulation | |
| CN109993646A (en) | Accounting entry information determines method and device, accounting data recording method and device | |
| CN107578338A (en) | A kind of service publishing method, device and equipment | |
| CN109597678A (en) | Task processing method and device | |
| CN109003090A (en) | risk control method and device | |
| CN108255471A (en) | A kind of system configuration item configuration device based on configuration external member, method and apparatus | |
| Faroukhi et al. | A novel approach for big data monetization as a service | |
| CN109886804A (en) | A kind of task processing method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20200918 Address after: Cayman Enterprise Centre, 27 Hospital Road, George Town, Grand Cayman Islands Applicant after: Advanced innovation technology Co.,Ltd. Address before: A four-storey 847 mailbox in Grand Cayman Capital Building, British Cayman Islands Applicant before: Alibaba Group Holding Ltd. Effective date of registration: 20200918 Address after: Cayman Enterprise Centre, 27 Hospital Road, George Town, Grand Cayman Islands Applicant after: Innovative advanced technology Co.,Ltd. Address before: Cayman Enterprise Centre, 27 Hospital Road, George Town, Grand Cayman Islands Applicant before: Advanced innovation technology Co.,Ltd. |
|
| TA01 | Transfer of patent application right | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |