CN109002470A - Knowledge mapping construction method and device, client - Google Patents
Knowledge mapping construction method and device, client Download PDFInfo
- Publication number
- CN109002470A CN109002470A CN201810603636.3A CN201810603636A CN109002470A CN 109002470 A CN109002470 A CN 109002470A CN 201810603636 A CN201810603636 A CN 201810603636A CN 109002470 A CN109002470 A CN 109002470A
- Authority
- CN
- China
- Prior art keywords
- data
- spectrum
- source
- data source
- target
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
This application discloses a kind of knowledge mapping construction method and devices, client.This method includes increasing by the second data source into the first data source to obtain target data source；It accesses the target data source and handles the data in the target data source according to preset rules；And generate spectrum data.Present application addresses lack for the application technology problem in terms of financial business.The knowledge mapping that can obtain supporting more business function according to the spectrum data for expanding application range by the application can effectively assist the related sales service of internet finance.
Description
Technical field
This application involves Internet technical fields, in particular to a kind of knowledge mapping construction method and device, client
End.
Background technique
Knowledge mapping, also referred to as mapping knowledge domains are known as knowledge domain visualization or ken in books and information group
Map is mapped, is a series of a variety of different figures of explicit knowledge's development process and structural relation, is described with visualization technique
Knowledge resource and its carrier, excavation, analysis, building, drafting and the correlation between explicit knowledge and knowledge.Pass through knowledge graph
Spectrum such as is substantially intended to describe between the knowledge and knowledge of real world objective reality at the semantic network of incidence relations.
Inventors have found that existing knowledge mapping is generally focused on personnel's relationship quarter in internet financial field at present
It draws, in fraud information discovery, the output of products in terms of the financial business being related to is less, passes through knowledge mapping bring business valence
It is worth smaller.Further, lack the support to sales service.
For lacking in the related technology for the application problem in terms of financial business, effective solution side is not yet proposed at present
Case.
Summary of the invention
The main purpose of the application is to provide a kind of knowledge mapping construction method, to solve to lack for financial business side
The technical issues of application in face.
To achieve the goals above, according to the one aspect of the application, a kind of knowledge mapping construction method is provided.It is based on
Product purchase relationship, the real social networks of user, virtual social relationship etc. of user are collected in the basis of Data Integration, can be with
More technical supports are provided for operation department.
Knowledge mapping construction method according to the application includes: to increase by the second data source into the first data source to obtain target
Data source；It accesses the target data source and handles the data in the target data source according to preset rules；And it is raw
At spectrum data.
Further, increase by the second data source into the first data source and obtain target data source and comprise determining that described first
First object data in data source, wherein the first object data are used to be used as internal data；Receive second data
The second target data in source, wherein second target data is used to be used as external data；And by the first object number
Unified structure is configured to according to preset interface rule according to, second target data.
Further, it accesses the target data source and handles the number in the target data source according to preset rules
According to including: the access target data source and establish database model according to presetting database model rule；And according to default
Data label integrates the data in the target data source in the database model.
Further, it accesses the target data source and handles the number in the target data source according to preset rules
According to including: to determine the first spectrum data according to the data scale of construction in target data source；Pass through the data type in target data source
Generate the second spectrum data, third spectrum data and the 4th spectrum data；Judge first spectrum data, the second map number
Whether meet business need when using response according to, third spectrum data and the 4th spectrum data；If it is determined that described first
Spectrum data, the second spectrum data, third spectrum data and the 4th spectrum data meet business need when using response, then
Execute the default conversion operation from source terminal to destination, wherein first spectrum data, for as basic attribute data
Model；Second spectrum data, for being used as social relationships data model；The third spectrum data, for as transaction
Behavioral data model；4th spectrum data, for being used as marketing relationship data model.
Further, it includes following any or a variety of for generating spectrum data: the knowledge mapping for semantic understanding
Using；Knowledge mapping application for intelligent search；For interacting the knowledge mapping application of question and answer；And for aid decision
Knowledge mapping application.
To achieve the goals above, according to the another aspect of the application, a kind of knowledge mapping construction device is provided.
Knowledge mapping construction device according to the application includes: increase module, for increasing by second into the first data source
Data source obtains target data source；AM access module, for accessing the target data source and handling the institute according to preset rules
State the data in target data source；And generation module, for generating spectrum data.
Further, the increase module comprises determining that unit, for determining the first object in first data source
Data, wherein the first object data are used to be used as internal data；Receiving unit, for receiving in second data source
The second target data, wherein second target data be used for be used as external data；And configuration unit, being used for will be described
First object data, second target data are configured to unified structure according to preset interface rule.
Further, the AM access module includes: access unit, for accessing the target data source and according to present count
Database model is established according to library model rule；And integral unit, it is used for according to preset data label in the database model
Data in the middle integration target data source.
Further, the AM access module includes: spectrum data determination unit, for according to the data in target data source
The scale of construction determines the first spectrum data；Spectrum data generation unit, for generating second by the data type in target data source
Spectrum data, third spectrum data and the 4th spectrum data；Judging unit, for judging first spectrum data, second
Whether spectrum data, third spectrum data and the 4th spectrum data meet business need when using response；Execution unit is used
In judge first spectrum data, the second spectrum data, third spectrum data and the 4th spectrum data using response when
When meeting business need, the default conversion operation from source terminal to destination is executed, wherein first spectrum data is used for
As basic attribute data model；Second spectrum data, for being used as social relationships data model；The third map number
According to for being used as trading activity data model；4th spectrum data, for being used as marketing relationship data model.
To achieve the goals above, according to the another aspect of the application, a kind of client is provided, including the knowledge
Map construction device.
In the embodiment of the present application, the side of target data source is obtained using increasing by the second data source into the first data source
Formula is reached by accessing the target data source and handling the data in the target data source according to preset rules
The purpose for expanding the spectrum data of application range is generated, to realize the technical effect for assisting financial sales service, and then is solved
It has determined the application technology problem lacked in terms of financial business.
Detailed description of the invention
The attached drawing constituted part of this application is used to provide further understanding of the present application, so that the application's is other
Feature, objects and advantages become more apparent upon.The illustrative examples attached drawing and its explanation of the application is for explaining the application, not
Constitute the improper restriction to the application.In the accompanying drawings:
Fig. 1 is the knowledge mapping schematic diagram of construction method according to the application first embodiment；
Fig. 2 is the knowledge mapping schematic diagram of construction method according to the application second embodiment；
Fig. 3 is the knowledge mapping schematic diagram of construction method according to the application 3rd embodiment；
Fig. 4 is the knowledge mapping schematic diagram of construction method according to the application fourth embodiment；
Fig. 5 is the knowledge mapping construction device schematic diagram according to the application first embodiment；
Fig. 6 is the knowledge mapping construction device schematic diagram according to the application second embodiment；
Fig. 7 is the knowledge mapping construction device schematic diagram according to the application 3rd embodiment；And
Fig. 8 is the knowledge mapping construction device schematic diagram according to the application fourth embodiment.
Specific embodiment
In order to make those skilled in the art more fully understand application scheme, below in conjunction in the embodiment of the present application
Attached drawing, the technical scheme in the embodiment of the application is clearly and completely described, it is clear that described embodiment is only
The embodiment of the application a part, instead of all the embodiments.Based on the embodiment in the application, ordinary skill people
Member's every other embodiment obtained without making creative work, all should belong to the model of the application protection
It encloses.
It should be noted that the description and claims of this application and term " first " in above-mentioned attached drawing, "
Two " etc. be to be used to distinguish similar objects, without being used to describe a particular order or precedence order.It should be understood that using in this way
Data be interchangeable under appropriate circumstances, so as to embodiments herein described herein.In addition, term " includes " and " tool
Have " and their any deformation, it is intended that cover it is non-exclusive include, for example, containing a series of steps or units
Process, method, system, product or equipment those of are not necessarily limited to be clearly listed step or unit, but may include without clear
Other step or units listing to Chu or intrinsic for these process, methods, product or equipment.
Knowledge mapping construction method in the embodiment of the present application is obtained by increasing by the second data source into the first data source
Target data source has been introduced from outside into the multi-party face data such as consumer consumption behavior, Social behaviors.Access the target data source simultaneously
The data in the target data source are handled according to preset rules, the quality of data is done to internal data source and external data source
It checks and data cleansing, data is associated with, are polymerize, are split, converts etc. after processing and generating generation spectrum data, spectrum data
It can support effective operation of knowledge mapping function.
It should be noted that in the absence of conflict, the features in the embodiments and the embodiments of the present application can phase
Mutually combination.The application is described in detail below with reference to the accompanying drawings and in conjunction with the embodiments.
As shown in Figure 1, this method includes the following steps, namely S102 to step S106:
Step S102 increases by the second data source into the first data source and obtains target data source；
It may include using data in first data source.
It is relevant using data to be primarily referred to as being related to business using data.For example, being related to the loan in financial application, financing
Or the application services such as dealing.
It may include daily record data in first data source.
Daily record data is primarily referred to as the relevant daily record data of record traffic.For example, the transaction being related in financial application produce,
It is transferred to the log recording of formality.
It may include file data in first data source.
File data is primarily referred to as being related to the relevant file data of business.For example, be related to the deal contract in financial application,
The file data of loan agreement, investment control.
Above-mentioned first data source mainly as the data of content source, while when into the first data source increase by second number
According to the range that can effectively expand business datum behind source.
It may include the external data for the first data source in second data source.
External data can be consumer consumption behavior data.
For example, the consumer record of user, spending amount, consumption place etc..
External data is also possible to Social behaviors data.
For example, user is issued by social network server, the content of concern, comment, reference.
For another example, user passes through the relationship of social network server and good friend.
Step S104 accesses the target data source and handles the number in the target data source according to preset rules
According to；
By relevant data access rule, the target data source is accessed.
Data access can be carried out by data buffering layer.
Data access can also be carried out by data post active layer.
Preferably, handling the data in the target data source according to preset rules can be
For example, handling the data in the target data source according to preset data quality cleaning rule.
Preferably, handling the data in the target data source according to preset rules can be
For example, handling the data in the target data source according to preset data integration rules.
Step S106 generates spectrum data.
Specifically, according to knowledge mapping function to the needs of data, according to the Data Model Designing of software itself, to data
It does ETL processing and is converted into the structure for being suitble to knowledge mapping function to need.It is set in the Data Model Designing and function for doing knowledge mapping
Timing has fully considered the complexity that the scale of construction of data, data use, it is ensured that function using when page response speed meet industry
Business requires.
ETL handles (full name in English Extract-Transform-Load), for describing data from source terminal by extracting
Take (extract), transposition (transform), the process for loading (load) to destination.
By building a whole set of for the spectrum data that knowledge mapping function uses, final support knowledge mapping function has
Effect operation.
It can be seen from the above description that the application realizes following technical effect:
In the embodiment of the present application, the side of target data source is obtained using increasing by the second data source into the first data source
Formula is reached by accessing the target data source and handling the data in the target data source according to preset rules
The purpose for expanding the spectrum data of application range is generated, to realize the technical effect for assisting financial sales service, and then is solved
It has determined the application technology problem lacked in terms of financial business.
According to the embodiment of the present application, as preferred in the present embodiment, as shown in Fig. 2, increasing by into the first data source
Two data sources obtain target data source
Step S202 determines the first object data in first data source,
The first object data are used to be used as internal data, can be using data, daily record data and file data
Etc. internal datas.
Step S204 receives the second target data in second data source,
Second target data is used to be used as external data；External data can be consumer consumption behavior data.It is external
Data are also possible to Social behaviors data.
The first object data, second target data are configured to unite by step S206 according to preset interface rule
One structure.
Specifically, by establishing buffer layer, after the data access of data source, first it is stored in buffer layer.According to different numbers
According to the transmission feature in source, combined data transmits the interface specification of increment or full dose, and arrangement is reduced into as prototype structure
Partial data is transferred to ODS layers.
Specifically, by establishing ODS layers, ODS layers i.e. patch active layer keeps same as data source or almost similarly ties
Structure guarantees that data information is undistorted to the greatest extent.
According to the embodiment of the present application, as preferred in the present embodiment, as shown in figure 3, accessing the target data source simultaneously
Handling the data in the target data source according to preset rules includes:
Step S302 accesses the target data source and establishes database model according to presetting database model rule；With
And
Firstly, also needing to carry out quality of data cleaning before executing step S302, for the number of targets in inside and outside data source
According to first in terms of the integrality of data, timeliness, legitimacy, uniqueness, consistency, accuracy are several, integrated survey is each
The quality of data of data is rejected as caused by a variety of causes such as mistake fills out, accidentally fill out, data modification, data storage logic transition
Distortion data forms a set of data basis that can be trusted.
On the basis of ensuring data correctly and accurately, according to the requirement of data model specification, build specification, easily extension,
The database model of standard, evades redundant data.
Step S304 integrates the number in the target data source according to preset data label in the database model
According to.
Standardized data library model is established in the database model according to preset data label, establishes norm structure
Data.
Specifically, preset data label may include:
Label A { client, transaction, user, payment, account, assets }；
Label B { product, contract, channel, activity, event }；
The data in the target data source are integrated in database model by different types of label A and label B.
In view of in software development process, business demand is most easily produced when changing and is reported an error, and establishes norm structure
Data can ensure that when data source changes, and change content is reduced to minimum, evades most of development function modification and produces
The raw risk that reports an error.
According to the embodiment of the present application, as preferred in the present embodiment, as shown in figure 4, accessing the target data source simultaneously
Handling the data in the target data source according to preset rules includes:
Step S402 determines the first spectrum data according to the data scale of construction in target data source；
First spectrum data, for as basic attribute data model.
The data scale of construction in target data source refers to the data volume size being capable of handling.Pass through accessible data volume size
Obtain primary attribute.
Step S404, by data type in target data source generate the second spectrum data, third spectrum data and
4th spectrum data；
Second spectrum data, for being used as social relationships data model；The third spectrum data, for as friendship
Easy is data model；4th spectrum data, for being used as marketing relationship data model.
When generating spectrum data by the data type in target data source, according to knowledge mapping function to the need of data
It wants, according to the Data Model Designing of software itself, ETL processing is done to data, be converted into the knot for being suitble to knowledge mapping function to need
Structure.
Step S406 judges first spectrum data, the second spectrum data, third spectrum data and the 4th map number
Whether meet business need according to when using response；
Whether meet business need when using response to refer to, it is ensured that WEB page response speed when function uses meets
Related service requirement.
Step S408, if it is determined that first spectrum data, the second spectrum data, third spectrum data and the 4th figure
Modal data meets business need when using response, then executes the default conversion operation from source terminal to destination.
Specifically, executing the default conversion operation from source terminal to destination can use ETL, ETL to handle (full name in English
Extract-Transform-Load), for describe by data from source terminal by extraction extract, transposition transform,
Load the process of load to destination.Pass through data such as relation data, flat data file will be distributed, in heterogeneous data source
It cleaned, converted, integrated etc. being drawn into after interim middle layer, be finally loaded into data warehouse or Data Mart, become connection
The basis of machine analysis processing, data mining.
Specifically, data pick-up is the process for extracting data from data source.In practical application, data source is relatively mostly used
Be relational database.It may include: full dose extraction, increment extraction.
In addition, the data source of ETL processing is other than relational database, it is also possible to file, such as txt file, excel text
Part, xml document etc..Extraction to file data is usually to carry out full dose extraction, the primary timestamp for extracting preceding storable file
Or the MD5 check code of calculation document, next time are compared when extracting, this extraction if the same can be ignored.
The method that delta data is commonly captured in incremental data extraction at present can be, trigger: in the table to be extracted
It is upper to establish the trigger needed, it generally to establish insertion, modification, delete three triggers, whenever the data in the table of source become
Change, the data of variation is just written by an interim table by corresponding trigger, extraction thread extracts data from interim table, temporarily
Decimated data are labeled in table or delete.Timestamp: increase a timestamp field on the table of source, update modification in system
When table data, while the value of modification time stamp field.When carrying out data pick-up, by comparing system time and timestamp
Which data is the value of field determine to extract.Full table compares: the mode that typical full table compares is using MD5 check code.Log
Comparison: the data of variation are judged by the log of analytical database itself.
Further, specifically, data conversion and processing in ETL processing, from referring to the data extracted in data source
Not necessarily fully meet the requirement in purpose library, such as the inconsistent of data format, data entry error, data are imperfect etc.,
It is therefore desirable to carry out data conversion and processing to the data extracted.The conversion and processing of data can in ETL engine into
Row, can also be carried out in data extraction process using the characteristic of relational database simultaneously.Pass through the data conversion in ETL engine
And processing, data conversion is generally realized in a manner of modularization in ETL engine.Common data transformation components have field mapping,
Data filtering, data cleansing, data replacement, data calculating, data verification, data encrypting and deciphering, data merging, data fractionation etc..
Further, specifically, the data in ETL processing load, the data after conversion and processing are loaded into purpose
It is usually the final step of ETL process in library.The best approach of loading data depends on the type and needs of performed operation
How many data be packed into.When purpose library is relational database, load mode may is that direct SQL statement carry out insert,
Update, delete operation；Load mode is also possible to: using batch stowage, the distinctive batch of relational database loads
Tool or API.
It is handled by ETL, it can be ensured that daily data processing is various promptly and accurately to be completed.Those skilled in the art's energy
It is enough to be illustrated, it can be using including but not limited to above-mentioned ETL processing mode to realize the various energy of daily data processing in time
The technical effect accurately completed.
As preferred in the present embodiment, the knowledge mapping construction method of the embodiment of the present application, comprising: to the first data source
The second data source of middle increase obtains target data source；It accesses the target data source and handles the mesh according to preset rules
Mark the data in data source；And generate spectrum data.
Increase by the second data source into the first data source and obtain target data source and includes:
Determine the first object data in first data source, wherein the first object data are used for as internal
Data；
Receive the second target data in second data source, wherein second target data is used for as external
Data；And
Unified structure is configured according to preset interface rule by the first object data, second target data.
It accesses the target data source and handles the data in the target data source according to preset rules and include:
It accesses the target data source and establishes database model according to presetting database model rule；And
The data in the target data source are integrated in the database model according to preset data label.
It accesses the target data source and handles the data in the target data source according to preset rules and include:
The first spectrum data is determined according to the data scale of construction in target data source；
The second spectrum data, third spectrum data and the 4th map number are generated by the data type in target data source
According to；
Judge that first spectrum data, the second spectrum data, third spectrum data and the 4th spectrum data are using
Whether meet business need when response；
If it is determined that first spectrum data, the second spectrum data, third spectrum data and the 4th spectrum data exist
Meet business need when using response, then execute the default conversion operation from source terminal to destination,
Wherein, first spectrum data, for as basic attribute data model；Second spectrum data, is used for
As social relationships data model；The third spectrum data, for being used as trading activity data model；The 4th map number
According to for being used as marketing relationship data model.
It includes following any or a variety of for generating spectrum data: the knowledge mapping application for semantic understanding；For
The knowledge mapping application of intelligent search；For interacting the knowledge mapping application of question and answer；And the knowledge mapping for aid decision
Using.
Specifically, in order to more comprehensively understand user behavior information, providing more has business in embodiments herein
The knowledge mapping function of help has been introduced from outside into the multi-party face data such as user's vehicle, house property, Social behaviors, has formd user
Comprehensive knowledge mapping system.By taking Social behaviors as an example, by introducing external data, the good friend for obtaining user in social network sites is closed
It is, relation datas and the behavioral data such as article, the article of reading, the comment of participation, the picture of upload or the document information delivered,
Natural language processing technique is introduced, the social relationships net of user is identified, interprets the attribute informations such as education background, the occupation of user,
Judge the consumer expectations such as shopping, the tourism of user, to form a set of data system for supporting Marketing plan, assists operation department
The formulation and adjustment of marketing strategy, marketing strategy assist the selection and advertisement of target customer in marketing activity to launch, promote marketing
Movable business output, to bring more profits for company.
It should be noted that step shown in the flowchart of the accompanying drawings can be in such as a group of computer-executable instructions
It is executed in computer system, although also, logical order is shown in flow charts, and it in some cases, can be with not
The sequence being same as herein executes shown or described step.
According to the embodiment of the present application, additionally provide a kind of for implementing the device of above-mentioned knowledge mapping construction method, such as Fig. 5
Shown, which includes: to increase module 10, obtains target data source for increasing by the second data source into the first data source；It connects
Enter module 20, for accessing the target data source and handling the data in the target data source according to preset rules；
And generation module 30, for generating spectrum data.In embodiments herein, by introducing wider data source,
Give full play to that Volume data volume in the big data ecosphere in 4v is big and Variety wide variety two major features, and in data
It does a lot of work in accuracy, it is ensured that the accuracy and timeliness of data.
It may include using data in the first data source in the increase module 10 of the embodiment of the present application.
It is relevant using data to be primarily referred to as being related to business using data.For example, being related to the loan in financial application, financing
Or the application services such as dealing.
It may include daily record data in first data source.
Daily record data is primarily referred to as the relevant daily record data of record traffic.For example, the transaction being related in financial application produce,
It is transferred to the log recording of formality.
It may include file data in first data source.
File data is primarily referred to as being related to the relevant file data of business.For example, be related to the deal contract in financial application,
The file data of loan agreement, investment control.
Above-mentioned first data source mainly as the data of content source, while when into the first data source increase by second number
According to the range that can effectively expand business datum behind source.
It may include the external data for the first data source in second data source.
External data can be consumer consumption behavior data.
For example, the consumer record of user, spending amount, consumption place etc..
External data is also possible to Social behaviors data.
For example, user is issued by social network server, the content of concern, comment, reference.
For another example, user passes through the relationship of social network server and good friend.
By relevant data access rule in the AM access module 20 of the embodiment of the present application, the target data source is accessed.
Data access can be carried out by data buffering layer.
Data access can also be carried out by data post active layer.
Preferably, handling the data in the target data source according to preset rules can be
For example, handling the data in the target data source according to preset data quality cleaning rule.
Preferably, handling the data in the target data source according to preset rules can be
For example, handling the data in the target data source according to preset data integration rules.
In the generation module 30 of the embodiment of the present application specifically, according to knowledge mapping function to the needs of data, according to soft
The Data Model Designing of part itself does ETL processing to data and is converted into the structure for being suitble to knowledge mapping function to need.Doing knowledge
When the Data Model Designing and Functional Design of map, the complexity that the scale of construction of data, data use has been fully considered, it is ensured that function
Page response speed when use meets business need.
ETL handles (full name in English Extract-Transform-Load), for describing data from source terminal by extracting
Take (extract), transposition (transform), the process for loading (load) to destination.
By building a whole set of for the spectrum data that knowledge mapping function uses, final support knowledge mapping function has
Effect operation.
According to the embodiment of the present application, as preferred in the present embodiment, as shown in fig. 6, the increase module comprises determining that
Unit 101, for determining the first object data in first data source, wherein the first object data are used for conduct
Internal data；Receiving unit 102, for receiving the second target data in second data source, wherein second target
Data are used to be used as external data；And configuration unit 103, it is used for the first object data, second target data
Unified structure is configured to according to preset interface rule.
First object data described in the determination unit 101 of the embodiment of the present application are used to be used as internal data, can be and answer
With internal datas such as data, daily record data and file datas.
Second target data described in the receiving unit 102 of the embodiment of the present application is used to be used as external data；External data
It can be consumer consumption behavior data.External data is also possible to Social behaviors data.
In the configuration unit 103 of the embodiment of the present application specifically, by establishing buffer layer, the data access of data source it
Afterwards, first it is stored in buffer layer.According to the transmission feature of different data sources, combined data transmits the interface rule of increment or full dose
Model arranges the partial data being reduced into as prototype structure, is transferred to ODS layers.
Specifically, by establishing ODS layers, ODS layers i.e. patch active layer keeps same as data source or almost similarly ties
Structure guarantees that data information is undistorted to the greatest extent.
According to the embodiment of the present application, as preferred in the present embodiment, as shown in fig. 7, the AM access module 20 includes: to connect
Enter unit 201, for accessing the target data source and establishing database model according to presetting database model rule；And it is whole
Unit 202 is closed, for integrating the data in the target data source in the database model according to preset data label.
It needs to carry out quality of data cleaning in the access unit 201 of the embodiment of the present application, for the mesh in inside and outside data source
Mark data, first in terms of the integrality of data, timeliness, legitimacy, uniqueness, consistency, accuracy are several, integrated survey
The quality of data of each data is rejected since a variety of causes such as mistake fills out, accidentally fill out, data modification, data storage logic transition are led
The distortion data of cause forms a set of data basis that can be trusted.
On the basis of ensuring data correctly and accurately, according to the requirement of data model specification, build specification, easily extension,
The database model of standard, evades redundant data.
Standard is established in the database model according to preset data label in the integral unit 202 of the embodiment of the present application
Change database model, establishes the data of norm structure.
Specifically, preset data label may include:
Label A { client, transaction, user, payment, account, assets }；
Label B { product, contract, channel, activity, event }；
The data in the target data source are integrated in database model by different types of label A and label B.
In view of in software development process, business demand is most easily produced when changing and is reported an error, and establishes norm structure
Data can ensure that when data source changes, and change content is reduced to minimum, evades most of development function modification and produces
The raw risk that reports an error.
According to the embodiment of the present application, as preferred in the present embodiment, as shown in figure 8, the AM access module 20 includes: figure
Modal data determination unit 203, for determining the first spectrum data according to the data scale of construction in target data source；Spectrum data generates
Unit 204, for generating the second spectrum data, third spectrum data and the 4th figure by the data type in target data source
Modal data；Judging unit 205, for judging first spectrum data, the second spectrum data, third spectrum data and the 4th
Whether spectrum data meets business need when using response；Execution unit 206, for judging first spectrum data,
When two spectrum datas, third spectrum data and the 4th spectrum data meet business need when using response, execute from source
It holds to the default conversion operation of destination, first spectrum data, for as basic attribute data model；Second figure
Modal data, for being used as social relationships data model；The third spectrum data, for being used as trading activity data model；Institute
The 4th spectrum data is stated, for being used as marketing relationship data model.
First spectrum data described in the spectrum data determination unit 203 of the embodiment of the present application, for as basic attribute
Data model.
The data scale of construction in target data source refers to the data volume size being capable of handling.Pass through accessible data volume size
Obtain primary attribute.
Second spectrum data described in the spectrum data generation unit 204 of the embodiment of the present application, for being used as social relationships
Data model；The third spectrum data, for being used as trading activity data model；4th spectrum data is used for conduct
Marketing relationship data model.
When generating spectrum data by the data type in target data source, according to knowledge mapping function to the need of data
It wants, according to the Data Model Designing of software itself, ETL processing is done to data, be converted into the knot for being suitble to knowledge mapping function to need
Structure.
Whether meet business need when using response in the judging unit 205 of the embodiment of the present application to refer to, it is ensured that function
WEB page response speed when use meets related service requirement.
In the execution unit 206 of the embodiment of the present application specifically, the default conversion operation from source terminal to destination is executed
ETL, ETL can be used to handle (full name in English Extract-Transform-Load), for describing to pass through data from source terminal
Cross the process of extraction extract, transposition transform, load load to destination.By it is will being distributed, in heterogeneous data source
Data such as cleaned after relation data, flat data file are drawn into interim middle layer, convert, integrate, finally load
Into data warehouse or Data Mart, become the basis of on-line analytical processing, data mining.
Specifically, data pick-up is the process for extracting data from data source.In practical application, data source is relatively mostly used
Be relational database.It may include: full dose extraction, increment extraction.
In addition, the data source of ETL processing is other than relational database, it is also possible to file, such as txt file, excel text
Part, xml document etc..Extraction to file data is usually to carry out full dose extraction, the primary timestamp for extracting preceding storable file
Or the MD5 check code of calculation document, next time are compared when extracting, this extraction if the same can be ignored.
The method that delta data is commonly captured in incremental data extraction at present can be, trigger: in the table to be extracted
It is upper to establish the trigger needed, it generally to establish insertion, modification, delete three triggers, whenever the data in the table of source become
Change, the data of variation is just written by an interim table by corresponding trigger, extraction thread extracts data from interim table, temporarily
Decimated data are labeled in table or delete.Timestamp: increase a timestamp field on the table of source, update modification in system
When table data, while the value of modification time stamp field.When carrying out data pick-up, by comparing system time and timestamp
Which data is the value of field determine to extract.Full table compares: the mode that typical full table compares is using MD5 check code.Log
Comparison: the data of variation are judged by the log of analytical database itself.
Further, specifically, data conversion and processing in ETL processing, from referring to the data extracted in data source
Not necessarily fully meet the requirement in purpose library, such as the inconsistent of data format, data entry error, data are imperfect etc.,
It is therefore desirable to carry out data conversion and processing to the data extracted.The conversion and processing of data can in ETL engine into
Row, can also be carried out in data extraction process using the characteristic of relational database simultaneously.Pass through the data conversion in ETL engine
And processing, data conversion is generally realized in a manner of modularization in ETL engine.Common data transformation components have field mapping,
Data filtering, data cleansing, data replacement, data calculating, data verification, data encrypting and deciphering, data merging, data fractionation etc..
Further, specifically, the data in ETL processing load, the data after conversion and processing are loaded into purpose
It is usually the final step of ETL process in library.The best approach of loading data depends on the type and needs of performed operation
How many data be packed into.When purpose library is relational database, load mode may is that direct SQL statement carry out insert,
Update, delete operation；Load mode is also possible to: using batch stowage, the distinctive batch of relational database loads
Tool or API.
It is handled by ETL, it can be ensured that daily data processing is various promptly and accurately to be completed.Those skilled in the art's energy
It is enough to be illustrated, it can be using including but not limited to above-mentioned ETL processing mode to realize the various energy of daily data processing in time
The technical effect accurately completed.
As preferred in the present embodiment, the knowledge mapping construction method of the embodiment of the present application, comprising: to the first data source
The second data source of middle increase obtains target data source；It accesses the target data source and handles the mesh according to preset rules
Mark the data in data source；And generate spectrum data.
Increase by the second data source into the first data source and obtain target data source and includes:
Determine the first object data in first data source, wherein the first object data are used for as internal
Data；
Receive the second target data in second data source, wherein second target data is used for as external
Data；And
Unified structure is configured according to preset interface rule by the first object data, second target data.
It accesses the target data source and handles the data in the target data source according to preset rules and include:
It accesses the target data source and establishes database model according to presetting database model rule；And
The data in the target data source are integrated in the database model according to preset data label.
It accesses the target data source and handles the data in the target data source according to preset rules and include:
The first spectrum data is determined according to the data scale of construction in target data source；
The second spectrum data, third spectrum data and the 4th map number are generated by the data type in target data source
According to；
Judge that first spectrum data, the second spectrum data, third spectrum data and the 4th spectrum data are using
Whether meet business need when response；
If it is determined that first spectrum data, the second spectrum data, third spectrum data and the 4th spectrum data exist
Meet business need when using response, then execute the default conversion operation from source terminal to destination,
Wherein, first spectrum data, for as basic attribute data model；Second spectrum data, is used for
As social relationships data model；The third spectrum data, for being used as trading activity data model；The 4th map number
According to for being used as marketing relationship data model.
It includes following any or a variety of for generating spectrum data: the knowledge mapping application for semantic understanding；For
The knowledge mapping application of intelligent search；For interacting the knowledge mapping application of question and answer；And the knowledge mapping for aid decision
Using.
Specifically, in order to more comprehensively understand user behavior information, providing more has business in embodiments herein
The knowledge mapping function of help has been introduced from outside into the multi-party face data such as user's vehicle, house property, Social behaviors, has formd user
Comprehensive knowledge mapping system.By taking Social behaviors as an example, by introducing external data, the good friend for obtaining user in social network sites is closed
It is, relation datas and the behavioral data such as article, the article of reading, the comment of participation, the picture of upload or the document information delivered,
Natural language processing technique is introduced, the social relationships net of user is identified, interprets the attribute informations such as education background, the occupation of user,
Judge the consumer expectations such as shopping, the tourism of user, to form a set of data system for supporting Marketing plan, assists operation department
The formulation and adjustment of marketing strategy, marketing strategy assist the selection and advertisement of target customer in marketing activity to launch, promote marketing
Movable business output, to bring more profits for company.
Preferably, it includes following any or a variety of that spectrum data is generated in the generation module: for semantic reason
The knowledge mapping application of solution；Knowledge mapping application for intelligent search；For interacting the knowledge mapping application of question and answer；And it uses
In the knowledge mapping application of aid decision.
According to the embodiment of the present application, a kind of client is additionally provided, including the knowledge mapping construction device.It is described to know
The implementing principle and technical effect of map construction device are known as described above, no longer repeated herein.
Specifically, the client in embodiments herein, in order to more comprehensively understand user behavior information, provide pair
The more helpful knowledge mapping function of business, has been introduced from outside into the multi-party face data such as user's vehicle, house property, Social behaviors, shape
At the comprehensive knowledge mapping system of user.By taking Social behaviors as an example, by introducing external data, user is obtained in social network sites
Friend relation, the relation datas such as article, the article of reading, the comment of participation, the picture of upload or the document information delivered and
Behavioral data introduces natural language processing technique, identifies the social relationships net of user, interpret education background, occupation of user etc.
Attribute information judges the consumer expectations such as shopping, the tourism of user, so that a set of data system for supporting Marketing plan is formed, it is auxiliary
The formulation and adjustment of auxiliary operation department marketing strategy, marketing strategy assist the selection and advertisement of target customer in marketing activity to throw
It puts, promotes the business output of marketing activity, to bring more profits for financing corporation.
Obviously, those skilled in the art should be understood that each module of above-mentioned the application or each step can be with general
Computing device realize that they can be concentrated on a single computing device, or be distributed in multiple computing devices and formed
Network on, optionally, they can be realized with the program code that computing device can perform, it is thus possible to which they are stored
Be performed by computing device in the storage device, perhaps they are fabricated to each integrated circuit modules or by they
In multiple modules or step be fabricated to single integrated circuit module to realize.In this way, the application be not limited to it is any specific
Hardware and software combines.
The foregoing is merely preferred embodiment of the present application, are not intended to limit this application, for the skill of this field
For art personnel, various changes and changes are possible in this application.Within the spirit and principles of this application, made any to repair
Change, equivalent replacement, improvement etc., should be included within the scope of protection of this application.
Claims (10)
1. a kind of knowledge mapping construction method characterized by comprising
Increase by the second data source into the first data source and obtains target data source；
It accesses the target data source and handles the data in the target data source according to preset rules；And
Generate spectrum data.
2. knowledge mapping construction method according to claim 1, which is characterized in that increase by the second number into the first data source
Obtaining target data source according to source includes:
Determine the first object data in first data source, wherein the first object data are used to be used as internal data；
Receive the second target data in second data source, wherein second target data is used to be used as external data；
And
Unified structure is configured according to preset interface rule by the first object data, second target data.
3. knowledge mapping construction method according to claim 1, which is characterized in that access the target data source and according to
The data that preset rules are handled in the target data source include:
It accesses the target data source and establishes database model according to presetting database model rule；And
The data in the target data source are integrated in the database model according to preset data label.
4. knowledge mapping construction method according to claim 1, which is characterized in that access the target data source and according to
The data that preset rules are handled in the target data source include:
The first spectrum data is determined according to the data scale of construction in target data source；
The second spectrum data, third spectrum data and the 4th spectrum data are generated by the data type in target data source；
Judge first spectrum data, the second spectrum data, third spectrum data and the 4th spectrum data using response
When whether meet business need；
If it is determined that first spectrum data, the second spectrum data, third spectrum data and the 4th spectrum data are using
Meet business need when response, then execute the default conversion operation from source terminal to destination,
Wherein,
First spectrum data, for as basic attribute data model；
Second spectrum data, for being used as social relationships data model；
The third spectrum data, for being used as trading activity data model；
4th spectrum data, for being used as marketing relationship data model.
5. knowledge mapping construction method according to claim 1, which is characterized in that generating spectrum data includes following appoint
One is one or more:
Knowledge mapping application for semantic understanding；
Knowledge mapping application for intelligent search；
For interacting the knowledge mapping application of question and answer；And
Knowledge mapping application for aid decision.
6. a kind of knowledge mapping construction device characterized by comprising
Increase module, obtains target data source for increasing by the second data source into the first data source；
AM access module, for accessing the target data source and handling the number in the target data source according to preset rules
According to；And
Generation module, for generating spectrum data.
7. knowledge mapping construction device according to claim 6, which is characterized in that the increase module includes:
Determination unit, for determining the first object data in first data source, wherein the first object data are used for
As internal data；
Receiving unit, for receiving the second target data in second data source, wherein second target data is used for
As external data；And
Configuration unit, for the first object data, second target data to be configured to unite according to preset interface rule
One structure.
8. knowledge mapping construction device according to claim 6, which is characterized in that the AM access module includes:
Access unit, for accessing the target data source and establishing database model according to presetting database model rule；With
And
Integral unit, for integrating the number in the target data source in the database model according to preset data label
According to.
9. knowledge mapping construction device according to claim 6, which is characterized in that the AM access module includes:
Spectrum data determination unit, for determining the first spectrum data according to the data scale of construction in target data source；
Spectrum data generation unit, for generating the second spectrum data, third map by the data type in target data source
Data and the 4th spectrum data；
Judging unit, for judging first spectrum data, the second spectrum data, third spectrum data and the 4th map number
Whether meet business need according to when using response；
Execution unit, for judging first spectrum data, the second spectrum data, third spectrum data and the 4th map number
When according to meeting business need when using response, the default conversion operation from source terminal to destination is executed,
Wherein,
First spectrum data, for as basic attribute data model；
Second spectrum data, for being used as social relationships data model；
The third spectrum data, for being used as trading activity data model；
4th spectrum data, for being used as marketing relationship data model.
10. a kind of client, which is characterized in that including the described in any item knowledge mapping construction devices of such as claim 6 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810603636.3A CN109002470A (en) | 2018-06-12 | 2018-06-12 | Knowledge mapping construction method and device, client |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810603636.3A CN109002470A (en) | 2018-06-12 | 2018-06-12 | Knowledge mapping construction method and device, client |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN109002470A true CN109002470A (en) | 2018-12-14 |
Family
ID=64601233
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810603636.3A Pending CN109002470A (en) | 2018-06-12 | 2018-06-12 | Knowledge mapping construction method and device, client |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109002470A (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110232524A (en) * | 2019-06-14 | 2019-09-13 | 哈尔滨哈银消费金融有限责任公司 | Social networks cheats the construction method of model, antifraud method and apparatus |
| WO2020134213A1 (en) * | 2018-12-25 | 2020-07-02 | 苏宁云计算有限公司 | Method and system for querying abnormal financial data on basis of knowledge map |
| CN111770236A (en) * | 2020-02-13 | 2020-10-13 | 北京沃东天骏信息技术有限公司 | Conversation processing method, device, system, server and storage medium |
| CN112632361A (en) * | 2020-12-29 | 2021-04-09 | 中科院计算技术研究所大数据研究院 | Iterative data acquisition method |
| CN114168608A (en) * | 2021-12-16 | 2022-03-11 | 中科雨辰科技有限公司 | Data processing system for updating knowledge graph |
Citations (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105183869A (en) * | 2015-09-16 | 2015-12-23 | 分众(中国)信息技术有限公司 | Building knowledge mapping database and construction method thereof |
| CN105930540A (en) * | 2016-03-23 | 2016-09-07 | 四川长虹电器股份有限公司 | Data processing system |
| CN106127521A (en) * | 2016-03-23 | 2016-11-16 | 四川长虹电器股份有限公司 | A kind of information processing method and data handling system |
| CN106548381A (en) * | 2016-12-19 | 2017-03-29 | 武汉理工数字传播工程有限公司 | Intelligent subscriber tag systems and implementation method |
| CN106777331A (en) * | 2017-01-11 | 2017-05-31 | 北京航空航天大学 | Knowledge mapping generation method and device |
| CN106897403A (en) * | 2017-02-14 | 2017-06-27 | 中国科学院电子学研究所 | Towards the fine granularity Chinese attribute alignment schemes that knowledge mapping builds |
| CN106909662A (en) * | 2017-02-27 | 2017-06-30 | 腾讯科技（上海）有限公司 | Knowledge mapping construction method and device |
| CN107038257A (en) * | 2017-05-10 | 2017-08-11 | 浙江大学 | A kind of city Internet of Things data analytical framework of knowledge based collection of illustrative plates |
| CN107330007A (en) * | 2017-06-12 | 2017-11-07 | 南京邮电大学 | A kind of Method for Ontology Learning based on multi-data source |
| CN107358315A (en) * | 2017-06-26 | 2017-11-17 | 深圳市金立通信设备有限公司 | A kind of information forecasting method and terminal |
| CN107729490A (en) * | 2017-10-18 | 2018-02-23 | 华南农业大学 | Clothes personalized designs method and apparatus based on multi-source dynamic knowledge collection of illustrative plates |
| CN107944898A (en) * | 2016-10-13 | 2018-04-20 | 驰众信息技术（上海）有限公司 | The automatic discovery of advertisement putting building information and sort method |
| CN108021718A (en) * | 2017-12-29 | 2018-05-11 | 中国电子科技集团公司信息科学研究院 | Capability of Internet of things knowledge mapping and its construction method |
-
2018
- 2018-06-12 CN CN201810603636.3A patent/CN109002470A/en active Pending
Patent Citations (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105183869A (en) * | 2015-09-16 | 2015-12-23 | 分众(中国)信息技术有限公司 | Building knowledge mapping database and construction method thereof |
| CN105930540A (en) * | 2016-03-23 | 2016-09-07 | 四川长虹电器股份有限公司 | Data processing system |
| CN106127521A (en) * | 2016-03-23 | 2016-11-16 | 四川长虹电器股份有限公司 | A kind of information processing method and data handling system |
| CN107944898A (en) * | 2016-10-13 | 2018-04-20 | 驰众信息技术（上海）有限公司 | The automatic discovery of advertisement putting building information and sort method |
| CN106548381A (en) * | 2016-12-19 | 2017-03-29 | 武汉理工数字传播工程有限公司 | Intelligent subscriber tag systems and implementation method |
| CN106777331A (en) * | 2017-01-11 | 2017-05-31 | 北京航空航天大学 | Knowledge mapping generation method and device |
| CN106897403A (en) * | 2017-02-14 | 2017-06-27 | 中国科学院电子学研究所 | Towards the fine granularity Chinese attribute alignment schemes that knowledge mapping builds |
| CN106909662A (en) * | 2017-02-27 | 2017-06-30 | 腾讯科技（上海）有限公司 | Knowledge mapping construction method and device |
| CN107038257A (en) * | 2017-05-10 | 2017-08-11 | 浙江大学 | A kind of city Internet of Things data analytical framework of knowledge based collection of illustrative plates |
| CN107330007A (en) * | 2017-06-12 | 2017-11-07 | 南京邮电大学 | A kind of Method for Ontology Learning based on multi-data source |
| CN107358315A (en) * | 2017-06-26 | 2017-11-17 | 深圳市金立通信设备有限公司 | A kind of information forecasting method and terminal |
| CN107729490A (en) * | 2017-10-18 | 2018-02-23 | 华南农业大学 | Clothes personalized designs method and apparatus based on multi-source dynamic knowledge collection of illustrative plates |
| CN108021718A (en) * | 2017-12-29 | 2018-05-11 | 中国电子科技集团公司信息科学研究院 | Capability of Internet of things knowledge mapping and its construction method |
Non-Patent Citations (2)
| Title |
|---|
| 崔立真等: "《大数据驱动创新方法工作平台》", 31 October 2017 * |
| 田萍芳: "《面向云出版的语义关键技术》", 30 April 2015 * |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2020134213A1 (en) * | 2018-12-25 | 2020-07-02 | 苏宁云计算有限公司 | Method and system for querying abnormal financial data on basis of knowledge map |
| CN110232524A (en) * | 2019-06-14 | 2019-09-13 | 哈尔滨哈银消费金融有限责任公司 | Social networks cheats the construction method of model, antifraud method and apparatus |
| CN111770236A (en) * | 2020-02-13 | 2020-10-13 | 北京沃东天骏信息技术有限公司 | Conversation processing method, device, system, server and storage medium |
| CN112632361A (en) * | 2020-12-29 | 2021-04-09 | 中科院计算技术研究所大数据研究院 | Iterative data acquisition method |
| CN112632361B (en) * | 2020-12-29 | 2021-10-29 | 中科院计算技术研究所大数据研究院 | Iterative data acquisition method |
| CN114168608A (en) * | 2021-12-16 | 2022-03-11 | 中科雨辰科技有限公司 | Data processing system for updating knowledge graph |
| CN114168608B (en) * | 2021-12-16 | 2022-07-15 | 中科雨辰科技有限公司 | Data processing system for updating knowledge graph |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11875400B2 (en) | Systems, methods, and apparatuses for dynamically assigning nodes to a group within blockchains based on transaction type and node intelligence using distributed ledger technology (DLT) | |
| US11809458B2 (en) | System and method for providing database abstraction and data linkage | |
| CN109002470A (en) | Knowledge mapping construction method and device, client | |
| Padmanabhan et al. | An empirical analysis of the value of complete information for eCRM models | |
| CN110781236A (en) | Method for constructing government affair big data management system | |
| CN109947998A (en) | The calculating data lineage of network across heterogeneous system | |
| US20130080197A1 (en) | Evaluating a trust value of a data report from a data processing tool | |
| O'Leary | Some issues in blockchain for accounting and the supply chain, with an application of distributed databases to virtual organizations | |
| US20080208661A1 (en) | Method and system of using anrtifacts to identify elements of a component business model | |
| US8892545B2 (en) | Generating a compiler infrastructure | |
| CN110929039A (en) | Data processing method, device, equipment and storage medium | |
| US11762920B2 (en) | Composite index on hierarchical nodes in the hierarchical data model within a case model | |
| CN107103064A (en) | Data statistical approach and device | |
| CN115129879A (en) | Method for constructing enterprise relational knowledge base based on knowledge graph | |
| CN104346371A (en) | Business integration system report driven by in-memory database | |
| US9652740B2 (en) | Fan identity data integration and unification | |
| CN102222060A (en) | Receipt conversion device and receipt conversion method | |
| Faisal et al. | Handling slowly changing dimensions in data warehouses | |
| US20120124110A1 (en) | Database, management server, and management program | |
| KR102184048B1 (en) | System and method for checking of information about estate development plan based on geographic information system | |
| Volk et al. | Ask the Right Questions: Requirements Engineering for the Execution of Big Data Projects. | |
| Sadath | Data mining in E-commerce: a CRM platform | |
| Härer | Towards Interoperability of Open and Permissionless Blockchains: A Cross-Chain Query Language | |
| Pytel et al. | ENABLING UTXO-BASED BACKWARDS AND FORWARDS TRACEABILTY | |
| CN110796425A (en) | Intellectual property transfer and conversion management method, system and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20181214 |
|
| RJ01 | Rejection of invention patent application after publication |