CN106909570A - A kind of data transfer device and device - Google Patents

A kind of data transfer device and device Download PDF

Info

Publication number
CN106909570A
CN106909570A CN201510981357.7A CN201510981357A CN106909570A CN 106909570 A CN106909570 A CN 106909570A CN 201510981357 A CN201510981357 A CN 201510981357A CN 106909570 A CN106909570 A CN 106909570A
Authority
CN
China
Prior art keywords
data
source data
field
conversion
information
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201510981357.7A
Other languages
Chinese (zh)
Other versions
CN106909570B (en
Inventor
宋忠平
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Alipay Hangzhou Information Technology Co Ltd
Original Assignee
Alibaba Group Holding Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Alibaba Group Holding Ltd filed Critical Alibaba Group Holding Ltd
Priority to CN201510981357.7A priority Critical patent/CN106909570B/en
Publication of CN106909570A publication Critical patent/CN106909570A/en
Application granted granted Critical
Publication of CN106909570B publication Critical patent/CN106909570B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/25Integrating or interfacing systems involving database management systems
    • G06F16/258Data format conversion from or to a database

Abstract

This application discloses a kind of data transfer device and device.The data transfer device is used to that the target data using unified demand data model will to be converted to using the source data of different pieces of information model, and the method includes:Obtain source data to be converted;The conversion markup information of the source data corresponding data model that reading prestores, at least includes in the conversion markup information:Need the source data word segment information for retaining;According to the conversion markup information for being read, the source data field for needing to retain is extracted from the source data, obtain target data.Technical scheme not only reduces the code quantity of realizing of conversion logic compared with prior art, and scattered many sets of data conversion logics originally are concentrated in together, so as to effectively reduce development and maintenance cost.

Description

A kind of data transfer device and device
Technical field
The application is related to technical field of data administration, more particularly to a kind of data transfer device and device.
Background technology
Data management refers to the process of that various data are collected, stored, processed and applied, in big data Epoch, in order to tackle the data volume of notable growth, in addition to needing to provide sufficient memory space, logarithm Requirement higher is it is also proposed according to administrative skill.
For data storage side or data user, generally require to obtain data from outside data source, And when the data model that external data source is provided is inconsistent with actually used required data model, just need Will according to the actual requirements to carrying out conversion process from the outside data for obtaining, the content of conversion is potentially included to word Choice, the change of field name, change of field type of section etc..
In practical application, the data model acquired in different external data sources be it is omnifarious, according to The implementation of prior art, in order to meet unified data storage and use demand, it is necessary to be directed to every kind of outer Portion's data model writes conversion logic respectively., it is necessary to put into substantial amounts of in the case of data source is a fairly large number of Exploitation and maintenance cost, and in many application scenarios, the quantity of data source be ever-increasing, this Also so that the exploitation of entirety and maintenance cost are difficult to control to.
The content of the invention
For above-mentioned technical problem, the application provides a kind of data transfer device and device, and technical scheme is as follows:
A kind of data transfer device, is needed for will be converted to using the source data of different pieces of information model using unified Asking the target data of data model, the method includes:
Obtain source data to be converted;
The conversion markup information of the source data corresponding data model that reading prestores, the conversion markup information In at least include:Need the source data word segment information for retaining;
According to the conversion markup information for being read, the source data field for needing to retain is extracted from the source data, Obtain target data.
A kind of DTU, is needed for will be converted to using the source data of different pieces of information model using unified The target data of data model is sought, the device includes:
Source data obtains module, for obtaining source data to be converted;
Markup information read module, the conversion mark for reading the source data corresponding data model for prestoring Information, at least includes in the conversion markup information:Need the source data word segment information for retaining;
Target data generation module, for according to the conversion markup information for being read, being carried from the source data The source data field for needing to retain is taken, target data is obtained.
Technical scheme provided herein, is not by way of being labeled to source data model in advance Same source data model configuration data conversion requirements, when needing to change source data, only need to call one Set logic that is general, being capable of identify that above-mentioned data conversion demand, you can realize to using different pieces of information model Source data unify conversion.Compared with prior art, the realization generation of conversion logic is not only reduced Number of codes, and scattered many sets of data conversion logics originally are concentrated in together, so as to effectively reduce out Hair and maintenance cost.
It should be appreciated that the general description of the above and detailed description hereinafter are only exemplary and explanatory, The application can not be limited.
Brief description of the drawings
In order to illustrate more clearly of the embodiment of the present application or technical scheme of the prior art, below will be to implementing Example or the accompanying drawing to be used needed for description of the prior art are briefly described, it should be apparent that, describe below In accompanying drawing be only some embodiments described in the application, for those of ordinary skill in the art, Other accompanying drawings can also be obtained according to these accompanying drawings.
Fig. 1 is the schematic flow sheet of the data transfer device of the application;
Fig. 2 is the first structural representation of the DTU of the application;
Fig. 3 is second structural representation of the DTU of the application.
Specific embodiment
In order that those skilled in the art more fully understand the technical scheme in the application, below in conjunction with this Shen Accompanying drawing that please be in embodiment, is described in detail, it is clear that institute to the technical scheme in the embodiment of the present application The embodiment of description is only some embodiments of the present application, rather than whole embodiments.Based on the application In embodiment, the every other embodiment that those of ordinary skill in the art are obtained should all belong to this Shen The scope that please be protect.
Being brought for conversion logic is developed respectively using various data models in order to solve the problems, such as to be directed to, this Shen The scheme that please be provided is:According to application demand, the field to various source data models is labeled in advance, marks Note content can include:Need to retain which field, whether need to change field name, if need to change Field type, etc..When needing to change source data, a set of general, Neng Goushi only need to be called The logic of not above-mentioned data conversion demand, you can realization is unified to the source data using different pieces of information model Conversion.
It is the flow chart of the data transfer device that the application is provided shown in Fig. 1, the method can include following Step:
S101, obtains source data to be converted;
S102, the conversion markup information of the source data corresponding data model that reading prestores, the conversion mark At least include in note information:Need the source data word segment information for retaining;
S103, according to the conversion markup information for being read, extracts the source number for needing to retain from the source data According to field, target data is obtained.
Data transfer device provided herein, i.e., corresponding to foregoing soft coral logic, below Above-mentioned data transfer device will be illustrated with reference to specific example:
Field name Field description Field type
name Name String
age Age Long
birthday Birthday Date
height Height Double
Table 1
Field name Field description Field type
name Name String
age Age Long
birthday Birthday Date
tel Telephone number String
Table 2
Tables 1 and 2 respectively illustrates two kinds of field informations of different source data models, wherein data model 1 In contain tetra- fields of name, age, birthday, height, contained in data model 2 name, Tetra- fields of age, birthday, tel.And in actual application demand, it is only necessary to using name, age, Tri- data of field of birthday.So, according to application scheme, can in advance to the sum of data model 1 It is labeled respectively according to the field of model 2, obtains two kinds of conversion markup informations of data model respectively such as table 3 With shown in table 4:
Field name Field description Field type Mark
name Name String 1
age Age Long 1
birthday Birthday Date 1
height Height Double 0
Table 3
Field name Field description Field type Mark
name Name String 1
age Age Long 1
birthday Birthday Date 1
tel Telephone number String 0
Table 4
In table 3 and table 4, represent whether corresponding field is retained in Change-over knot respectively with " 0 " and " 1 " In fruit, it can be seen that " height " in table 3 is noted as 0, table with " tel " field in table 4 Show that corresponding field need not be retained in transformation result.After mark is finished, by the markup information of every kind of model It is corresponding with source data mark to preserve.
In the conversion stage, it is assumed that the incoming source data based on data model 1 is as shown in table 5:
name age birthday height
Zhang San 30 1980-12-12 1.87
Li Si 20 1990-01-12 1.70
King five 15 1995-08-12 1.60
Table 5
According to data transfer device provided herein, after acquisition source data, it is first determined the data make Model is data model 1, and then reads the conversion markup information of data model 1, according to conversion mark letter Breath can determine that the field that needs retain includes name, age, birthday, and these three fields are extracted respectively After data, target data is obtained as shown in table 6:
name age birthday
Zhang San 30 1980-12-12
Li Si 20 1990-01-12
King five 15 1995-08-12
Table 6
In the above-described embodiments, conversion markup information be only used for identify source data each field whether need protect Stay in transformation result, namely acquiescence need not carry out other conversion process.And in actual conversion application, In addition to needing the field to source data to carry out choice treatment, may also need to carry out other conversion process, For example field name is changed, field type change etc., using application scheme, can be with Expanded to realize the demand by conversion markup information.
Assuming that when being changed to data model 1, except needing to retain tri- words of name, age, birthday Section is outer, in order to be adapted with demand data model, it is desirable to the title difference of three fields of target data of conversion It is " username ", " age ", " birthday ", field type is respectively string, short, date, can be with Find out, the title of former name fields, the type of original age fields there occurs change, and the demand can lead to The notation methods shown in table 7 are crossed to realize.
Field name Field description Field type Mark
name Name String @Define (targetName=" username " targetType=" String ")
age Age Long @Define (targetName=" age " targetType=" Short ")
birthday Birthday Date @Define (targetName=" birthday " targetType=" Date ")
height Height Double Null
Table 7
In table 7, in addition to the field height (being labeled with Null) that need not retain after conversion, After 3 fields to needing to retain specify the aiming field title targetName after conversion and conversion Aiming field type targetType, when data conversion is actually carried out, first from source data extract need protect The field stayed, then according to specified targetName and targetType generation target datas.
It is understood that in addition to setting the goal field name, aiming field type in markup information middle finger, Can also specify other conversion requirements, the conversion of such as measurement unit, the treatment that rounds up of numerical value, or even The numerical value of specific function pair source data field is called to be processed, those skilled in the art can be according to reality Demand is extended to conversion markup information.It is further to note that information labeling provided herein Mode is only used for schematically illustrating, and should not be construed as the limitation to application scheme.
In actual applications, the mark to source data model can operate realize by simple configuration, and The income for being exchanged for is:Only with a set of general conversion logic, you can realize to using different pieces of information model Source data is carried out unifying conversion, and for existing many set conversion logics, development cost is substantially reduced; In addition, in general conversion logic, bulk redundancy code is not only reduced, and can be originally scattered Many sets of data conversion logics are concentrated in together so that the readable enhancing of code, are more easily safeguarded.
After generation target data, the data forwarding that will directly can be generated gives data application side, it is also possible to Data to being generated are stored.In a kind of specific embodiment provided herein, for convenience Data are carried out to unify storage, target data can be saved as the form of single-value data.I.e. often row is all single Value Data., it is necessary to embody the key-value pair information of each field of target data in this single-value data.Tool For body, the every a line for target data by each field name of the row and correspondence numerical value, it is necessary to write one In individual character string, while need to be provided in character string segmentation mark to distinguish each field, this character string The as numerical value of single-value data.Alternatively, the type information of field can also be added in single-value data.
In practical application, when storage operation is carried out, can directly using the target word changed in markup information The information such as name section, aiming field type, obtain needing the data of storage, example after the numerical value for assigning source data Such as, 3 datas shown in table 8, wherein preceding 4 row are the source data based on foregoing data model 1, root According to the markup information shown in table 7, the final target data result for preserving such as table 8 last arrange shown in:
Table 8
Using above-mentioned data save mode, any type, any can be just stored using only a content field The data content of amount field, it is not only easy to maintenance, and data structure has effectively been simplified, improve number According to write efficiency.Subsequently when using data, directly according to the create-rule of single-value data, it is possible to reduce Obtain specific field values.
Corresponding to above method embodiment, the application also provides a kind of DTU, shown in Figure 2, The device can include:
Source data obtains module 110, for obtaining source data to be converted;
Markup information read module 120, the conversion for reading the source data corresponding data model for prestoring marks letter Breath, wherein, at least include in conversion markup information:Need the source data word segment information for retaining;
Target data generation module 130, for according to the conversion markup information for being read, being extracted from source data The source data field for retaining is needed, target data is obtained.
In a kind of specific embodiment of the application, can also include in conversion markup information:Need to retain Source data field conversion after aiming field title;
Correspondingly, target data generation module 130 can be specifically for:Letter is marked according to the conversion read Breath, extracts the source data field for needing to retain, and field name is converted into corresponding target from source data Field name, obtains target data.
In a kind of specific embodiment of the application, can also include in conversion markup information:Need to retain Source data field conversion after aiming field type;
Correspondingly, target data generation module 130 can be specifically for:Letter is marked according to the conversion read Breath, extracts the source data field for needing to retain, and field type is converted into corresponding target from source data Field type, obtains target data.
Shown in Figure 3, in a kind of specific embodiment of the application, above-mentioned DTU may be used also Including preserving module 140, for target data to be saved as into single-value data, in the numerical value of single-value data, to take Key-value pair information with each field of the target data.
Alternatively, in the numerical value of single-value data, the field class of each field of the target data can also be carried Type information.
The function of modules and the implementation process of effect specifically refer to correspondence step in the above method in said apparatus Rapid implementation process, will not be repeated here.
As seen through the above description of the embodiments, those skilled in the art can be understood that this Application can add the mode of required general hardware platform to realize by software.Based on such understanding, this Shen The part that technical scheme please substantially contributes to prior art in other words can be in the form of software product Embody, the computer software product can be stored in storage medium, such as ROM/RAM, magnetic disc, CD etc., including some instructions are used to so that computer equipment (can be personal computer, server, Or the network equipment etc.) perform method described in some parts of each embodiment of the application or embodiment.
Each embodiment in this specification is described by the way of progressive, identical phase between each embodiment As part mutually referring to what each embodiment was stressed is the difference with other embodiment. For especially for device embodiment, because it is substantially similar to embodiment of the method, so describing to compare Simply, the relevent part can refer to the partial explaination of embodiments of method.Device embodiment described above is only It is only illustrative, wherein the module illustrated as separating component can be or may not be physics It is upper separate, when application scheme is implemented can the function of each module in same or multiple softwares and/or Realized in hardware.Some or all of module therein can also be according to the actual needs selected to realize this reality Apply the purpose of a scheme.Those of ordinary skill in the art are without creative efforts, you can with Understand and implement.
The above is only the specific embodiment of the application, it is noted that common for the art For technical staff, on the premise of the application principle is not departed from, some improvements and modifications can also be made, These improvements and modifications also should be regarded as the protection domain of the application.

Claims (10)

1. a kind of data transfer device, unification is used for will be converted to using the source data of different pieces of information model The target data of demand data model, it is characterised in that the method includes:
Obtain source data to be converted;
The conversion markup information of the source data corresponding data model that reading prestores, the conversion markup information In at least include:Need the source data word segment information for retaining;
According to the conversion markup information for being read, the source data field for needing to retain is extracted from the source data, Obtain target data.
2. method according to claim 1, it is characterised in that also include in the conversion markup information: Need the aiming field title after the source data field conversion for retaining;
It is described according to the conversion markup information for being read, the source data for needing to retain is extracted from the source data Field, obtains target data, including:According to the conversion markup information for being read, carried from the source data The source data field for needing to retain is taken, and field name is converted into corresponding aiming field title, obtain mesh Mark data.
3. method according to claim 1, it is characterised in that also include in the conversion markup information: Need the aiming field type after the source data field conversion for retaining;
It is described according to the conversion markup information for being read, the source data for needing to retain is extracted from the source data Field, obtains target data, including:According to the conversion markup information for being read, carried from the source data The source data field for needing to retain is taken, and field type is converted into corresponding aiming field type, obtain mesh Mark data.
4. method according to claim 1, it is characterised in that methods described also includes:
The target data is saved as into single-value data, in the numerical value of the single-value data, the number of targets is carried According to the key-value pair information of each field.
5. method according to claim 4, it is characterised in that in the numerical value of the single-value data, also Carry the field type information of each field of the target data.
6. a kind of DTU, unification is used for will be converted to using the source data of different pieces of information model The target data of demand data model, it is characterised in that the device includes:
Source data obtains module, for obtaining source data to be converted;
Markup information read module, the conversion mark for reading the source data corresponding data model for prestoring Information, at least includes in the conversion markup information:Need the source data word segment information for retaining;
Target data generation module, for according to the conversion markup information for being read, being carried from the source data The source data field for needing to retain is taken, target data is obtained.
7. device according to claim 6, it is characterised in that also include in the conversion markup information: Need the aiming field title after the source data field conversion for retaining;
The target data generation module, specifically for:According to the conversion markup information for being read, from described The source data field for needing to retain is extracted in source data, and field name is converted into corresponding aiming field name Claim, obtain target data.
8. device according to claim 6, it is characterised in that also include in the conversion markup information: Need the aiming field type after the source data field conversion for retaining;
The target data generation module, specifically for:According to the conversion markup information for being read, from described The source data field for needing to retain is extracted in source data, and field type is converted into corresponding aiming field class Type, obtains target data.
9. device according to claim 6, it is characterised in that described device also includes:
Preserving module, for the target data to be saved as into single-value data, in the numerical value of the single-value data, Carry the key-value pair information of each field of the target data.
10. device according to claim 9, it is characterised in that in the numerical value of the single-value data, Also carry the field type information of each field of the target data.
CN201510981357.7A 2015-12-23 2015-12-23 Data conversion method and device Active CN106909570B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201510981357.7A CN106909570B (en) 2015-12-23 2015-12-23 Data conversion method and device

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201510981357.7A CN106909570B (en) 2015-12-23 2015-12-23 Data conversion method and device

Publications (2)

Publication Number Publication Date
CN106909570A true CN106909570A (en) 2017-06-30
CN106909570B CN106909570B (en) 2020-12-15

Family

ID=59200046

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201510981357.7A Active CN106909570B (en) 2015-12-23 2015-12-23 Data conversion method and device

Country Status (1)

Country Link
CN (1) CN106909570B (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109346163A (en) * 2018-12-21 2019-02-15 邬超 Data transfer device, equipment and system

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101196927A (en) * 2007-12-29 2008-06-11 中国建设银行股份有限公司 Data mapping method and device thereof
CN101247589A (en) * 2007-07-04 2008-08-20 华为技术有限公司 Mobile terminal data conversion/backup method, device and system
CN102629263A (en) * 2012-02-29 2012-08-08 上海安捷力信息系统有限公司 Method and system for format conversion of interactive business data among enterprises
US20140019889A1 (en) * 2012-07-16 2014-01-16 Uwe Klinger Regenerating a user interface area
CN103970905A (en) * 2014-05-27 2014-08-06 重庆大学 Automatic composition and integration method for multi-source vector geographic information data
CN104662535A (en) * 2012-07-24 2015-05-27 起元科技有限公司 Mapping entities in data models
US20150169667A1 (en) * 2013-12-16 2015-06-18 International Business Machines Corporation System and method of integrating time-aware data from multiple sources

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101247589A (en) * 2007-07-04 2008-08-20 华为技术有限公司 Mobile terminal data conversion/backup method, device and system
CN101196927A (en) * 2007-12-29 2008-06-11 中国建设银行股份有限公司 Data mapping method and device thereof
CN102629263A (en) * 2012-02-29 2012-08-08 上海安捷力信息系统有限公司 Method and system for format conversion of interactive business data among enterprises
US20140019889A1 (en) * 2012-07-16 2014-01-16 Uwe Klinger Regenerating a user interface area
CN104662535A (en) * 2012-07-24 2015-05-27 起元科技有限公司 Mapping entities in data models
US20150169667A1 (en) * 2013-12-16 2015-06-18 International Business Machines Corporation System and method of integrating time-aware data from multiple sources
CN103970905A (en) * 2014-05-27 2014-08-06 重庆大学 Automatic composition and integration method for multi-source vector geographic information data

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109346163A (en) * 2018-12-21 2019-02-15 邬超 Data transfer device, equipment and system

Also Published As

Publication number Publication date
CN106909570B (en) 2020-12-15

Similar Documents

Publication Publication Date Title
EP2597573B1 (en) Test data generation
CN105094707B (en) A kind of data storage, read method and device
US8812947B1 (en) Ranking graphical visualizations of a data set according to data attributes
CN107463668B (en) Method and device, computer equipment and the storage medium of serializing and unserializing
CN110287480A (en) A kind of name entity recognition method, device, storage medium and terminal device
CN106250104B (en) A kind of remote operating system for server, method and device
CN110472068A (en) Big data processing method, equipment and medium based on heterogeneous distributed knowledge mapping
CN110413690A (en) Method of data synchronization, server, electronic equipment, the storage medium of database
CN106649212A (en) Business data processing method and device
CN102122280B (en) Method and system for intelligently extracting content object
CN103559184B (en) The display packing of forms pages and device
CN106844682A (en) Method for interchanging data, apparatus and system
CN111538774B (en) Data storage and display method, system, equipment and storage medium
CN108280056A (en) A kind of Excel file analytic method
CN110457260A (en) Document handling method, device, equipment and computer readable storage medium
CN108171528A (en) A kind of attribution method and attribution system
CN105653255A (en) Method arranging report template to automatically generate report files in office frame
CN108228551A (en) Data lead-in method, device, equipment and the computer readable storage medium of excel
CN110019111A (en) Data processing method, device, storage medium and processor
CN104881461A (en) Rapid data storage method
WO2014137258A1 (en) Selection of data storage settings for an application
CN106407172A (en) XML (Extensible Markup Language)-based report generating method
CN106294128A (en) A kind of automated testing method deriving report data and device
CN107851002A (en) A kind of code compiling method and code encoder
CN105550220A (en) Fetching method and apparatus for heterogeneous system

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
TA01 Transfer of patent application right

Effective date of registration: 20200923

Address after: Cayman Enterprise Centre, 27 Hospital Road, George Town, Grand Cayman Islands

Applicant after: Innovative advanced technology Co.,Ltd.

Address before: Cayman Enterprise Centre, 27 Hospital Road, George Town, Grand Cayman Islands

Applicant before: Advanced innovation technology Co.,Ltd.

Effective date of registration: 20200923

Address after: Cayman Enterprise Centre, 27 Hospital Road, George Town, Grand Cayman Islands

Applicant after: Advanced innovation technology Co.,Ltd.

Address before: A four-storey 847 mailbox in Grand Cayman Capital Building, British Cayman Islands

Applicant before: Alibaba Group Holding Ltd.

TA01 Transfer of patent application right
GR01 Patent grant
GR01 Patent grant
TR01 Transfer of patent right

Effective date of registration: 20220330

Address after: 310000 801-11 section B, 8th floor, 556 Xixi Road, Xihu District, Hangzhou City, Zhejiang Province

Patentee after: Alipay (Hangzhou) Information Technology Co.,Ltd.

Address before: Ky1-9008 Cayman Enterprise Centre, 27 Hospital Road, George Town, Grand Cayman Islands, ky1-9008

Patentee before: Innovative advanced technology Co.,Ltd.

TR01 Transfer of patent right