Invention content
It is a primary object of the present invention to provide a kind of data conversion information processing method and device, to solve the relevant technologies
Middle searching data converts the problem of successfully low with the information efficiency of data conversion failure.
To achieve these goals, according to an aspect of the invention, there is provided a kind of data convert information processing method.
Data conversion information processing method according to the present invention includes:Obtain data to be transformed;Treat conversion data execution
Conversion processing obtains conversion results;Conversion successful data and the data to be transformed in data to be transformed are obtained according to conversion results
Middle conversion miss data, wherein, conversion successful data refers to data of the successful conversion for target data in data to be transformed, conversion
Miss data refers to be converted into the data of target data not successfully in data to be transformed;And by being converted in data to be transformed
Work(data are stored to first object storage location, and the conversion miss data in data to be transformed is stored to the second target and is stored
Position.
Further, data to be transformed are obtained to include:Source data is obtained from presetting database;Source data is stored to
Three target storage positions;And obtain data to be transformed from third target storage position.
Further, conversion data execution conversion processing is treated to include:Determine default conversion type;According to default conversion class
Data to be transformed are converted into target data by type, wherein, the type of target data is default conversion type;Establish target data with
The correspondence of data to be transformed;And the correspondence of output target data and data to be transformed.
Further, the conversion successful data in data to be transformed is stored to first object storage location, and will waits to turn
Conversion miss data in change data, which is stored to the second target storage position, to be included:Obtain preset data file format;And it presses
The conversion successful data in data to be transformed will be stored according to preset data file format to first object storage location, and according to
Preset data file format stores the conversion miss data in data to be transformed to the second target storage position.
Further, the conversion successful data in data to be transformed are obtained according to conversion results and data relay to be transformed
After changing miss data, this method further includes:Turned according to conversion successful data and conversion miss data parsing conversion miss data
Change the type of failure;And the type of output conversion miss data conversion failure.
To achieve these goals, according to another aspect of the present invention, a kind of data conversion information processing unit is provided.
Data conversion information processing unit according to the present invention includes:First acquisition unit, for obtaining data to be transformed;
Conversion unit performs conversion processing for treating conversion data, obtains conversion results;Second acquisition unit, for according to conversion
As a result the conversion successful data in data to be transformed and data relay miss data to be transformed are obtained, wherein, it converts and successfully counts
According to data of the successful conversion for target data in data to be transformed are referred to, conversion miss data refers to failed in data to be transformed
It is converted into the data of target data;And storage unit, for storing the conversion successful data in data to be transformed to first
Target storage position, and the conversion miss data in data to be transformed is stored to the second target storage position.
Further, first acquisition unit includes:First acquisition module, for obtaining source data from presetting database;
First memory module, for storing source data to third target storage position;And second acquisition module, for from third mesh
It marks storage location and obtains data to be transformed.
Further, conversion unit includes:Determining module, for determining default conversion type;Conversion module, for basis
Data to be transformed are converted into target data by default conversion type, wherein, the type of target data is default conversion type;It establishes
Module, for establishing the correspondence of target data and data to be transformed;And output module, for exporting target data with treating
The correspondence of conversion data.
Further, storage unit includes:Third acquisition module, for obtaining preset data file format;And second
Memory module, for will store the conversion successful data in data to be transformed to first object according to preset data file format
Storage location, and the conversion miss data in data to be transformed is stored to the second target according to preset data file format and is stored
Position.
Further, which further includes:Resolution unit, for according to conversion successful data and conversion miss data parsing
Convert the type of miss data conversion failure;And output unit, for exporting the type of conversion miss data conversion failure.
By the present invention, using the conversion successful data and data relay to be transformed stored respectively automatically in data to be transformed
Change miss data, solve the problems, such as that searching data converts successfully low with the information efficiency of data conversion failure in the relevant technologies,
And then achieve the effect that improving searching data converts the successfully information efficiency with data conversion failure.
Specific embodiment
It should be noted that in the absence of conflict, the feature in embodiment and embodiment in the application can phase
Mutually combination.The present invention will be described in detail below with reference to the accompanying drawings and in conjunction with the embodiments.
In order to which those skilled in the art is made to more fully understand application scheme, below in conjunction in the embodiment of the present application
The technical solution in the embodiment of the present application is clearly and completely described in attached drawing, it is clear that described embodiment is only
The embodiment of the application part, instead of all the embodiments.Based on the embodiment in the application, ordinary skill people
Member's all other embodiments obtained without making creative work should all belong to the model of the application protection
It encloses.
It should be noted that term " first " in the description and claims of this application and above-mentioned attached drawing, "
Two " etc. be the object for distinguishing similar, and specific sequence or precedence are described without being used for.It should be appreciated that it uses in this way
Data can be interchanged in the appropriate case, so as to embodiments herein described herein.In addition, term " comprising " and " tool
Have " and their any deformation, it is intended that cover it is non-exclusive include, for example, containing series of steps or unit
Process, method, system, product or equipment are not necessarily limited to those steps or unit clearly listed, but may include without clear
It is listing to Chu or for the intrinsic other steps of these processes, method, product or equipment or unit.
According to embodiments of the present invention, a kind of data conversion information processing method is provided.Fig. 1 is according to embodiments of the present invention
Data conversion information processing method flow chart.
As shown in Figure 1, data conversion information processing method includes steps S102 to step S108:
Step S102:Obtain data to be transformed.
Data conversion refers to initial data being converted into target data according to preset rules, for example, data type conversion, number
According to integration, data mining etc., the embodiment of the present invention is illustrated for being based on SSIS platforms and carrying out data conversion.Number to be transformed
Can be the data that are obtained from database or from other electronic watches according to the data for referring to need to perform conversion processing
The data obtained in lattice, for example, the data obtained from Excel tables.Preferably, data to be transformed are obtained to include:From default
Source data is obtained in database;Source data is stored to third target storage position;And it is obtained from third target storage position
Data to be transformed.
Presetting database is data source data library, specifically, can be filtered out by SQL statement from presetting database
Need the data converted, i.e. source data.In order to detect in data conversion process with the presence or absence of loss of data, can will obtain
The source data that arrives carries out a dump, for example, be stored in third target storage position, so as to, can be with after data conversion is completed
It is that data conversion process whether there is loss of data data after conversion and the source data to be compared to judge, for example, number
It is 100 according to source data amount before conversion, it is 85 that successfully data volume is converted after data conversion, and conversion miss data amount is 10, then can obtain
It is 5 to lose data volume to conversion.Cause the above results inaccurate in order to avoid occurring loss of data in dump procedure, the present invention is real
It applies example and obtains data to be transformed from third target storage position.
Step S104:It treats conversion data and performs conversion processing, obtain conversion results.
SSIS platforms can easily realize that data convert, and the embodiment of the present invention is with using the progress data conversion of SSIS platforms
For illustrate.Specifically, conversion data execution conversion processing is treated to include:Determine default conversion type;Turned according to default
Change type and data to be transformed are converted into target data, wherein, the type of target data is default conversion type;Establish number of targets
According to the correspondence with data to be transformed;And the correspondence of output target data and data to be transformed.
Data conversion includes multiple types, for example, type conversion, Data Integration etc., it can be arbitrary to preset conversion type
A kind of data convert type.When carrying out data conversion using SSIS platforms, user can pre-set data conversion type,
When data convert task start, SSIS platforms can perform corresponding data conversion operation automatically according to default conversion type, obtain
Target data further analyzes data conversion results for the ease of user, establishes target data pass corresponding with data to be transformed
System.Specifically, it is the data of successful conversion in data to be transformed are corresponding with the target data that it is converted and export or into one
Step is stored in target database.
Step S106:Conversion successful data and the data relay to be transformed in data to be transformed are obtained according to conversion results
Miss data, wherein, conversion successful data refers to data of the successful conversion for target data in data to be transformed, and conversion unsuccessfully counts
According to the data for referring to be converted into target data not successfully in data to be transformed.
After treating conversion data execution conversion processing, the conversion successful data in data to be transformed and data to be transformed are obtained
Middle conversion miss data, for example, can be data relay successful data to be transformed and number to be transformed in data conversion process
According to middle conversion miss data plus different marks, distinguish the conversion successful data in data to be transformed according to mark and wait to turn
Change data relay miss data, it can also be in the conversion process by the conversion successful data in data to be transformed from number to be transformed
The memory headroom of other positions is stored according to middle taking-up, so as to be distinguished according to the position of content space in data to be transformed
Conversion successful data and data relay miss data to be transformed.
Step S108:Conversion successful data in data to be transformed is stored to first object storage location, and will wait to turn
The conversion miss data changed in data is stored to the second target storage position.
First object storage location and the second target storage position can be the different storage locations of any two.This hair
Bright embodiment is by obtaining the conversion in the conversion successful data and data to be transformed in data conversion process in data to be transformed
Miss data, and the conversion miss data in the conversion successful data and data to be transformed in data to be transformed is stored respectively in
Different storage locations are realized in automatic record data conversion process and are converted to work(data and conversion miss data, so as to side
Just quickly obtain and check conversion successful data and conversion miss data, solve searching data convert successfully with data conversion
The problem of information efficiency of failure is low.In addition, it can quickly and accurately be analyzed according to conversion successful data and conversion miss data
The defects of going out current data conversion, improves working efficiency, reduces error rate.
Preferably, the conversion successful data in data to be transformed is stored to first object storage location, and will be to be transformed
Conversion miss data in data, which is stored to the second target storage position, to be included:Obtain preset data file format;And according to
Preset data file format will store the conversion successful data in data to be transformed to first object storage location, and according to pre-
If document format data stores the conversion miss data in data to be transformed to the second target storage position.
Preset data file format refers to the file format of data storage, for example, it may be TXT file formats, Ke Yishi
Excel forms or alternative document form.It, will be in data to be transformed after the file format stored in setting data
Conversion successful data and data to be transformed in conversion miss data according to pre-set stored in file format, for example, depositing
It stores up as Excel tables.The embodiment of the present invention can flexibly select the file format that data store, and subsequently treat and turn convenient for user
Change the use of the conversion successful data in data and the conversion miss data in data to be transformed.
Preferably, the conversion successful data in data to be transformed are obtained according to conversion results and data relay to be transformed
After miss data, method further includes:It is lost according to conversion successful data and conversion miss data parsing conversion miss data conversion
The type lost;And the type of output conversion miss data conversion failure.
Data conversion may be unsuccessfully caused by due to character length, character types, character sequence etc..Specifically,
Multiple rules can be pre-set to be detected the type of data conversion failure.For example, character length, character type are detected successively
Type, character sequence.It is illustrated by taking character length as an example, detection first is totally converted successfully the character length of data, obtains most
Big character length and minimum character length, then conversion miss data is traversed, the character of each conversion miss data is judged respectively
Whether between maximum character length and minimum character length, specifically, a threshold value can be set, if turned by calculating
Character length is not at the data volume between maximum character length and minimum character length and character length in change miss data
The ratio of data volume between maximum character length and minimum character length, the number is determined if the ratio is more than preset value
It is inconsistent for character length according to the type of conversion failure, otherwise continue the type detection of other conversion failures, for example, detection detection
Character types if detecting that be totally converted successfully data is the character string that letter is formed, and detect conversion and unsuccessfully count
According to being not by the alphabetical character string formed, determine that the type that data conversion fails is inconsistent for character types at this time.Respectively
After the type for detecting preset multiple conversion failures, testing result is exported, for example, it is that character length mismatch causes to count to detect
During according to conversion failure, it can be mismatched with output character length, user can be further according to the testing result rapidly locating
The root of failure is converted, improves the efficiency of searching data conversion failure cause.
It can be seen from the above description that the present invention realizes following technique effect:
The embodiment of the present invention treats conversion data and performs conversion processing, obtain conversion results by obtaining data to be transformed,
Conversion successful data in data to be transformed and data relay miss data to be transformed are obtained according to conversion results and will be treated
Conversion successful data in conversion data is stored to first object storage location, and by the conversion miss data in data to be transformed
It stores to the second target storage position.The embodiment of the present invention will be in the conversion successful data and data to be transformed in data to be transformed
Conversion miss data be stored respectively in different storage locations, realize be converted in automatic record data conversion process work(data and
Miss data is converted, so as to obtain and check conversion successful data and conversion miss data easily and fast, solves and looks into
Data is looked for convert the problem of successfully low with the information efficiency of data conversion failure.In addition, the embodiment of the present invention can further root
Data conversion failure is automatically analyzed according to the conversion miss data in the conversion successful data and data to be transformed in data to be transformed
The reason of, improve the efficiency that searching data converts failure cause.
It should be noted that step shown in the flowchart of the accompanying drawings can be in such as a group of computer-executable instructions
It is performed in computer system, although also, show logical order in flow charts, it in some cases, can be with not
The sequence being same as herein performs shown or described step.
Another aspect according to embodiments of the present invention provides a kind of data conversion information processing unit, data conversion
Information processing unit can be used for performing the data conversion information processing method of the embodiment of the present invention, the method for the embodiment of the present invention
Data that can also be through the embodiment of the present invention convert information processing unit to perform.
Fig. 2 is the schematic diagram of data conversion information processing unit according to embodiments of the present invention, as shown in Fig. 2, the data
Conversion information processing unit includes:First acquisition unit 10, conversion unit 20, second acquisition unit 30 and storage unit 40.
First acquisition unit 10, for obtaining data to be transformed.
Data conversion refers to initial data being converted into target data according to preset rules, for example, data type conversion, number
According to integration, data mining etc..Data to be transformed refer to the data for needing to perform conversion processing, can be obtained from database
Data or the data obtained from other electrical forms, for example, the data obtained from Excel tables.
Preferably, first acquisition unit 10 includes:First acquisition module, for obtaining source data from presetting database;
First memory module, for storing source data to third target storage position;And second acquisition module, for from third mesh
It marks storage location and obtains data to be transformed.
Conversion unit 20 performs conversion processing for treating conversion data, obtains conversion results.
Optionally, conversion unit 20 includes:Determining module, for determining default conversion type;Conversion module, for basis
Data to be transformed are converted into target data by default conversion type, wherein, the type of target data is default conversion type;It establishes
Module, for establishing the correspondence of target data and data to be transformed;And output module, for exporting target data with treating
The correspondence of conversion data.
Second acquisition unit 30, for obtaining conversion successful data in data to be transformed and to be transformed according to conversion results
Data relay miss data, wherein, conversion successful data refers to data of the successful conversion for target data in data to be transformed,
Conversion miss data refers to be converted into the data of target data not successfully in data to be transformed.
After treating conversion data execution conversion processing, the conversion successful data in data to be transformed and data to be transformed are obtained
Middle conversion miss data, for example, can be data relay successful data to be transformed and number to be transformed in data conversion process
According to middle conversion miss data plus different marks, distinguish the conversion successful data in data to be transformed according to mark and wait to turn
Change data relay miss data, it can also be in the conversion process by the conversion successful data in data to be transformed from number to be transformed
The memory headroom of other positions is stored according to middle taking-up, so as to be distinguished according to the position of content space in data to be transformed
Conversion successful data and data relay miss data to be transformed.
Storage unit 40, for storing the conversion successful data in data to be transformed to first object storage location, and
Conversion miss data in data to be transformed is stored to the second target storage position.
First object storage location and the second target storage position are the different storage locations of any two.Locating respectively
After storing conversion successful data and the data relay miss data to be transformed in data to be transformed, user can combine and treat
The number recorded in conversion successful data and data relay miss data to be transformed and data conversion daily record in conversion data
According to the information that conversion is abnormal and fails, for example, SSIS platforms carry out the log information recorded during data conversion, number is quickly analyzed
The reason of according to conversion failure.
Preferably, storage unit 40 includes:Third acquisition module, for obtaining preset data file format;And second
Memory module, for will store the conversion successful data in data to be transformed to first object according to preset data file format
Storage location, and the conversion miss data in data to be transformed is stored to the second target according to preset data file format and is stored
Position.
Preferably, which further includes:Resolution unit, for being turned according to conversion successful data and conversion miss data parsing
Change the type of miss data conversion failure;And output unit, for exporting the type of conversion miss data conversion failure.
Data conversion may be unsuccessfully caused by due to character length, character types, character sequence etc..Specifically,
Multiple rules can be pre-set to be detected the type of data conversion failure.For example, character length, character type are detected successively
Type, character sequence.
The embodiment of the present invention obtains data to be transformed by first acquisition unit 10, and converting unit 20 is treated conversion data and held
Row conversion processing, obtains conversion results, and second acquisition unit 30 obtains the conversion success in data to be transformed according to conversion results
Data and data relay miss data to be transformed and storage unit 40 store the conversion successful data in data to be transformed
It is stored to first object storage location, and by the conversion miss data in data to be transformed to the second target storage position.Pass through
Conversion miss data in conversion successful data and data to be transformed in data to be transformed is stored respectively in different storage positions
It puts, realizes in automatic record data conversion process and be converted to work(data and conversion miss data, so as to obtain easily and fast
It takes and checks conversion successful data and convert miss data, solve searching data and convert the successfully information with data conversion failure
The problem of efficiency is low.
Obviously, those skilled in the art should be understood that each module of the above-mentioned present invention or each step can be with general
Computing device realize that they can concentrate on single computing device or be distributed in multiple computing devices and be formed
Network on, optionally, they can be realized with the program code that computing device can perform, it is thus possible to which they are stored
In the storage device by computing device come perform either they are fabricated to respectively each integrated circuit modules or by they
In multiple modules or step be fabricated to single integrated circuit module to realize.In this way, the present invention is not limited to any specific
Hardware and software combines.
The foregoing is only a preferred embodiment of the present invention, is not intended to restrict the invention, for the skill of this field
For art personnel, the invention may be variously modified and varied.All within the spirits and principles of the present invention, that is made any repaiies
Change, equivalent replacement, improvement etc., should all be included in the protection scope of the present invention.