CN107203564B - Data transmission method, device and system - Google Patents
Data transmission method, device and system Download PDFInfo
- Publication number
- CN107203564B CN107203564B CN201610157785.2A CN201610157785A CN107203564B CN 107203564 B CN107203564 B CN 107203564B CN 201610157785 A CN201610157785 A CN 201610157785A CN 107203564 B CN107203564 B CN 107203564B
- Authority
- CN
- China
- Prior art keywords
- data
- file
- original
- field
- configuration information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Computing Systems (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The application discloses a method, a device and a system for data transmission, wherein the method comprises the following steps: checking whether dirty data exists in original data, wherein the original data is data in an original file received from a source system; and feeding back the file generated after verification to the source system so that the source system intervenes after scanning the dirty data from the feedback result. The method can automatically find dirty data and feed the dirty data back to the source system to complete automatic verification of the file, solves the problem that the dirty data can only be found in the storage stage in the past, and automatically feeds the result back to the source system after the dirty data is detected, so that the problem of the dirty data can be solved in time, and the cost for communication between the source system and the target system is reduced.
Description
Technical Field
The present disclosure generally relates to the field of data management technologies, and in particular, to a method, an apparatus, and a system for data transmission.
Background
In various applications at present, data synchronization between different systems is usually required, that is, a user needs to obtain data from a data source system and load the data into a target system. The method for realizing data synchronization among different systems comprises the steps of extracting the data from the database, locally extracting the data by an active system (namely a data provider), generating a file on the ground, and then transmitting the file to a target system (namely a data receiver) for warehousing. The influence on a source system (a business system) can be reduced through an extraction mode of file transmission, and the method is widely applied to a plurality of important scenes (such as industries of banks, telecommunications and the like) of the business system.
A general data transmission principle is schematically shown in fig. 1, where data is transmitted from a source system to a target system only by a single data transmission. The traditional transmission is simple, the source system extracts data from the related service system, the data are generated into files in text format, then the data transmission is carried out in network transmission modes such as FTP/HTTP and the like, and after the target system receives the files in text format, the files are converted and put in storage and the like. Due to the fact that file transmission goes through two links of extraction and land (certain manual intervention exists in the process of file transmission), if data with illegal data formats or data which is not in a given range (generally called dirty data) exists in the data generation process, the dirty data sometimes cannot be loaded into a target system, or even if the data is loaded into a database, the data use is adversely affected.
For the processing of dirty data, no specific technology and method exist at present, and usually, the intervention processing is performed only when a program error is found in the data storage link.
Therefore, a new method, apparatus and system for data transmission is needed.
The above information disclosed in this background section is only for enhancement of understanding of the background of the disclosure and therefore it may contain information that does not constitute prior art that is already known to a person of ordinary skill in the art.
Disclosure of Invention
The present disclosure provides a data transmission management method, device and system, so as to solve the technical problem in the prior art that the use of data is affected by finding dirty data only at a data storage stage.
Additional features and advantages of the disclosure will be set forth in the detailed description which follows, or in part will be obvious from the description, or may be learned by practice of the disclosure.
According to an aspect of the present disclosure, there is provided a method of data transmission, including:
before warehousing, checking whether dirty data exists in original data, wherein the original data is data in an original file received from a source system;
and feeding back the file generated after verification to the source system so that the source system intervenes after scanning the dirty data from the feedback result.
According to an embodiment of the present disclosure, checking whether dirty data exists in the original data includes:
establishing a configuration file;
verifying the original data according to the configuration file;
if the verification result is passed, outputting a mark file passed by the verification; and if the verification result is failure, outputting a verification failure mark file and outputting dirty data obtained by verification.
According to another embodiment of the present disclosure, the configuration file includes at least one piece of configuration information, and each piece of configuration information corresponds to one original file.
According to another embodiment of the present disclosure, the configuration information includes a file name, a number of fields, and a data type corresponding to each field.
According to another embodiment of the present disclosure, the configuration information further includes a data range for each field, the data range being an enumerated value or a range value.
According to another embodiment of the present disclosure, verifying the original data according to the configuration file includes:
reading and judging whether the number of actual fields in the original data is consistent with the number of fields in the configuration information, whether the actual data type of each field in the original data is consistent with the data type of each field in the configuration information, and whether the actual value of each field in the original data is in the data range of each field in the configuration information;
and when the actual field number in the original data is consistent with the field number in the configuration information, the actual data type of each field in the original data is consistent with the data type of each field in the configuration information, and the actual value of each field in the original data is in the data range of each field in the configuration information, the check result is passed.
According to another aspect of the present disclosure, there is also provided an apparatus for data transmission, including:
the system comprises a verification module, a storage module and a storage module, wherein the verification module is used for verifying whether dirty data exists in original data before warehousing, and the original data is data in an original file received from a source system; and
and the feedback module is used for feeding back the file generated after the verification to the source system so that the source system can intervene after scanning the dirty data from the feedback result.
According to an embodiment of the present disclosure, the verification module includes:
the configuration submodule is used for establishing a configuration file;
the verification submodule is used for verifying the original data according to the configuration file; and
the output submodule is used for outputting the verification result of the verification submodule, and outputting a mark file which passes the verification when the verification result is passed; and when the verification result is failure, outputting a verification failure mark file and outputting dirty data obtained by verification.
According to still another aspect of the present disclosure, there is also provided a system for data transmission, including a source system and a target system, the source system transmitting an original file to the target system, the target system including:
the verification module is used for verifying whether dirty data exists in the original data of the original file before warehousing; and
the feedback module is used for feeding back the file generated after the inspection to the source system;
the source system includes:
and the scanning module is used for carrying out cyclic scanning and intervening on the dirty data after scanning the feedback result of the feedback module.
According to an embodiment of the present disclosure, the source system further includes:
and the retransmission module is used for checking and repairing the dirty data after the intervention is carried out on the dirty data, and retransmitting the repaired data to the target system.
According to the technical scheme of the disclosure, the following technical effects can be obtained:
the dirty data can be automatically found and fed back to the source system, automatic verification of the file is completed, the defect that the dirty data can only be found in the storage stage in the past is overcome, and after the dirty data is detected, the result is automatically fed back to the source system, so that the problem of the dirty data can be timely solved, and the cost for communication between the source system and the target system is reduced.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.
Drawings
The above and other objects, features and advantages of the present disclosure will become more apparent by describing in detail exemplary embodiments thereof with reference to the attached drawings.
Fig. 1 schematically shows a schematic diagram of data transmission in the related art.
Fig. 2 schematically illustrates a flow chart of steps of a method of data transmission provided in an embodiment according to the present disclosure.
FIG. 3 is a flow chart that schematically illustrates steps for verifying the presence of dirty data in raw data, in accordance with an embodiment of the present disclosure.
Fig. 4 schematically illustrates a schematic diagram of a method of data transmission provided in an embodiment according to the present disclosure.
Fig. 5 schematically illustrates a schematic diagram of an apparatus for data transmission provided in another embodiment of the present disclosure.
Fig. 6 schematically illustrates a schematic diagram of a system for data transmission provided in accordance with another embodiment of the present disclosure.
Detailed Description
Example embodiments will now be described more fully with reference to the accompanying drawings. Example embodiments may, however, be embodied in many different forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the concept of example embodiments to those skilled in the art. The drawings are merely schematic illustrations of the present disclosure and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and thus their repetitive description will be omitted.
Furthermore, the described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided to give a thorough understanding of embodiments of the disclosure. One skilled in the relevant art will recognize, however, that the subject matter of the present disclosure can be practiced without one or more of the specific details, or with other methods, components, devices, steps, and the like. In other instances, well-known structures, methods, devices, implementations, materials, or operations are not shown or described in detail to avoid obscuring aspects of the disclosure.
Some of the block diagrams shown in the figures are functional entities and do not necessarily correspond to physically or logically separate entities. These functional entities may be implemented in the form of software, or in one or more hardware modules or integrated circuits, or in different networks and/or processor devices and/or microcontroller devices.
Because the dirty data of the file in the prior art is often not solved in the first time, even if the dirty data is found in the data storage stage, a large amount of cost for manual communication between the source system and the target system is required. The purpose of the present disclosure is to detect dirty data in time and automatically feed back the dirty data to a source system in a first time, so that the problem of dirty data can be processed most quickly, and the communication cost of both systems can be reduced.
Fig. 2 is a flowchart illustrating steps of a method for data transmission provided in an embodiment of the present disclosure, which may perform an automatic check on dirty data of original data transmitted by a source system, and automatically feed back a check result to the source system, so as to discover and process the dirty data in time.
As shown in fig. 2, in step S10, before warehousing, it is checked whether dirty data exists in the original data, where the original data is the data in the original file received from the source system. The source system transmits required data to the target system, usually in the form of an original file, so that the source system is also referred to as a data provider, and mainly provides and transmits the original file, and the target system is also referred to as a data receiver, and receives and verifies the data transmitted from the source system.
As shown in fig. 2, in step S20, the file generated after verification is fed back to the source system, so that the source system intervenes after scanning dirty data from the feedback result.
Fig. 3 is a flowchart illustrating steps of checking whether dirty data exists in original data according to the present embodiment.
As shown in fig. 3, in step S11, a configuration file is created, where the configuration file includes at least one piece of configuration information, and each piece of configuration information corresponds to one original file.
Before starting data transmission between a target system and a source system, an interface protocol is established in advance to agree with information such as data format in a file transmitted between the target system and the source system, namely configuration information. The configuration information comprises file names, field numbers and data types corresponding to each field, and a file list needing to be transmitted and related configuration information are added into the configuration file. Because the original data contained in different original files transmitted from the source system to the target system are often data with completely different contents, for example, the original data may be order data, basic information data, etc., and the agreed configuration information is different, a piece of configuration information may be established for original files which are all order data and a piece of configuration information is established for original files which are all user basic information data.
In this embodiment, after a piece of configuration information is created, the following record may be added to the configuration file:
Filename.txt
4
string double string int
filename field number data type
Txt, the file contains a total of 4 fields, and the data types of the 4 fields are respectively as follows in sequence:string double string inti.e. character, floating point, character, integer.
If the source system transmits a plurality of original files to the target system, a plurality of pieces of configuration information need to be established and added to the configuration files, as follows:
number of Filename1 … … field … data type (Range)
Number of Filename2 … … field … data type (Range)
………
It should be noted that, in addition to the number of fields and the data type corresponding to each field, the configuration information may also include a data range of each field, where the data range is an enumerated value or a range value. For example, configuration int [10-20], indicates that the range of the field must be between 10-20 (range value), or configuration string [ a, b, c ], indicates that the range of the field contains 3 values a, b, c (enumerated value). The configuration information may therefore also be as follows:
Filename1 3 int[1-10] string[a,b,c] int[11-20]
the above shows a file representing Filename1, which contains a total of 3 fields, of which the 1 st field is an integer in the data range 1-10, the 2 nd field is a character of a, b or c, and the 3 rd field is an integer in the data range 11-20.
As shown in fig. 3, in step S12, the original data is verified according to the configuration file, wherein the verification process includes:
and reading and judging whether the number of actual fields in the original data is consistent with the number of fields in the configuration information, whether the actual data type of each field in the original data is consistent with the data type of each field in the configuration information, and whether the actual value of each field in the original data is in the data range of each field in the configuration information. And when the actual field number in the original data is consistent with the field number in the configuration information, the actual data type of each field in the original data is consistent with the data type of each field in the configuration information, and the actual value of each field in the original data is in the data range of each field in the configuration information, the verification result is passed.
In this embodiment, the number of check fields, the data type, and the sequence of the data range are not specifically limited in the checking process, and when the original data is read, whether the original data is read all at once and then one by one or a part of the original data is read is determined, and a part of the original data is also not limited.
That is, the verification method may also adopt the following manner:
firstly, reading an original file and judging whether the number of fields of each row of data (namely one piece of original data) in the original file is consistent with the number in a configuration file. For example: 10 fields are defined in the configuration file, and the associated file must also contain 10 fields.
Secondly, reading the original file and judging whether each field type in the original data is consistent with the type of the configuration file. Such as: must be integer defined as int type, must be character type data defined as string.
And finally, reading the original file and judging whether the actual value of each field in the original data is in the range configured in the configuration file. Such as: int [10-20], if 21, exceeds the range of 10-20, it is not preferable.
As shown in fig. 3, in step S13, if the verification result is pass, outputting a flag file that passes the verification, for example, outputting a filename. And if the verification result is failure, outputting a verification failure flag file, and outputting dirty data obtained by verification, such as outputting a Filename.
The target system not only checks the original data, but also further checks the result file output after checking, including: and automatically feeding back to the source system through files (ok files), check failure files (err files and wrg files), FTP or HTTP protocol, wherein the file name is the name of the file.
And continuously and circularly scanning whether a result file fed back from the target system exists in the source system, if so, confirming that a certain type of file is normally transmitted, and if so, indicating that dirty data exists, needing related intervention, inquiring specific dirty data (wrg), checking and repairing the original data, and transmitting the repaired file to the target system again from the source system.
For the method provided in this embodiment, a corresponding schematic diagram is shown in fig. 4, and the service data (i.e., original data) to be transmitted in the source system is generally transmitted to the target system in the form of a log file, and then dirty data check is performed, and if the check passes, an ok identification file is generated and fed back to the source system; if the check fails, an err identification file and a dirty data wrg file are generated and fed back to the source system. If the source system scans the ok identification file, the data transmission is successful, and if the source system scans the err identification file, the data transmission is failed.
In summary, the method provided by this embodiment can automatically find dirty data and feed back the dirty data to the source system, so as to complete automatic verification of the file, thereby solving the problem that the dirty data can only be found in the previous storage stage, and after the dirty data is detected, the result is automatically fed back to the source system, so that the problem of the dirty data can be solved in time, and the cost for communication between the source system and the target system is reduced.
Fig. 5 is a schematic diagram of a data transmission apparatus according to another embodiment of the present embodiment, where the apparatus 100 includes: a verification module 110 and a feedback module 120. The checking module 110 is configured to check whether dirty data exists in original data before warehousing, where the original data is data in an original file received from a source system; the feedback module 120 is configured to feed back the file generated after the verification to the source system, so that the source system intervenes after scanning dirty data from the feedback result.
In this embodiment, as shown in fig. 5, the verification module 110 includes: the configuration sub-module 111, the check sub-module 112 and the output sub-module 113, wherein the configuration sub-module 111 is used for establishing a configuration file; the verification submodule 112 is configured to verify the original data according to the configuration file; the output submodule 113 is configured to output a verification result of the verification submodule, and when the verification result is a pass, output a flag file that the verification passes; and when the verification result is failure, outputting a verification failure mark file and outputting dirty data obtained by verification.
The configuration sub-module 111 creates a piece of configuration information for each original file in creating the configuration file. The configuration information comprises file names, field numbers and data types corresponding to each field, and a file list needing to be transmitted and related configuration information are added into the configuration file. Because the original data contained in different original files transmitted from the source system to the target system are often data with completely different contents, for example, the original data may be order data, basic information data, etc., and the agreed configuration information is different, a piece of configuration information may be established for original files which are all order data and a piece of configuration information is established for original files which are all user basic information data.
In this embodiment, after a piece of configuration information is created, the following record may be added to the configuration file:
Filename.txt
4
string double string int
filename field number data type
Txt above represents a Filename.txt file, which contains a total of 4 fieldsThe data types of the 4 fields are respectively as follows in sequence:string double string inti.e. character, floating point, character, integer.
If the source system transmits a plurality of original files to the target system, a plurality of pieces of configuration information need to be established and added to the configuration files, as follows:
number of Filename1 … … field … data type (Range)
Number of Filename2 … … field … data type (Range)
………
It should be noted that, in addition to the number of fields and the data type corresponding to each field, the configuration information may also include a data range of each field, where the data range is an enumerated value or a range value. For example, configuration int [10-20], indicates that the range of the field must be between 10-20 (range value), or configuration string [ a, b, c ], indicates that the range of the field contains 3 values a, b, c (enumerated value). The configuration information may therefore also be as follows:
Filename1 3 int[1-10] string[a,b,c] int[11-20]
the above shows a file representing Filename1, which contains a total of 3 fields, of which the 1 st field is an integer in the data range 1-10, the 2 nd field is a character of a, b or c, and the 3 rd field is an integer in the data range 11-20.
The process of verifying the original data by the verification submodule 112 according to the configuration file is as follows:
firstly, reading an original file and judging whether the number of fields of each row of data (namely one piece of original data) in the original file is consistent with the number in a configuration file. For example: 10 fields are defined in the configuration file, and the associated file must also contain 10 fields.
Secondly, reading the original file and judging whether each field type in the original data is consistent with the type of the configuration file. Such as: must be integer defined as int type, must be character type data defined as string.
And finally, reading the original file and judging whether the actual value of each field in the original data is in the range configured in the configuration file. Such as: int [10-20], if 21, exceeds the range of 10-20, it is not preferable.
The output sub-module 113 outputs the verification result, and if the verification result is passed, outputs a flag file that is passed through the verification, for example, outputs a filename. And if the verification result is failure, outputting a verification failure flag file, and outputting dirty data obtained by verification, such as outputting a Filename.
The target system not only checks the original data, but also further checks the result file output after checking, including: the file is automatically fed back to the source system through the feedback module 120 through an FTP or HTTP protocol (ok file), a check failure file (err file and an aster wrg file), wherein the file is a file name, so that the source system can continuously and circularly scan whether a result file fed back from the target system exists, if the file is scanned, the normal transmission of a certain type of file can be confirmed, if the file is scanned, the file indicates that dirty data exists, at this time, related manual intervention is required, specific dirty data can be inquired (wrg), and the original data is checked and repaired. The source system performs data check according to the dirty data check rule set by the target system, for example: whether the number of the fields is consistent or not; whether the field data types are consistent; and if the field data ranges are consistent, repairing the data in a manual data repairing mode or a file regenerating mode according to specific data conditions after the checking is finished, and transmitting the repaired file to the target system from the source system again.
Fig. 6 also shows a schematic diagram of a system for data transmission according to another embodiment of the present embodiment, where the system 1000 includes a source system 1100 and a target system 1200.
The target system includes a verification module 1110 and a feedback module 1120. The checking module 1110 is configured to check whether dirty data exists in original data of an original file before warehousing, and the feedback module 1120 is configured to feed back a file generated after checking to a source system;
the source system 1200 includes a scanning module 1210 and a retransmission module 1220, the scanning module 1210 is configured to perform cyclic scanning, intervene on dirty data after scanning a feedback result of the feedback module, and the retransmission module 1220 is configured to perform inspection and repair after intervening on dirty data.
The source system 1100 transmits an original file to the target system 1200, the target system 1200 receives the original file, checks data in the original file, feeds a check result and the checked dirty data back to the source system 1100, finally, the source system 1100 scans the file fed back by the target system, checks and repairs the dirty data if the dirty data exist, and retransmits the repaired data to the target system, so that the dirty data can be found in time before the data are put in storage, the check result is fed back to the source system so as to be repaired in time, the repaired data is retransmitted to the target system, and adverse effects on data transmission caused by the dirty data are avoided.
It should be clearly understood that this disclosure describes how to make and use particular examples, but the principles of this disclosure are not limited to any details of these examples. Rather, these principles can be applied to many other embodiments based on the teachings of the present disclosure.
Exemplary embodiments of the present disclosure are specifically illustrated and described above. It is to be understood that the present disclosure is not limited to the precise arrangements, instrumentalities, or instrumentalities described herein; on the contrary, the disclosure is intended to cover various modifications and equivalent arrangements included within the spirit and scope of the appended claims.
Claims (7)
1. A method of data transmission, comprising:
before warehousing, checking whether dirty data exists in original data, wherein the original data is data in an original file received from a source system; the checking whether dirty data exists in the original data comprises: establishing a configuration file; verifying the original data according to the configuration file; if the verification result is passed, outputting a mark file passed by the verification; if the check result is failure, outputting a check failure mark file and outputting dirty data obtained by checking; the configuration file comprises at least one piece of configuration information, and each piece of configuration information corresponds to an original file; the configuration information comprises a file name, field number and a data type corresponding to each field;
feeding back the file generated after verification to the source system so that the source system can intervene after scanning the dirty data from the feedback result;
and if the dirty data exists, checking and repairing the original data, and retransmitting the repaired file to a target system.
2. The method of claim 1, wherein the configuration information further comprises a data range for each field, wherein the data range is an enumerated value or a range value.
3. The method of claim 2, wherein verifying the raw data according to the configuration file comprises:
reading and judging whether the number of actual fields in the original data is consistent with the number of fields in the configuration information, whether the actual data type of each field in the original data is consistent with the data type of each field in the configuration information, and whether the actual value of each field in the original data is in the data range of each field in the configuration information;
and when the actual field number in the original data is consistent with the field number in the configuration information, the actual data type of each field in the original data is consistent with the data type of each field in the configuration information, and the actual value of each field in the original data is in the data range of each field in the configuration information, the check result is passed.
4. An apparatus for data transmission, comprising:
the system comprises a verification module, a storage module and a storage module, wherein the verification module is used for verifying whether dirty data exists in original data before warehousing, and the original data is data in an original file received from a source system; the checking module comprises:
the configuration submodule is used for establishing a configuration file; the configuration file comprises at least one piece of configuration information, and each piece of configuration information corresponds to an original file; the configuration information comprises a file name, field number and a data type corresponding to each field;
the verification submodule is used for verifying the original data according to the configuration file; and
the output submodule is used for outputting the verification result of the verification submodule, and outputting a mark file which passes the verification when the verification result is passed; when the checking result is failure, outputting a checking failure mark file and outputting dirty data obtained by checking;
and
the feedback module is used for feeding back the file generated after the inspection to the source system so that the source system can intervene after scanning the dirty data from the feedback result; and if the dirty data exists, checking and repairing the original data, and retransmitting the repaired file to a target system.
5. A system for data transmission, comprising a source system and a target system, wherein the source system transmits an original file to the target system, and the target system comprises:
the verification module is used for verifying whether dirty data exists in the original data of the original file before warehousing; the checking whether dirty data exists in the original data of the original file comprises the following steps: establishing a configuration file; verifying the original data according to the configuration file; if the verification result is passed, outputting a mark file passed by the verification; if the check result is failure, outputting a check failure mark file and outputting dirty data obtained by checking; the configuration file comprises at least one piece of configuration information, and each piece of configuration information corresponds to an original file; the configuration information comprises a file name, field number and a data type corresponding to each field; and
the feedback module is used for feeding back the file generated after the inspection to the source system;
the source system includes:
the scanning module is used for carrying out cyclic scanning and intervening on the dirty data after scanning the feedback result of the feedback module;
the source system further comprises:
and the retransmission module is used for checking and repairing the dirty data after the intervention is carried out on the dirty data, and retransmitting the repaired data to the target system.
6. An electronic device, comprising:
a processor;
a memory storing instructions for the processor to control the method steps of any of claims 1-3.
7. A computer-readable medium having stored thereon computer-executable instructions, which when executed by a processor, perform the method steps of any one of claims 1-3.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610157785.2A CN107203564B (en) | 2016-03-18 | 2016-03-18 | Data transmission method, device and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610157785.2A CN107203564B (en) | 2016-03-18 | 2016-03-18 | Data transmission method, device and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107203564A CN107203564A (en) | 2017-09-26 |
| CN107203564B true CN107203564B (en) | 2020-11-24 |
Family
ID=59904416
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610157785.2A Active CN107203564B (en) | 2016-03-18 | 2016-03-18 | Data transmission method, device and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107203564B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110347483B (en) * | 2018-04-08 | 2021-05-11 | 中兴通讯股份有限公司 | Method and device for migrating physical machine to virtual machine and storage medium |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103873567A (en) * | 2014-03-03 | 2014-06-18 | 北京智谷睿拓技术服务有限公司 | Task-based data transmission method and device |
| CN105223946A (en) * | 2015-09-28 | 2016-01-06 | 上海海洋大学 | The system and method for extreme environment scientific investigation ship Real-time Monitoring Data treatment and monitoring |
Family Cites Families (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20040107203A1 (en) * | 2002-12-03 | 2004-06-03 | Lockheed Martin Corporation | Architecture for a data cleansing application |
| CN100531023C (en) * | 2007-02-15 | 2009-08-19 | 大唐移动通信设备有限公司 | Method and apparatus for condition report feedback |
| CN102073912A (en) * | 2009-11-23 | 2011-05-25 | 中国移动通信集团黑龙江有限公司 | Data quality control method, device and system |
| US8347093B1 (en) * | 2010-07-19 | 2013-01-01 | Google Inc. | Privacy preserving name verification |
| CN102637008A (en) * | 2012-03-26 | 2012-08-15 | 唐山钢铁集团微尔自动化有限公司 | Tracking method of hot rolling basic automation control data security timeliness |
| CN104361064A (en) * | 2014-11-04 | 2015-02-18 | 中国银行股份有限公司 | Data cleaning method for data files and data files processing method |
| CN104462562B (en) * | 2014-12-29 | 2018-05-18 | 浪潮软件集团有限公司 | Data migration system and method based on data warehouse automation |
| CN105389402B (en) * | 2015-12-29 | 2019-04-12 | 曙光信息产业(北京)有限公司 | A kind of ETL method and apparatus towards big data |
-
2016
- 2016-03-18 CN CN201610157785.2A patent/CN107203564B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103873567A (en) * | 2014-03-03 | 2014-06-18 | 北京智谷睿拓技术服务有限公司 | Task-based data transmission method and device |
| CN105223946A (en) * | 2015-09-28 | 2016-01-06 | 上海海洋大学 | The system and method for extreme environment scientific investigation ship Real-time Monitoring Data treatment and monitoring |
Non-Patent Citations (2)
| Title |
|---|
| "ETL集群优化技术研究与实现";李兰友 等;《电脑知识与技术》;20120505;第8卷(第13期);1065-1068 * |
| "Febrl-An open source data cleaning,deduplication and record linkage system with a graphical user interface";Peter Christen;《Acm Sigkdd International Conference on Knowledge Discovery & Data Mining，2008》;20080831;2949-2952 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN107203564A (en) | 2017-09-26 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111353180A (en) | Block chain evidence storing method, evidence obtaining method and system | |
| US10331439B2 (en) | Source code transfer control method, computer program therefor, and recording medium therefor | |
| CN111245934A (en) | Feedback method, device and equipment for file transmission and storage medium | |
| CN103259797B (en) | data file transmission method and platform | |
| CN107423163B (en) | Error surveying method and system for block chain of overall serial added blocks | |
| CN107347048A (en) | The method, apparatus and system of data check | |
| CN107391557B (en) | Block chain serial query method and system for setting out-of-chain fault table | |
| WO2020056458A1 (en) | A block chain-based system for multi-party, multistage process verification | |
| CN105245392A (en) | Method and device for base line checking and repairing | |
| CN108804315B (en) | Test method and device applied to dynamic development, electronic equipment and storage medium | |
| CN111290998A (en) | Method, device and equipment for calibrating migration data and storage medium | |
| CN107203564B (en) | Data transmission method, device and system | |
| CN112114903A (en) | File loading method and device | |
| CN113220312A (en) | Method and system for checking burning data of electronic equipment | |
| CN106897166A (en) | A kind of mobile terminal and its restorative procedure | |
| CN105094959B (en) | It is a kind of that the method and apparatus of reconciliation file are provided | |
| CN115002328A (en) | Camera module data burning fool-proof detection method and system | |
| CN114612101A (en) | Reliable inter-link route cross-link method and system for connection | |
| CN107247614A (en) | The method that application message is checked in software translating | |
| CN113486358A (en) | Vulnerability detection method and device | |
| CN107451007B (en) | Verification method and system of block chain for error correction aiming at added blocks | |
| CN106598690B (en) | Management method and device for codes | |
| CN105812085A (en) | Error correction method and system of audio USB Key | |
| CN114710345B (en) | Authentication characteristic information writing method and system | |
| CN115774881A (en) | Code auditing method, device, equipment and medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |