CN112597250A - Track traffic data relay station implementation method based on DataX data synchronization - Google Patents
Track traffic data relay station implementation method based on DataX data synchronization Download PDFInfo
- Publication number
- CN112597250A CN112597250A CN202011593213.1A CN202011593213A CN112597250A CN 112597250 A CN112597250 A CN 112597250A CN 202011593213 A CN202011593213 A CN 202011593213A CN 112597250 A CN112597250 A CN 112597250A
- Authority
- CN
- China
- Prior art keywords
- data
- file
- datax
- synchronization
- executed
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 35
- 230000001360 synchronised effect Effects 0.000 claims description 18
- 238000004590 computer program Methods 0.000 claims description 8
- 238000006243 chemical reaction Methods 0.000 claims description 7
- 230000005540 biological transmission Effects 0.000 claims description 6
- 238000013523 data management Methods 0.000 abstract description 5
- 230000008569 process Effects 0.000 description 6
- 238000004891 communication Methods 0.000 description 5
- 238000007726 management method Methods 0.000 description 5
- 230000006870 function Effects 0.000 description 4
- 238000012545 processing Methods 0.000 description 4
- 238000010586 diagram Methods 0.000 description 3
- 230000003993 interaction Effects 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 238000010276 construction Methods 0.000 description 1
- 238000007418 data mining Methods 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 230000004069 differentiation Effects 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 238000003032 molecular docking Methods 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 230000008439 repair process Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
- 230000007704 transition Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
- G06F16/275—Synchronous replication
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/28—Databases characterised by their database models, e.g. relational or object models
- G06F16/284—Relational databases
- G06F16/285—Clustering or classification
Abstract
The invention provides a track traffic data middling station implementation method based on DataX data synchronization, which receives json format data through a uniform data interface; carrying out data classification on the json format data and converting the json format data into a file to be executed; the file to be executed accords with a DataX framework identification condition; and synchronizing the file to be executed through the DataX. In this way, the application data of each system can be received, different data sources are merged and perfected, the format is stored uniformly, and the interface standard is unified; the data application cost is reduced, and data service is provided for different business applications by using standard interfaces; the data management capability is improved, the required data can be more conveniently positioned, the data can be understood, and the access authority of the data and the usability of the data can be ensured.
Description
Technical Field
The embodiment of the invention relates to the field of rail transit, in particular to a rail transit data middling station implementation method based on DataX data synchronization.
Background
The rail transit is a complex system integrating multiple specialties and multiple kinds, and is generally composed of a rail road line, stations, vehicles, maintenance and repair bases, power supply and transformation, communication signals, a command and control center and the like. Each specialty becomes an independent system, and data interaction with other dependent systems is needed, so that the purpose of rail transit operation is achieved.
Data in the current rail transit system are respectively stored by the system according to different specialties, so that the reusability of the data in each system is low, and an information island is formed; if the data in the system needs to be multiplexed, different systems need to develop specific interfaces for interaction, so that the cost in the data application process is high; in the application process, because the data structures of the data in different systems are different, unified management cannot be performed, the cost is increased for data management, the data value is difficult to realize finally, and data mining and service innovation are not facilitated.
Disclosure of Invention
According to the embodiment of the invention, a track traffic data middling station implementation scheme based on DataX data synchronization is provided.
In a first aspect of the invention, a track traffic data middling station implementation method based on DataX data synchronization is provided. The method comprises the following steps:
receiving json format data through a uniform data interface;
carrying out data classification on the json format data and converting the json format data into a file to be executed; the file to be executed accords with a DataX framework identification condition;
and synchronizing the file to be executed through the DataX.
Further, the data classification of the json format data includes:
and acquiring configuration content of the corresponding data subject from a database according to the data subject ID of the json format data, and carrying out data classification on the json format data according to the configuration content.
Further, the configuring the content includes:
the target application address, the implementation of the target application receiving the current data, and the data structure of the target application.
Further, the DataX framework identifies conditions comprising:
the data file is a data format text and contains synchronous task information.
Further, still include:
and after the file to be executed is synchronized through the DataX, managing a synchronization result.
Further, the managing the synchronization result includes:
judging whether the file to be executed is successfully synchronized, if so, archiving the successfully synchronized data file to a history record; otherwise, recording the synchronization failure times, and filing the data file with the synchronization failure to the failure record.
Further, the recording the number of synchronization failures triggers the number of failures to be determined, including:
when the synchronization failure times are smaller than the preset failure times, synchronizing the file to be executed again through the DataX, and if the resynchronization is successful, deleting the corresponding data file in the synchronization failure record;
and when the synchronization failure times reach preset times, sending out a synchronization failure message.
In a second aspect of the present invention, a track traffic data middling station implementation apparatus based on DataX data synchronization is provided. The device includes:
the data transmission component is used for receiving json format data through a uniform data interface;
the classification conversion module is used for carrying out data classification on the json format data and converting the json format data into a file to be executed; the file to be executed accords with a DataX framework identification condition;
and the synchronization module is used for synchronizing the file to be executed through the DataX.
In a third aspect of the invention, an electronic device is provided. The electronic device includes: a memory having a computer program stored thereon and a processor implementing the method as described above when executing the program.
In a fourth aspect of the invention, a computer-readable storage medium is provided, on which a computer program is stored which, when being executed by a processor, carries out the method as according to the first aspect of the invention.
It should be understood that the statements herein reciting aspects are not intended to limit the critical or essential features of any embodiment of the invention, nor are they intended to limit the scope of the invention. Other features of the present invention will become apparent from the following description.
The invention can receive the application data of each system, combine and perfect different data sources, store in a unified format and unify interface standards; the data application cost is reduced, and data service is provided for different business applications by using standard interfaces; the data management capability is improved, the required data can be more conveniently positioned, the data can be understood, and the access authority of the data and the usability of the data can be ensured.
Drawings
The above and other features, advantages and aspects of various embodiments of the present invention will become more apparent by referring to the following detailed description when taken in conjunction with the accompanying drawings. In the drawings, like or similar reference characters designate like or similar elements, and wherein:
FIG. 1 is a flow chart illustrating a method for implementing a rail transit data middlebox based on DataX data synchronization according to the present invention;
FIG. 2 is a block diagram of a device for implementing the rail transit data center based on the DataX data synchronization according to the present invention;
FIG. 3 illustrates a block diagram of an exemplary electronic device capable of implementing embodiments of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention. All other embodiments, which can be obtained by a person skilled in the art without any inventive step based on the embodiments of the present invention, are within the scope of the present invention.
In addition, the term "and/or" herein is only one kind of association relationship describing an associated object, and means that there may be three kinds of relationships, for example, a and/or B, which may mean: a exists alone, A and B exist simultaneously, and B exists alone. In addition, the character "/" herein generally indicates that the former and latter related objects are in an "or" relationship.
The invention can receive the application data of each system, combine and perfect different data sources, store in a unified format and unify interface standards; the data application cost is reduced, and data service is provided for different business applications by using standard interfaces; the data management capability is improved, the required data can be more conveniently positioned, the data can be understood, and the access authority of the data and the usability of the data can be ensured.
Fig. 1 shows a flowchart of a track traffic data relay implementation method based on DataX data synchronization according to an embodiment of the present invention.
The method comprises the following steps:
s101, json format data are received through a unified data interface;
as an embodiment of the present invention, a data transmission component of a data center station receives application data of each system through an Http Restful service, where the application data is json format data and includes a data topic and a data content.
"REST" is an abbreviation for "presentation State Transfer," and describes an operation with HTTP verbs (GET, POST, DELETE, PUSH, etc.) for presentation layer State transitions, i.e., URL-located resources; and the API built based on REST is the Restful style. Restful is a typical HTTP-based protocol.
The data center receives application data of each client system or application data of different data sources uploaded by a third party through an Http Restful service, wherein the application data is in a json format and comprises data subjects and data contents in each data. The application data is accessed to the data transmission component of the data center station, and then the application data needs to be unified through a standard interface, so that the data accessed to the data center station is in a json format.
The data theme is a service differentiation of data, and is used for differentiating the service type of the data belonging to specific applications, such as driving data, signal data and the like.
The data content is data containing content, for example, the driving data contains parking precision, early-late point state and the like. And generating the needed jobs and data files when the DataX task is executed through the data content.
Data are collected uniformly through the Http Restful service, data services are provided for different business applications through standard interfaces, and data application cost is reduced.
And S102, carrying out data classification on the json format data and converting the json format data into a file to be executed.
After data in the json format is received, the data needs to be classified and stored or forwarded according to different requirements.
Each data topic uniquely corresponds to an ID, and the ID is used for identifying the data topic.
And analyzing the data theme ID of the application data, matching the data theme ID with the configuration content of the corresponding data theme in the database, and acquiring the corresponding configuration content.
In some embodiments, the database may be any database, depending on the target application. For example, if the database used by the target application is a relational database, the database is also a relational database, such as a MySQL database, which can receive commands and perform corresponding operations.
And identifying the ID of the data theme, searching the corresponding ID from the database according to the ID, and matching corresponding configuration information of the corresponding ID in the database, thereby obtaining the configuration content of the data theme.
The configuration content of the data topic generally comprises:
the target application address, the implementation of the target application receiving the current data, and the data structure of the target application.
The target application address, i.e. the target database address, is used for linking to the target application according to the address. The implementation of the target application receiving the current data, such as a database, a message queue, an http service, etc. And the data structure of the target application is used for defining the content of the json format data.
In some embodiments, the configuration content of the data topic may also include personalized configuration requirements of the target application, if the target application has specific configuration requirements. Such as passwords, http header information, etc.
By storing the data in a classified manner, the application data of different systems can be synchronously stored according to the service types, unified management can be realized, and a data basis is provided for data forwarding.
In this embodiment, further, after classifying the data, according to the configuration content of the data topic, the data content is converted into a data file with a content and format that meet the identification condition of the DataX framework.
The data file with the content and the format conforming to the identification condition of the DataX framework comprises synchronization task information joband a data format text. Wherein jobs is used for describing the synchronization task; the data text is converted data, namely, the data content is converted into the data text through format conversion.
S103, synchronizing the file to be executed through the DataX.
In S102, a file to be executed that meets the DataX framework identification condition is obtained. In S103, the file to be executed is synchronized by calling the synchronization function of DataX. And docking the file to be executed to the DataX, and performing seamless data synchronization with the existing data source through the DataX.
The DataX is an offline synchronization tool for heterogeneous data sources, and is used for realizing a stable and efficient data synchronization function between various heterogeneous data sources including a relational database (MySQL, Oracle and the like), an HDFS, Hive, ODPS, an HBase, an FTP and the like. The DataX adopts the framework construction of frame + plugin, can install different plugins to realize storage of various different forms, and provides a scheme easy to expand for supporting various specific systems.
As an embodiment of the present invention, optionally, after the file to be executed is synchronized by DataX, a synchronization result may be managed.
The synchronization of DataX results in a synchronization result that includes a synchronization success and a synchronization failure. The managing the synchronization result requires first determining whether the synchronization result is successful or failed, and specifically includes:
judging whether the file to be executed is successfully synchronized, if so, archiving the successfully synchronized data file to a history record; otherwise, recording the synchronization failure times, and filing the data file with the synchronization failure to the failure record. The history is used for recording the data files which are successfully synchronized. Namely, when the execution result is that the synchronization is successful, the data file with the successful synchronization is filed to the history record, so that the repeated synchronization of the data is avoided. The failure record is used for recording the data file with synchronization failure. That is, when the execution result is synchronization failure, the data file with synchronization failure is filed in the failure record, and the number of times of synchronization failure of the current data file is updated, that is, the number of times is increased by 1.
In this embodiment, preferably, the recording of the number of synchronization failures triggers a determination process of the number of failures.
The upper limit value of the number of data file synchronization failures is preset in the system. When the synchronization failure times of the data files are smaller than the upper limit value of the preset times, re-executing the DataX synchronization task, and re-synchronizing the data files with the synchronization failure; if the data file is successfully synchronized due to the re-execution of the DataX synchronization task, deleting the corresponding data file in the failure record; data are prevented from missing synchronization.
And when the synchronization times of the data files reach preset times, sending a synchronization failure message. Namely, when the synchronization is unsuccessful repeatedly, people are required to participate in determining the reason, and the unsuccessful data files are cleaned, so that the synchronization is prevented from being missed. The data of different systems are stored, the aims of unified storage and unified management of the data are achieved, interaction of the data among the different systems is further achieved, data service is provided for a third party, and business innovation is achieved on the basis of the data.
According to the embodiment of the invention, the invention can receive the application data of each system, combine and perfect different data sources, store in a unified format and unify interface standards; the data application cost is reduced, and data service is provided for different business applications by using standard interfaces; the data management capability is improved, the required data can be more conveniently positioned, the data can be understood, and the access authority of the data and the usability of the data can be ensured.
It should be noted that, for simplicity of description, the above-mentioned method embodiments are described as a series of acts or combination of acts, but those skilled in the art will recognize that the present invention is not limited by the order of acts, as some steps may occur in other orders or concurrently in accordance with the invention. Further, those skilled in the art should also appreciate that the embodiments described in the specification are exemplary embodiments and that the acts and modules illustrated are not necessarily required to practice the invention.
The above is a description of method embodiments, and the embodiments of the present invention are further described below by way of apparatus embodiments.
As shown in fig. 2, the apparatus 200 includes:
and a data transmission component 210 for receiving json format data through a unified data interface.
The data transmission assembly is used as one of the components of the data center station, receives a large amount of json format data of different service types in each subway line through a special network and sends the json format data to the classification conversion module.
A classification conversion module 220, configured to perform data classification on the json-format data, and convert the json-format data into a file to be executed; and the file to be executed conforms to the DataX framework identification condition.
The classification conversion module 220 further includes:
the classification module 221 is configured to obtain configuration content of a corresponding data topic from a database according to the data topic ID of the json-format data, and perform data classification on the json-format data according to the configuration content. The configuration content comprises a target application address, an implementation mode of the target application for receiving the current data and a data structure of the target application.
A conversion module 222, configured to convert the classified data into a file to be executed; and the file to be executed conforms to the DataX framework identification condition.
The DataX framework identifies conditions comprising:
the data file is a data format text and contains synchronous task information.
A synchronization module 230, configured to synchronize the file to be executed through DataX.
The apparatus 200, further comprising:
and the management module 240 is configured to manage a synchronization result after the to-be-executed file is synchronized by DataX.
The management module 240 is further configured to determine whether the file to be executed is successfully synchronized, and if so, archive the successfully synchronized data file to a history record; otherwise, recording the synchronization failure times, and filing the data file with the synchronization failure to the failure record.
The recording synchronization failure times triggering failure times judgment comprises the following steps:
when the synchronization failure times are smaller than the preset failure times, synchronizing the file to be executed again through the DataX, and if the resynchronization is successful, deleting the corresponding data file in the synchronization failure record;
and when the synchronization failure times reach preset times, sending out a synchronization failure message.
It can be clearly understood by those skilled in the art that, for convenience and brevity of description, the specific working process of the described module may refer to the corresponding process in the foregoing method embodiment, and is not described herein again.
As shown in fig. 3, the electronic device includes a Central Processing Unit (CPU) that can perform various appropriate actions and processes according to computer program instructions stored in a Read Only Memory (ROM) or computer program instructions loaded from a storage unit into a Random Access Memory (RAM). In the RAM, various programs and data required for the operation of the device can also be stored. The CPU, ROM, and RAM are connected to each other via a bus. An input/output (I/O) interface is also connected to the bus.
A plurality of components in an electronic device are connected to an I/O interface, including: an input unit such as a keyboard, a mouse, etc.; an output unit such as various types of displays, speakers, and the like; storage units such as magnetic disks, optical disks, and the like; and a communication unit such as a network card, modem, wireless communication transceiver, etc. The communication unit allows the electronic device to exchange information/data with other devices via a computer network such as the internet and/or various telecommunication networks.
The processing unit executes the respective methods and processes described above, for example, methods S101 to S103. For example, in some embodiments, methods S101-S103 may be implemented as a computer software program tangibly embodied in a machine-readable medium, such as a storage unit. In some embodiments, part or all of the computer program may be loaded and/or installed onto the device via ROM and/or the communication unit. When the computer program is loaded into RAM and executed by the CPU, one or more of the steps of methods S101-S103 described above may be performed. Alternatively, in other embodiments, the CPU may be configured to perform methods S101-S103 by any other suitable means (e.g., by way of firmware).
The functions described herein above may be performed, at least in part, by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: a Field Programmable Gate Array (FPGA), an Application Specific Integrated Circuit (ASIC), an Application Specific Standard Product (ASSP), a system on a chip (SOC), a load programmable logic device (CPLD), and the like.
Program code for implementing the methods of the present invention may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the program codes, when executed by the processor or controller, cause the functions/operations specified in the flowchart and/or block diagram to be performed. The program code may execute entirely on the machine, partly on the machine, as a stand-alone software package partly on the machine and partly on a remote machine or entirely on the remote machine or server.
In the context of the present invention, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
Further, while operations are depicted in a particular order, this should be understood as requiring that such operations be performed in the particular order shown or in sequential order, or that all illustrated operations be performed, to achieve desirable results. Under certain circumstances, multitasking and parallel processing may be advantageous. Likewise, while several specific implementation details are included in the above discussion, these should not be construed as limitations on the scope of the invention. Certain features that are described in the context of separate embodiments can also be implemented in combination in a single implementation. Conversely, various features that are described in the context of a single implementation can also be implemented in multiple implementations separately or in any suitable subcombination.
Although the subject matter has been described in language specific to structural features and/or methodological acts, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are disclosed as example forms of implementing the claims.
Claims (10)
1. A rail transit data relay implementation method based on DataX data synchronization is characterized by comprising the following steps:
receiving json format data through a uniform data interface;
carrying out data classification on the json format data and converting the json format data into a file to be executed; the file to be executed accords with a DataX framework identification condition;
and synchronizing the file to be executed through the DataX.
2. The method of claim 1, wherein the data classifying the json-formatted data comprises:
and acquiring configuration content of the corresponding data subject from a database according to the data subject ID of the json format data, and carrying out data classification on the json format data according to the configuration content.
3. The method of claim 2, wherein configuring the content comprises:
the target application address, the implementation of the target application receiving the current data, and the data structure of the target application.
4. The method of claim 1, wherein the DataX framework identifies conditions comprising:
the data file is a data format text and contains synchronous task information.
5. The method of claim 1, further comprising:
and after the file to be executed is synchronized through the DataX, managing a synchronization result.
6. The method of claim 5, wherein managing synchronization results comprises:
judging whether the file to be executed is successfully synchronized, if so, archiving the successfully synchronized data file to a history record; otherwise, recording the synchronization failure times, and filing the data file with the synchronization failure to the failure record.
7. The method of claim 6, wherein the recording the number of synchronization failures triggers a number of failures determination, comprising:
when the synchronization failure times are smaller than the preset failure times, synchronizing the file to be executed again through the DataX, and if the resynchronization is successful, deleting the corresponding data file in the synchronization failure record;
and when the synchronization failure times reach preset times, sending out a synchronization failure message.
8. A rail transit data middling station realization device based on DataX data synchronization is characterized by comprising:
the data transmission component is used for receiving json format data through a uniform data interface;
the classification conversion module is used for carrying out data classification on the json format data and converting the json format data into a file to be executed; the file to be executed accords with a DataX framework identification condition;
and the synchronization module is used for synchronizing the file to be executed through the DataX.
9. An electronic device comprising a memory and a processor, the memory having stored thereon a computer program, wherein the processor, when executing the program, implements the method of any of claims 1-7.
10. A computer-readable storage medium, on which a computer program is stored, which program, when being executed by a processor, carries out the method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011593213.1A CN112597250A (en) | 2020-12-29 | 2020-12-29 | Track traffic data relay station implementation method based on DataX data synchronization |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011593213.1A CN112597250A (en) | 2020-12-29 | 2020-12-29 | Track traffic data relay station implementation method based on DataX data synchronization |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112597250A true CN112597250A (en) | 2021-04-02 |
Family
ID=75203391
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011593213.1A Pending CN112597250A (en) | 2020-12-29 | 2020-12-29 | Track traffic data relay station implementation method based on DataX data synchronization |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112597250A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115883678A (en) * | 2023-03-08 | 2023-03-31 | 国网瑞嘉(天津)智能机器人有限公司 | Data processing system, method, device, equipment and medium |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107609061A (en) * | 2017-08-28 | 2018-01-19 | 武汉奇米网络科技有限公司 | A kind of method and apparatus of data syn-chronization |
| CN111753012A (en) * | 2020-06-19 | 2020-10-09 | 北京字节跳动网络技术有限公司 | Data synchronization method, device, equipment and storage medium |
| US20210271555A1 (en) * | 2017-06-16 | 2021-09-02 | Ping An Technology (Shenzhen) Co., Ltd. | Traffic data self-recovery processing method, readable storage medium, server and apparatus |
-
2020
- 2020-12-29 CN CN202011593213.1A patent/CN112597250A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20210271555A1 (en) * | 2017-06-16 | 2021-09-02 | Ping An Technology (Shenzhen) Co., Ltd. | Traffic data self-recovery processing method, readable storage medium, server and apparatus |
| CN107609061A (en) * | 2017-08-28 | 2018-01-19 | 武汉奇米网络科技有限公司 | A kind of method and apparatus of data syn-chronization |
| CN111753012A (en) * | 2020-06-19 | 2020-10-09 | 北京字节跳动网络技术有限公司 | Data synchronization method, device, equipment and storage medium |
Non-Patent Citations (2)
| Title |
|---|
| 梁丽琴等: "利用大数据技术进行海量数据治理", 《网络安全技术与应用》 * |
| 田翠姣 等: ""DataX工具在新冠肺炎数据上报中的应用"", 《计算机技术与发展》 * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115883678A (en) * | 2023-03-08 | 2023-03-31 | 国网瑞嘉(天津)智能机器人有限公司 | Data processing system, method, device, equipment and medium |
| CN115883678B (en) * | 2023-03-08 | 2023-06-16 | 国网瑞嘉(天津)智能机器人有限公司 | Data processing system, method, device, equipment and medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111339186B (en) | Workflow engine data synchronization method, device, medium and electronic equipment | |
| CN111459944B (en) | MR data storage method, device, server and storage medium | |
| CN111221793B (en) | Data mining method, platform, computer equipment and storage medium | |
| CN107704550A (en) | File migration method, apparatus and computer-readable recording medium | |
| CN111782690A (en) | Multi-source heterogeneous data aggregation method and device, storage medium and terminal | |
| CN113485962A (en) | Log file storage method, device, equipment and storage medium | |
| CN111338834B (en) | Data storage method and device | |
| CN112015816A (en) | Data synchronization method, device, medium and electronic equipment | |
| CN113032379A (en) | Distribution network operation and inspection-oriented multi-source data acquisition method | |
| CN112597250A (en) | Track traffic data relay station implementation method based on DataX data synchronization | |
| CN114925140A (en) | Data synchronization system | |
| CN106919574B (en) | Method for processing remote synchronous file in real time | |
| CN111782618B (en) | Data synchronization system | |
| CN112328372A (en) | Kubernetes node self-healing method and system | |
| CN115866035A (en) | Multithreading data high-speed pushing method, system, controller and storage medium | |
| CN112990806A (en) | Method and system for dispatch control of transportation equipment | |
| CN111767176A (en) | Method and device for remotely controlling terminal equipment | |
| KR102365743B1 (en) | Apparatus and method for monitoring and controlling equipment status of multiple ground stations | |
| CN115577160A (en) | Production line data acquisition method, device, equipment and medium | |
| CN113905057A (en) | Multi-node data synchronization method, apparatus, and medium for intelligent yard system | |
| CN114647686A (en) | Data processing method, device, equipment, medium and product | |
| CN112667393A (en) | Method and device for building distributed task computing scheduling framework and computer equipment | |
| CN112291312B (en) | ETL data synchronization method and device, electronic equipment and storage medium | |
| US7693846B2 (en) | Data management system and method for data synchronization | |
| CN110166561B (en) | Data processing method, device, system, equipment and medium for wearable equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20210402 |
|
| RJ01 | Rejection of invention patent application after publication |