WO2021179589A1 - Procédé de transmission de données et système de canal de données - Google Patents

Procédé de transmission de données et système de canal de données Download PDF

Info

Publication number
WO2021179589A1
WO2021179589A1 PCT/CN2020/119709 CN2020119709W WO2021179589A1 WO 2021179589 A1 WO2021179589 A1 WO 2021179589A1 CN 2020119709 W CN2020119709 W CN 2020119709W WO 2021179589 A1 WO2021179589 A1 WO 2021179589A1
Authority
WO
WIPO (PCT)
Prior art keywords
data
format
data channel
channel
cluster
Prior art date
Application number
PCT/CN2020/119709
Other languages
English (en)
Chinese (zh)
Inventor
任超
李大虎
Original Assignee
北京旷视科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 北京旷视科技有限公司 filed Critical 北京旷视科技有限公司
Publication of WO2021179589A1 publication Critical patent/WO2021179589A1/fr

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/50—Network services
    • H04L67/56—Provisioning of proxy services
    • H04L67/565—Conversion or adaptation of application format or content
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L12/00—Data switching networks
    • H04L12/66—Arrangements for connecting between networks having differing types of switching systems, e.g. gateways
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/01—Protocols
    • H04L67/10—Protocols in which an application is distributed across nodes in the network

Definitions

  • This application relates to the field of computer technology, in particular to a data transmission method and a data channel system.
  • video image information application systems (hereinafter referred to as application systems) have been widely used, and there are more and more types.
  • the application system usually does not have the function of sending and receiving data, which leads to the inability to transmit data between different application systems, which in turn leads to the inability to coordinate the video image application between the users of the application systems.
  • the application system data standard specification was formulated.
  • the video image information database (hereinafter referred to as the database) is used as the hub of data interaction, and each application system realizes data interaction through the database.
  • the application system mainly focuses on the realization of video image application functions in the early stage of development, and does not support standard specifications and databases. Therefore, when the application system needs to interact with the database for data, it is necessary to spend manpower and material resources on the secondary development of the application system.
  • a data transmission method is provided, the method is applied to a data channel system, the data channel system includes a gateway control module and a data channel cluster, the gateway control module is connected to a database, and the data channel cluster Connecting to the application system, the method includes:
  • the gateway control module receives the first data in the first format sent by the database, and sends the first data to the data channel cluster; the first format is that the first data is stored in the database Format
  • the data channel cluster converts the first format of the first data into a second format, obtains the first data after the format conversion, and sends the first data after the format conversion to the application system;
  • the second format is a format in which the first data is stored in the application system;
  • the data channel cluster receives the second data in the third format sent by the application system, converts the third format of the second data into a fourth format, obtains the second data after the format conversion, and combines
  • the second data after the format conversion is sent to the gateway control module;
  • the third format is the format for storing the second data in the application system, and the fourth format is for storing the first data in the database. 2.
  • the format of the data is the format of the data;
  • the gateway control module sends the second data after the format conversion to the database.
  • the gateway control module includes an application program interface unit, the application program interface unit includes an application program interface corresponding to the first data, and the data channel cluster includes an application program interface corresponding to the first data.
  • the gateway control module receiving the first data in the first format sent by the database and sending the first data to the data channel cluster includes:
  • the application program interface unit receives the first data in the first format sent by the database through the first application program interface, and sends the first data to the first data channel in the data channel cluster .
  • the data channel cluster includes a first data channel corresponding to the first data, and the first data channel includes a first data mapping table and first data corresponding to the first data. driver;
  • the data channel cluster converts the first format of the first data into a second format, obtains the first data after the format conversion, and sends the first data after the format conversion to the application system, include:
  • the first data channel sends the first data after the format conversion to the application system through the first data driver.
  • the data channel cluster includes a second data channel corresponding to the second data, and the second data channel includes a second data mapping table and second data corresponding to the second data. driver;
  • the data channel cluster receives the second data in the third format sent by the application system, converts the third format of the second data into a fourth format, obtains the second data after the format conversion, and combines
  • the second data after the format conversion is sent to the gateway control module, including:
  • the second data channel receives the second data in the third format sent by the application system through the second data driver;
  • the second data channel converts the third format of the second data into a fourth format according to the second data mapping table to obtain the second data after the format conversion, and the second data after the format conversion is obtained.
  • the second data is sent to the gateway control module.
  • the gateway control module includes a gateway unit
  • the gateway control module sending the second data after the format conversion to the database includes:
  • the gateway control module sends the second data after the format conversion to the database through the gateway unit.
  • the data channel system further includes a channel management module, and the method further includes:
  • the channel management module obtains the data driver and the data mapping table corresponding to each data
  • the channel management module creates a data channel corresponding to each data according to a data driver and a data mapping table corresponding to each data to form the data channel cluster.
  • the data channel system includes multiple data channel clusters, and the multiple data channel clusters include a primary data channel cluster and a backup data channel cluster, and the method further includes:
  • the second data channel cluster is switched to the main data channel cluster, the first data channel cluster is the main data channel cluster, and the second data channel cluster is Backup data channel cluster.
  • the data channel system further includes a metadata management module
  • the method further includes:
  • the metadata management module obtains metadata of each data
  • the metadata management module analyzes and processes the metadata of each data to obtain data driver metadata and data mapping table metadata corresponding to each data;
  • the metadata management module performs translation processing on the data driver metadata and data mapping table metadata corresponding to each data to obtain the data driver and data mapping table corresponding to each data, and stores the data corresponding to each data Drive and data mapping table.
  • a data channel system in a second aspect, includes a gateway control module and a data channel cluster, the gateway control module is connected to a database, and the data channel cluster is connected to an application system;
  • the gateway control module is configured to receive first data in a first format sent by the database, and send the first data to the data channel cluster; the first format is that the database stores the The format of the first data;
  • the data channel cluster is configured to convert the first format of the first data into a second format, obtain the first data after the format conversion, and send the first data after the format conversion to the Application system;
  • the second format is a format in which the first data is stored in the application system;
  • the data channel cluster is configured to receive second data in a third format sent by the application system, convert the third format of the second data into a fourth format, and obtain the second data after the format conversion, And send the second data after the format conversion to the gateway control module;
  • the third format is the format in which the second data is stored in the application system, and the fourth format is the format stored in the database The format of the second data;
  • the gateway control module is configured to send the second data after the format conversion to the database.
  • the gateway control module includes an application program interface unit, the application program interface unit includes an application program interface corresponding to the first data, and the data channel cluster includes an application program interface corresponding to the first data.
  • the application program interface unit is configured to receive first data in a first format sent by the database through the first application program interface, and send the first data to the first data in the data channel cluster One data channel.
  • the data channel cluster includes a first data channel corresponding to the first data, and the first data channel includes a first data mapping table and first data corresponding to the first data. driver;
  • the first data channel is configured to convert the first format of the first data into a second format according to the first data mapping table to obtain the first data after the format conversion;
  • the first data channel is used to send the first data after the format conversion to the application system through the first data driver.
  • the data channel cluster includes a second data channel corresponding to the second data, and the second data channel includes a second data mapping table and second data corresponding to the second data. driver;
  • the second data channel is configured to receive the second data in the third format sent by the application system through the second data driver;
  • the second data channel is configured to convert the third format of the second data into a fourth format according to the second data mapping table to obtain the second data after the format conversion, and convert the format The latter second data is sent to the gateway control module.
  • the gateway control module includes a gateway unit
  • the gateway control module is configured to send the second data after the format conversion to the database through the gateway unit.
  • the data channel system further includes a channel management module
  • the channel management module is used to obtain the data driver and the data mapping table corresponding to each data
  • the channel management module is configured to create a data channel corresponding to each data according to a data driver and a data mapping table corresponding to each data to form the data channel cluster.
  • the data channel system includes multiple data channel clusters, and the multiple data channel clusters include a primary data channel cluster and a backup data channel cluster;
  • the channel management module is configured to switch the second data channel cluster to the main data channel cluster when the failure of the first data channel cluster is detected, the first data channel cluster is the main data channel cluster, and the second data channel cluster is the main data channel cluster.
  • the channel cluster is a backup data channel cluster.
  • the data channel system further includes a metadata management module
  • the metadata management module is used to obtain metadata of each data
  • the metadata management module is configured to analyze and process the metadata of each data to obtain data driver metadata and data mapping table metadata corresponding to each data;
  • the metadata management module is used to translate the data driver metadata and data mapping table metadata corresponding to each data to obtain the data driver and data mapping table corresponding to each data, and to store the data Corresponding data driver and data mapping table.
  • a computer device including a memory and a processor, the memory stores a computer program that can run on the processor, and when the processor executes the computer program, any one of the first aspect is implemented The steps of the method described in item.
  • a computer-readable storage medium on which a computer program is stored, and the computer program, when executed by a processor, implements the steps of any one of the methods in the first aspect.
  • the data channel system includes a gateway control module and a data channel cluster.
  • the gateway control module is connected to the database, and the data channel cluster is connected to the application system.
  • the gateway control module receives the first data in the first format sent by the database, and sends the first data to the data channel cluster.
  • the first format is a format in which the first data is stored in the database.
  • the data channel cluster converts the first format of the first data into the second format, obtains the first data after the format conversion, and sends the first data after the format conversion to the application system.
  • the second format is a format in which the first data is stored in the application system.
  • the data channel cluster receives the second data in the third format sent by the application system, converts the third format of the second data into the fourth format, obtains the second data after the format conversion, and sends the second data after the format conversion to Gateway control module.
  • the third format is a format for storing the second data in the application system
  • the fourth format is a format for storing the second data in the database.
  • the gateway control module sends the converted second data to the database. In this way, there is no need to spend manpower and material resources on the secondary development of the application system.
  • By setting a data channel system between the application system and the database data interaction between the application system and the database is realized, and the database is the hub to realize the interaction between different application systems. .
  • FIG. 1 is an architecture diagram of an application environment provided by an embodiment of the application
  • FIG. 2 is a flowchart of a data transmission method provided by an embodiment of this application.
  • FIG. 3 is a flowchart of a method for creating a data channel cluster provided by an embodiment of the application
  • FIG. 4 is a flowchart of a metadata management method provided by an embodiment of the application.
  • FIG. 5A is a flowchart of an example of a data transmission method provided by an embodiment of this application.
  • 5B is a flowchart of an example of a data transmission method provided by an embodiment of this application.
  • FIG. 6 is a schematic structural diagram of a data channel system provided by an embodiment of this application.
  • FIG. 7 is a schematic structural diagram of a computer device provided by an embodiment of this application.
  • FIG. 1 is an architecture of an application environment provided by an embodiment of the application. picture.
  • the application environment includes a database 110, a data channel system 120 and an application system 130.
  • the database 110 and the application system 130 are connected through a data channel system 120.
  • the database 110 is used to store video image data resources (such as face information, vehicle information, alarm information, collection equipment information, etc.) of the application system based on a standard format (such as GAT1400 format); the data channel system 120 is used to check the database
  • the video image data transmitted between 110 and the application system 130 undergoes format conversion processing; the application system 130 is used to obtain the video image data resources uploaded by other application systems 130 from the database 110 through the data channel system 120, or through the data channel system 120 uploads the locally stored video image data resource to the database 110 so that other application systems 130 can obtain the video image data resource from the database 110.
  • the data channel system includes a gateway control module and a data channel cluster.
  • the gateway control module is connected to the database
  • the data channel cluster is connected to the application system.
  • Step 201 The gateway control module receives the first data in the first format sent by the database, and sends the first data to the data channel cluster.
  • the first format is a format in which the first data is stored in the database.
  • the database stores various types of data uploaded by each application system (such as snapshot data, alarm data, equipment data, etc.), and different types of data correspond to
  • the data in the first format is recorded as the first data.
  • the application system may send a data request to the database.
  • the data request carries the address of the first data.
  • the database can send the first data in the first format to the data channel system.
  • the first format is a format in which the first data is stored in the database.
  • the gateway control module in the data channel system can receive the first data in the first format sent by the database, and send the first data to the data channel cluster.
  • the gateway control module includes an application program interface unit, the application program interface unit includes an application program interface corresponding to the first data, and the data channel cluster includes a first data channel corresponding to the first data.
  • the gateway control module receives the first data in the first format sent by the database, and sends the first data to the data channel cluster.
  • the processing process is that the application program interface unit receives the first data in the first format sent by the database through the first application program interface. , And send the first data to the first data channel in the data channel cluster.
  • the gateway control module includes an application program interface unit.
  • the application program interface unit includes application program interfaces for receiving different types of data (that is, one application program interface in the application program interface unit corresponds to one type of data).
  • the data channel cluster includes data channels for processing different types of data (that is, one data channel in the data channel cluster corresponds to one type of data). Therefore, a type of data corresponds to an application program interface in the application program interface unit, and corresponds to a data channel in the data channel cluster (for example, snapshot data corresponds to the snapshot data channel, alarm data corresponds to the alarm data channel, and device data corresponds to the device data Channels, etc.).
  • the application program interface unit includes the first application program interface for the first data
  • the data channel cluster includes the first data channel corresponding to the first data.
  • the application program interface unit in the gateway control module can receive the first data in the first format sent by the database through the first application program interface. Then, the application program interface unit may send the first data to the first data channel in the data channel cluster.
  • Step 202 The data channel cluster converts the first format of the first data into the second format, obtains the first data after the format conversion, and sends the first data after the format conversion to the application system.
  • the second format is a format in which the first data is stored in the application system.
  • the application system since the application system mainly focuses on the realization of the video image application function in the early stage of development, it does not support the standard specification, that is, the format of the first data stored in the application system and the database is not necessarily the same. Therefore, in order to ensure that the application system can store and process the first data, after receiving the first data, the data channel cluster can further convert the first format of the first data into the second format to obtain the first data after the format conversion.
  • the second format is a format in which the first data is stored in the application system.
  • the data channel cluster can send the first data after the format conversion to the application system.
  • the application system can receive the first data in the second format for storage and processing. In this way, there is no need to spend manpower and material resources on the secondary development of the application system, and data interaction between the application system and the database is realized by setting a data channel system between the application system and the database.
  • the data channel cluster includes a first data channel corresponding to the first data, and the first data channel includes a first data mapping table and a first data driver corresponding to the first data.
  • the data channel cluster converts the first format of the first data into the second format, obtains the first data after the format conversion, and sends the first data after the format conversion to the application system.
  • the processing process is as follows.
  • Step 1 The first data channel converts the first format of the first data into the second format according to the first data mapping table to obtain the first data after the format conversion.
  • the storage methods of different types of data in the application system are also different, and the data drivers corresponding to different storage methods are also different (such as Representational State Transfer, REST )
  • the interface corresponds to the HyperText Transfer Protocol (HTTP) client
  • the database corresponds to the database connection pool
  • the message middleware corresponds to the consumer listener). Therefore, the data channel cluster contains data channels for processing different types of data.
  • the data channel includes a data mapping table for format conversion of different types of data and a data driver for sending different types of data.
  • the data channel cluster includes a first data channel corresponding to the first data, and the first data channel includes a first data mapping table and a first data driver corresponding to the first data.
  • the first data channel can convert the first format of the first data into the second format according to the first data mapping table to obtain the first data after the format conversion.
  • Step 2 The first data channel sends the converted first data to the application system through the first data driver.
  • the first data after the format conversion may be sent to the application system through the first data driver.
  • the application system can receive the first data in the second format for storage and processing.
  • Step 203 The data channel cluster receives the second data in the third format sent by the application system, converts the third format of the second data into the fourth format, obtains the second data after the format conversion, and converts the second data after the format conversion.
  • the data is sent to the gateway control module.
  • the third format is a format for storing the second data in the application system
  • the fourth format is a format for storing the second data in the database.
  • the application system can send the second format of the third format to the data channel cluster in the data channel system.
  • the third format is a format for storing the second data in the application system.
  • the data channel cluster in the data channel system can receive the second data in the third format sent by the application system. Because the format of the second data stored in the application system and the database is not necessarily the same. Therefore, in order to ensure that the database can store the second data, the data channel cluster may further convert the third format of the second data into the fourth format to obtain the second data after the format conversion.
  • the fourth format is a format for storing the second data in the database. After that, the data channel cluster can send the second data after the format conversion to the gateway control module.
  • the data channel cluster includes a second data channel corresponding to the second data
  • the second data channel includes a second data mapping table and a second data driver corresponding to the second data.
  • the data channel cluster receives the second data in the third format sent by the application system, converts the third format of the second data into the fourth format, obtains the second data after the format conversion, and sends the second data after the format conversion to
  • the processing procedure of the gateway control module is as follows.
  • Step 1 The second data channel receives the second data in the third format sent by the application system through the second data driver.
  • the storage methods of different types of data in the application system are different, and the data drivers corresponding to different storage methods are also different (for example, Representational State Transfer, REST)
  • the interface corresponds to the HyperText Transfer Protocol (HTTP) client
  • the database corresponds to the database connection pool
  • the message middleware corresponds to the consumer listener). Therefore, the data channel cluster contains data channels for processing different types of data.
  • the data channel includes a data mapping table for format conversion of different types of data and a data driver for receiving different types of data.
  • the data channel cluster includes a second data channel corresponding to the second data
  • the second data channel includes a second data mapping table and a second data driver corresponding to the second data.
  • the application system can send the second data in the third format to the second data driver of the second data channel in the data channel cluster.
  • the second data channel can receive the second data in the third format sent by the application system through the second data driver.
  • Step 2 The second data channel converts the third format of the second data into the fourth format according to the second data mapping table to obtain the second data after the format conversion, and sends the second data after the format conversion to the gateway control module .
  • the second data channel after the second data channel receives the second data in the third format sent by the application system, it can convert the third format of the second data into the fourth format according to the second data mapping table to obtain the converted format The second data. Then, the second data channel may send the second data after the format conversion to the gateway control module.
  • Step 204 The gateway control module sends the converted second data to the database.
  • the gateway control module after the gateway control module receives the format-converted second data, it can send the format-converted second data to the database.
  • the database can receive the second data in the fourth format to store the second data in the fourth format. In this way, there is no need to spend manpower and material resources on the secondary development of the application system, and data interaction between the application system and the database is realized by setting a data channel system between the application system and the database.
  • the gateway control module includes a gateway unit.
  • the processing process of the gateway control module sending the second data after the format conversion to the database is that the gateway control module sends the second data after the format conversion to the database through the gateway unit.
  • the gateway control module also includes a gateway unit for sending data to the database. After the gateway control module receives the format-converted second data, it can send the format-converted second data to the database through the gateway unit.
  • step 101 and step 102 are processes for the application system to obtain the first data from the database through the data channel system
  • steps 103 and 104 are processes for the application system to upload the second data to the database through the data channel.
  • the processing sequence of the above two processes does not distinguish the sequence.
  • the data channel system further includes a channel management module, and the channel management module is used to create a data channel cluster.
  • the specific processing process is as follows:
  • Step 301 The channel management module obtains the data driver and the data mapping table corresponding to each data.
  • data drivers and data mapping tables corresponding to different types of data are also different (that is, one type of data corresponds to one data driver and one data mapping table).
  • Data drivers and data mapping tables corresponding to various types of data can be pre-stored in the data channel system. Among them, the generation process of the data driver and the data mapping table corresponding to each type of data will be described in detail later.
  • the channel management module can obtain the data corresponding to each type from the data channel system pre-stored with data drivers and data mapping tables corresponding to each type of data. Data drive and data mapping table.
  • the channel management module can create a data channel cluster when the data channel system is initialized, or it can create a data channel cluster according to a user's instruction, which is not limited in this application.
  • the channel management module creates a data channel corresponding to each data according to the data driver and the data mapping table corresponding to each data to form a data channel cluster.
  • the channel management module since one type of data corresponds to a program interface in the application program interface unit, and a data channel in the data channel cluster, the channel management module obtains the data driver and data map corresponding to each type of data. After the table, for each type of data, the channel management module can create a data channel corresponding to the type of data according to the data driver and data mapping table corresponding to the type of data, so as to obtain the data channel corresponding to each type of data. Data channel cluster.
  • the channel management module can also open a data channel in the data channel cluster, pause operation, filter setting, traffic monitoring setting, destroy operation, re-creation operation, and data channel cluster copy operation according to the user's instructions. .
  • the channel management module can also provide a fuse protection strategy for the data channel when any one of the application system and the database is down abnormally.
  • the data channel system includes multiple data channel clusters, and the multiple data channel clusters include a main data channel cluster and a backup data channel cluster.
  • the channel management module can
  • the second data channel cluster is switched to the primary data channel cluster, the first data channel cluster is the primary data channel cluster, and the second data channel cluster is the backup data channel cluster.
  • the data channel system further includes a metadata management module.
  • the metadata management module can generate the drive and data mapping table corresponding to each data according to the metadata of each data. As shown in Figure 4, the specific process is as follows.
  • Step 401 The metadata management module obtains metadata of each data.
  • the data channel system may pre-store metadata of each view data resource (that is, each data) in the application system.
  • metadata is a method used to describe the acquisition of various view data resources (such as face information, vehicle information, alarm information, collection equipment information, etc.) in the application system (such as REST interface, message middleware, database, big data, etc.) Warehouse, etc.) data rules; metadata includes two parts: data sources (resources) and field mappings (mappings).
  • technicians can describe and configure the metadata of each data in XML (an extensible markup language) or JSON (a programming language) in advance.
  • the metadata of the captured data resource includes the mapping relationship between the captured data source and the captured field.
  • step 402 the metadata management module parses the metadata of each data to obtain data driver metadata and data mapping table metadata corresponding to each data.
  • the metadata management module can analyze and process the metadata of the data to obtain the data driver metadata and data corresponding to the data.
  • Data mapping table metadata corresponds to the data source in the metadata
  • the data mapping table metadata corresponds to the field mapping relationship in the metadata.
  • Step 403 The metadata management module translates the data driver metadata and data mapping table metadata corresponding to each data to obtain the data driver and data mapping table corresponding to each data, and stores the data driver and data mapping table corresponding to each data.
  • the metadata management module can the data driver metadata and data mapping table metadata corresponding to the data Perform translation processing to obtain the data driver and data mapping table corresponding to the data. Then, the metadata management module can store the data driver and data mapping table corresponding to each data locally.
  • the metadata management module can store the data driver and data mapping table corresponding to each data in ETCD (an open source, high-availability, high-concurrency distributed key-value storage system), so as to ensure the creation of data channels The data driver and data mapping table will not be lost, thereby improving the stability of the channel system.
  • ETCD an open source, high-availability, high-concurrency distributed key-value storage system
  • the data channel corresponding to each type of data can be deployed in multiple instances. Through the management of the data driver and data mapping table of the data through ETCD, the data idempotence between multiple instances can be ensured.
  • FIG. 5A is a flowchart of an example of a data transmission method provided by an embodiment of the application.
  • the snapshot data corresponds to the snapshot application program interface in the application program interface unit
  • the data channel cluster corresponds to the snapshot data channel.
  • the snapshot data channel includes the snapshot data driver and the snapshot data mapping table.
  • the processing procedure includes the following steps.
  • Step 511 The application program interface unit in the gateway control module receives the snapshot data in the first format sent by the database through the snapshot application program interface.
  • the first format is the format for storing the snapshot data in the database.
  • Step 512 The application program interface unit in the gateway control module sends the snapshot data in the first format to the snapshot data channel in the data channel cluster.
  • the captured data channel converts the first format of the captured data into the second format through the captured data mapping table to obtain the captured captured data after the format conversion.
  • the second format is a format for storing snapshot data in the application system.
  • the snapshot data channel sends the format-converted snapshot data to the application system through the snapshot data driver.
  • step 511 to step 514 The processing procedure from step 511 to step 514 and the processing procedure type from step 201 to step 202 are not repeated here.
  • FIG. 5B is a flowchart of an example of a data transmission method provided by an embodiment of the application.
  • the snapshot data is taken as an example.
  • the snapshot data corresponds to the snapshot data channel in the data channel cluster.
  • the snapshot data channel includes the snapshot data driver and the snapshot data mapping table.
  • the processing procedure includes the following steps.
  • Step 521 The snapshot data channel in the data channel cluster receives the snapshot data in the second format sent by the application system through the snapshot data driver.
  • the second format is a format for storing snapshot data in the application system.
  • Step 522 The captured data channel converts the second format of the captured data into the first format through the captured data mapping table to obtain the captured captured data after the format conversion.
  • the first format is the format for storing the snapshot data in the database.
  • Step 523 The snapshot data channel sends the format-converted snapshot data to the gateway unit in the gateway control module.
  • step 524 the gateway control module sends the converted snapshot data to the database through the gateway unit.
  • step 521 to step 524 The processing procedure from step 521 to step 524 and the processing procedure type from step 203 to step 204 are not repeated here.
  • the embodiment of the present application provides a data transmission method.
  • This method is applied to the data channel system.
  • the data channel system includes a gateway control module and a data channel cluster.
  • the gateway control module is connected to the database, and the data channel cluster is connected to the application system.
  • the gateway control module receives the first data in the first format sent by the database, and sends the first data to the data channel cluster.
  • the first format is a format in which the first data is stored in the database.
  • the data channel cluster converts the first format of the first data into the second format, obtains the first data after the format conversion, and sends the first data after the format conversion to the application system.
  • the second format is a format in which the first data is stored in the application system.
  • the data channel cluster receives the second data in the third format sent by the application system, converts the third format of the second data into the fourth format, obtains the second data after the format conversion, and sends the second data after the format conversion to Gateway control module.
  • the third format is a format for storing the second data in the application system
  • the fourth format is a format for storing the second data in the database.
  • the gateway control module sends the converted second data to the database. In this way, there is no need to spend manpower and material resources on the secondary development of the application system.
  • By setting a data channel system between the application system and the database data interaction between the application system and the database is realized, and the database is the hub to realize the interaction between different application systems. .
  • the data channel system includes a gateway control module 610 and a data channel cluster 620.
  • the gateway control module 610 is connected to a database, and the data channel cluster 620 is connected to the application system;
  • the gateway control module 610 is configured to receive first data in a first format sent by the database, and send the first data to the data channel cluster 620; the first format is stored in the database The format of the first data;
  • the data channel cluster 620 is configured to convert the first format of the first data into a second format, obtain the first data after the format conversion, and send the first data after the format conversion to the The application system;
  • the second format is the format in which the first data is stored in the application system;
  • the data channel cluster 620 is configured to receive second data in a third format sent by the application system, convert the third format of the second data into a fourth format, and obtain the second data after the format conversion , And send the second data after the format conversion to the gateway control module 610;
  • the third format is the format in which the second data is stored in the application system, and the fourth format is the database The format in which the second data is stored in;
  • the gateway control module 610 is configured to send the second data after the format conversion to the database.
  • the gateway control module 610 includes an application program interface unit, the application program interface unit includes an application program interface corresponding to the first data, and the data channel cluster 620 includes the first The first data channel corresponding to the data;
  • the application program interface unit is configured to receive the first data in the first format sent by the database through the first application program interface, and send the first data to the data channel cluster 620.
  • the first data channel is configured to receive the first data in the first format sent by the database through the first application program interface, and send the first data to the data channel cluster 620.
  • the first data channel is configured to receive the first data in the first format sent by the database through the first application program interface, and send the first data to the data channel cluster 620.
  • the first data channel is configured to receive the first data in the first format sent by the database through the first application program interface
  • the data channel cluster 620 includes a first data channel corresponding to the first data, and the first data channel includes a first data mapping table and a first data mapping table corresponding to the first data.
  • the first data channel is configured to convert the first format of the first data into a second format according to the first data mapping table to obtain the first data after the format conversion;
  • the first data channel is used to send the first data after the format conversion to the application system through the first data driver.
  • the data channel cluster 620 includes a second data channel corresponding to the second data, and the second data channel includes a second data mapping table and a second data mapping table corresponding to the second data.
  • the second data channel is configured to receive the second data in the third format sent by the application system through the second data driver;
  • the second data channel is configured to convert the third format of the second data into a fourth format according to the second data mapping table to obtain the second data after the format conversion, and convert the format The latter second data is sent to the gateway control module 610.
  • the gateway control module 610 includes a gateway unit
  • the gateway control module 610 is configured to send the second data after the format conversion to the database through the gateway unit.
  • the data channel system further includes a channel management module 630;
  • the channel management module 630 is used to obtain the data driver and data mapping table corresponding to each data
  • the channel management module 630 is configured to create a data channel corresponding to each data according to a data driver and a data mapping table corresponding to each data to form the data channel cluster 620.
  • the data channel system includes multiple data channel clusters, and the multiple data channel clusters include a primary data channel cluster and a backup data channel cluster;
  • the channel management module 630 is configured to switch the second data channel cluster to the primary data channel cluster when the failure of the first data channel cluster is detected, the first data channel cluster is the primary data channel cluster, and the second data channel cluster is the primary data channel cluster.
  • the data channel cluster is a backup data channel cluster.
  • the data channel system further includes a metadata management module 640;
  • the metadata management module 640 is used to obtain metadata of each data
  • the metadata management module 640 is configured to analyze and process the metadata of each data to obtain data driver metadata and data mapping table metadata corresponding to each data;
  • the metadata management module 640 is configured to perform translation processing on the data driver metadata and data mapping table metadata corresponding to each data to obtain the data driver and data mapping table corresponding to each data, and to store the data The data driver and data mapping table corresponding to the data.
  • the embodiment of the present application provides a data channel system.
  • the data channel system includes a gateway control module and a data channel cluster.
  • the gateway control module is connected to the database, and the data channel cluster is connected to the application system.
  • the gateway control module receives the first data in the first format sent by the database, and sends the first data to the data channel cluster.
  • the first format is a format in which the first data is stored in the database.
  • the data channel cluster converts the first format of the first data into the second format, obtains the first data after the format conversion, and sends the first data after the format conversion to the application system.
  • the second format is a format in which the first data is stored in the application system.
  • the data channel cluster receives the second data in the third format sent by the application system, converts the third format of the second data into the fourth format, obtains the second data after the format conversion, and sends the second data after the format conversion to Gateway control module.
  • the third format is a format for storing the second data in the application system
  • the fourth format is a format for storing the second data in the database.
  • the gateway control module sends the converted second data to the database. In this way, there is no need to spend manpower and material resources on the secondary development of the application system.
  • By setting a data channel system between the application system and the database data interaction between the application system and the database is realized, and the database is the hub to realize the interaction between different application systems. .
  • a computer device including a memory and a processor, the memory stores a computer program that can run on the processor, and the processor executes the computer program When realizing the above-mentioned data transmission method.
  • a computer-readable storage medium on which a computer program is stored, and the computer program is executed by a processor to realize the above-mentioned data transmission method.
  • Non-volatile memory may include read only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory.
  • Volatile memory may include random access memory (RAM) or external cache memory.
  • RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous chain Channel (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Computer And Data Communications (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

L'invention concerne un procédé de transmission de données et un système de canal de données Le procédé s'applique à un système de canal de données. Le procédé comprend les étapes suivantes : un module de commande de passerelle reçoit des premières données dans un premier format envoyé par une base de données, puis envoie les premières données à un groupe de canaux de données; le groupe de canaux de données convertit le premier format des premières données dans un second format pour obtenir des premières données soumises à une conversion de format, puis envoie les premières données soumises à une conversion de format à un système d'application; le groupe de canaux de données reçoit des secondes données dans un troisième format envoyées par le système d'application, convertit le troisième format des secondes données dans un quatrième format pour obtenir des secondes données soumises à une conversion de format, puis envoie les secondes données soumises à une conversion de format au module de commande de passerelle; le module de commande de passerelle envoie les secondes données soumises à une conversion de format à une base de données. L'utilisation de l'invention permet d'obtenir une interaction de données entre un système d'application et une base de données.
PCT/CN2020/119709 2020-03-13 2020-09-30 Procédé de transmission de données et système de canal de données WO2021179589A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202010174731.3A CN111405035A (zh) 2020-03-13 2020-03-13 数据的传输方法及数据通道系统
CN202010174731.3 2020-03-13

Publications (1)

Publication Number Publication Date
WO2021179589A1 true WO2021179589A1 (fr) 2021-09-16

Family

ID=71432507

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/119709 WO2021179589A1 (fr) 2020-03-13 2020-09-30 Procédé de transmission de données et système de canal de données

Country Status (2)

Country Link
CN (1) CN111405035A (fr)
WO (1) WO2021179589A1 (fr)

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111405035A (zh) * 2020-03-13 2020-07-10 北京旷视科技有限公司 数据的传输方法及数据通道系统
CN115103021B (zh) * 2022-05-18 2023-03-24 慧之安信息技术股份有限公司 基于gat 1400协议的网关接入管理方法
CN115103224B (zh) * 2022-06-07 2023-04-25 慧之安信息技术股份有限公司 支持gat 1400协议的视频智能分析方法
CN115037794A (zh) * 2022-06-07 2022-09-09 抖音视界(北京)有限公司 一种数据采集方法、装置、计算机设备及可读存储介质

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1448853A (zh) * 2002-04-01 2003-10-15 上海网上乐园信息技术有限公司 同网异构分布式信息处理系统及其实现方法
CN101754221A (zh) * 2008-12-19 2010-06-23 中国移动通信集团山东有限公司 异构系统间的数据传输方法及数据传输系统
US20180337889A1 (en) * 2017-05-19 2018-11-22 Vmware, Inc. Varying encryption level of traffic through network tunnels
CN109525539A (zh) * 2017-09-20 2019-03-26 中移（杭州）信息技术有限公司 一种能力调用方法、调用回执方法、信息处理方法及系统
CN110162571A (zh) * 2019-04-26 2019-08-23 厦门市美亚柏科信息股份有限公司 一种异构数据库之间数据同步的系统、方法、存储介质
CN111405035A (zh) * 2020-03-13 2020-07-10 北京旷视科技有限公司 数据的传输方法及数据通道系统

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7096033B1 (en) * 2000-07-18 2006-08-22 Koninkiljke Phillips Electronics N.V. Mobile apparatus enabling inter-network communication
US10325225B2 (en) * 2012-05-09 2019-06-18 Nasdaq Technology Ab Methods and arrangements for an automated exchange system
CN105094707B (zh) * 2015-08-18 2018-03-13 华为技术有限公司 一种数据存储、读取方法及装置
CN105743709A (zh) * 2016-04-05 2016-07-06 浪潮电子信息产业股份有限公司 一种管理装置、整机柜管理系统和管理方法

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1448853A (zh) * 2002-04-01 2003-10-15 上海网上乐园信息技术有限公司 同网异构分布式信息处理系统及其实现方法
CN101754221A (zh) * 2008-12-19 2010-06-23 中国移动通信集团山东有限公司 异构系统间的数据传输方法及数据传输系统
US20180337889A1 (en) * 2017-05-19 2018-11-22 Vmware, Inc. Varying encryption level of traffic through network tunnels
CN109525539A (zh) * 2017-09-20 2019-03-26 中移（杭州）信息技术有限公司 一种能力调用方法、调用回执方法、信息处理方法及系统
CN110162571A (zh) * 2019-04-26 2019-08-23 厦门市美亚柏科信息股份有限公司 一种异构数据库之间数据同步的系统、方法、存储介质
CN111405035A (zh) * 2020-03-13 2020-07-10 北京旷视科技有限公司 数据的传输方法及数据通道系统

Also Published As

Publication number Publication date
CN111405035A (zh) 2020-07-10

Similar Documents

Publication Publication Date Title
WO2021179589A1 (fr) Procédé de transmission de données et système de canal de données
US9348665B2 (en) Mapping messages between web services
US20070260750A1 (en) Adaptable data connector
US10372929B1 (en) Secure file transfer and notification server
US10419822B2 (en) Method, device, and system for switching at a mobile terminal of a smart television and acquiring information at a television terminal
US20140188802A1 (en) Pull and paste
US10708364B2 (en) Internet of things information system
AU2014403138A1 (en) Terminal, server, and terminal control method
WO2016145981A1 (fr) Système et procédé de transmission de données
US8970870B2 (en) Delivery of scan services over a device service port
US20110264753A1 (en) Method for providing message and device therefor
WO2021082844A1 (fr) Procédé, appareil, système, dispositif et support de stockage pour la gestion de dispositifs
US20140211248A1 (en) Web server apparatus, control method, and program therefor
CN112688952A (zh) 消息处理方法、装置、射频拉远单元和介质
US20150074794A1 (en) System for accessing data of cloud database using transparent technology
US9686361B2 (en) Communication device, communication system, communication method and computer program product
US10334004B2 (en) System and method for networked video capture, backup, and serving
US9137289B2 (en) Facilitating a stateless transmission of data in an information technology system
CN111212043A (zh) 一种多媒体文件的生成方法和装置
US20220052902A1 (en) Method for managing remote device through management device
CN112468886B (zh) 组播数据转发方法、装置、设备及可读存储介质
CN116095111A (zh) 数据采集设备的配置方法、装置以及物联网平台服务器
US9667724B2 (en) Cloud off-ramp management
CN103067419A (zh) 一种分布式文件系统及在该系统中控制文件存储的方法
CN111027300A (zh) 一种自定义生成报表的方法及装置

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20923773

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20923773

Country of ref document: EP

Kind code of ref document: A1