CN111711639A - Terminal, data transmission method, system, and computer-readable storage medium - Google Patents

Terminal, data transmission method, system, and computer-readable storage medium Download PDF

Info

Publication number
CN111711639A
CN111711639A CN202010604477.6A CN202010604477A CN111711639A CN 111711639 A CN111711639 A CN 111711639A CN 202010604477 A CN202010604477 A CN 202010604477A CN 111711639 A CN111711639 A CN 111711639A
Authority
CN
China
Prior art keywords
data
terminal
piece
preset
determining
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202010604477.6A
Other languages
Chinese (zh)
Other versions
CN111711639B (en
Inventor
郑正猛
李建峰
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
WeBank Co Ltd
Original Assignee
WeBank Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by WeBank Co Ltd filed Critical WeBank Co Ltd
Priority to CN202010604477.6A priority Critical patent/CN111711639B/en
Publication of CN111711639A publication Critical patent/CN111711639A/en
Application granted granted Critical
Publication of CN111711639B publication Critical patent/CN111711639B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L69/00—Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass
    • H04L69/04—Protocols for data compression, e.g. ROHC
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/01—Protocols
    • H04L67/10—Protocols in which an application is distributed across nodes in the network
    • H04L67/1095—Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D30/00—Reducing energy consumption in communication networks
    • Y02D30/70—Reducing energy consumption in communication networks in wireless communication networks

Abstract

The invention relates to the technical field of financial science and technology, and discloses a terminal, a data transmission method, a data transmission system and a computer readable storage medium. The data transmission method comprises the following steps: the method comprises the steps that a first terminal obtains various pieces of first data to be sent; determining the type of each piece of first data, and extracting a value field of a field name in each piece of first data; arranging the type of each piece of first data and each value field in a preset sequence to generate second data corresponding to each piece of first data; and sending each piece of the second data to a middleware so that the middleware broadcasts each piece of the second data to each second terminal. The invention can solve the problem of poor operation stability of the terminal.

Description

Terminal, data transmission method, system, and computer-readable storage medium
Technical Field
The present invention relates to the field of financial technology (Fintech), and in particular, to a terminal, a data transmission method, a data transmission system, and a computer-readable storage medium.
Background
With the development of computer technology, more and more technologies are applied in the financial field, and the traditional financial industry is gradually changing to financial technology (Fintech), but higher requirements are also put forward on the technologies due to the requirements of the financial industry on safety and real-time performance.
In a distributed system, data is distributed on each node in the system, and data may be coupled between nodes, so that the coupled data needs to be synchronized, and after one node modifies cache data, the changed data needs to be synchronized to other nodes in a cluster, thereby ensuring the consistency of the data of each node. When the node transmits a large amount of cache, a lot of network overhead is spent, the response of an application program on the node becomes slow, and even the node is stuck for a long time, so that the operation stability of the terminal serving as the node is poor.
Disclosure of Invention
The invention mainly aims to provide a terminal, a data transmission method, a data transmission system and a computer readable storage medium, and aims to solve the problem of poor operation stability of the terminal.
In order to achieve the above object, the present invention provides a data transmission method, including:
the method comprises the steps that a first terminal obtains various pieces of first data to be sent;
determining the type of each piece of first data, and extracting a value field of a field name in each piece of first data;
arranging the type of each piece of first data and each value field in a preset sequence to generate second data corresponding to each piece of first data;
and sending each piece of the second data to a middleware so that the middleware broadcasts each piece of the second data to each second terminal.
In an embodiment, the step of sending each piece of the second data to the middleware includes:
putting each piece of second data into each data sending queue, wherein the difference value between the total data volume of the data stored in the data sending queue and a first preset data volume is smaller than a first preset difference value, and the difference value between the first preset data volume and the maximum data volume of the middleware single broadcast is smaller than a second preset difference value;
and sequentially sending the second data in each data sending queue to a middleware.
In an embodiment, the step of placing each piece of the second data into each data sending queue includes:
determining a first residual storage amount of a current data sending queue;
when the data volume of the second data to be currently put in is smaller than or equal to the first remaining storage volume, putting the second data to be currently put in into the current data sending queue, and re-determining the second data to be currently put in so as to return to the step of executing the step of determining the first remaining storage volume of the current data sending queue;
when the data volume of the second data is larger than the first residual storage volume, updating a data sending queue without stored data into a current data sending queue, and putting the second data to be put into the current data sending queue after updating;
and re-determining the second data to be put in at present, and returning to the step of determining the first remaining storage capacity of the current data sending queue.
In an embodiment, the step of placing each piece of the second data into each data sending queue includes:
configuring a corresponding storage serial number for each piece of second data of the data sending queue, and associating the storage serial number with the second data corresponding to the storage serial number;
and determining the quantity of the second data stored in each data sending queue and the sending sequence number of the data sending queue, and associating the quantity and the sending sequence number with each second data in the data sending queue.
In an embodiment, after the step of sending each piece of the second data to the middleware so that the middleware broadcasts each piece of the second data to each second terminal, the method further includes:
receiving a retransmission request sent by the second terminal;
determining second data to be retransmitted according to the storage sequence number and the sending sequence number in the retransmission request;
and sending the second data to be retransmitted to the second terminal.
In an embodiment, the step of arranging the type of each piece of the first data and the value field in a preset order to generate second data corresponding to each piece of the first data includes:
determining a first version number corresponding to each piece of the first data;
and arranging the type, the first version number and the value field of each piece of first data in a preset sequence to generate second data corresponding to each piece of first data.
In order to achieve the above object, the present invention further provides a data transmission method, where the data transmission method includes:
the second terminal receives second data sent by the first terminal;
extracting types and each value field from the second data;
and determining a preset sequence according to the types, and configuring a field name for each value-taking field according to the preset sequence to obtain the first data corresponding to the second data.
In an embodiment, the step of determining a preset sequence according to the type, and configuring a field name for each value-taking field according to the preset sequence to obtain the first data corresponding to the second data includes:
determining a data template corresponding to the second data according to the type, wherein the field names in the data template are arranged according to a preset sequence;
and according to the extraction sequence of each value field, sequentially filling each value field into the field name in the data template to obtain the first data corresponding to the second data.
In an embodiment, the step of determining, according to the type, a data template corresponding to each piece of the second data includes:
determining a device identifier of the first terminal;
and determining a data template corresponding to the second data according to the equipment identification and the type.
In an embodiment, after the step of determining a preset order according to the type and configuring a field name for each value-taking field according to the preset order to obtain the first data corresponding to the second data, the method further includes:
acquiring a storage serial number, a sending serial number and the quantity associated with the second data;
when the quantity of the received second data with the same sending sequence number is smaller than the obtained quantity, determining the storage sequence number corresponding to the second data which is not received according to each storage sequence number;
generating a retransmission request according to the storage sequence number of the second data which is not received and the sending sequence number, and sending the retransmission request to the first terminal;
and receiving second data fed back by the first terminal based on the retransmission request.
In an embodiment, after the step of receiving, by the second terminal, the second data sent by the first terminal, the method further includes:
extracting the type and the first version number in the second data;
determining a second version number of third data corresponding to the type stored by the second terminal;
and when the second version number is smaller than the first version number, executing the step of extracting the type and each value field in the second data.
In an embodiment, after the step of determining a preset order according to the type and configuring a field name for each value-taking field according to the preset order to obtain the first data corresponding to the second data, the method further includes:
determining a data amount of the first data;
determining a target storage space corresponding to the first data according to the data volume of the first data, wherein when the data volume of the first data is smaller than a second preset data volume, the first storage space for storing small data is determined as the target storage space, and when the data volume of the second data volume is larger than or equal to the second preset data volume, the second storage space for storing large data is determined as the target storage space;
and storing the first data to the target storage space.
In an embodiment, after the step of storing the first data in the target storage space, the method further includes:
determining a second residual storage amount and a preset residual storage amount of the target storage space;
and when the second residual storage amount is smaller than the preset residual storage amount of the target storage space, deleting the expired data in the target storage space, wherein the expired data comprises data with the access frequency smaller than the preset frequency.
In an embodiment, before the step of determining the remaining storage amount of the target storage space and presetting the remaining storage amount, the method further includes:
acquiring a first access frequency corresponding to each fourth data in a first storage space and a second access frequency corresponding to each fifth data in a second storage space, wherein the data volume of the fourth data is smaller than a preset threshold value, and the data volume of the fifth data is larger than or equal to the preset threshold value;
determining a first reference value corresponding to each fourth data according to the data volume of each fourth data and the first access frequency;
determining a second reference value corresponding to each fifth data according to the data volume of each fifth data and the second access frequency;
and setting a preset residual storage amount of the first storage space and a preset residual storage amount of the second storage space according to the sum of the first reference values, the sum of the second reference values and the storage data amount of the second terminal.
To achieve the above object, the present invention further provides a data transmission system, which includes a first terminal and a second terminal communicatively connected to the first terminal, wherein,
the first terminal is configured to acquire each piece of first data to be sent, determine a type of each piece of first data, extract a value field of a field name in each piece of first data, arrange the type of each piece of first data and each value field in a preset order to generate second data corresponding to each piece of first data, and send each piece of second data to a middleware, so that the middleware broadcasts each piece of second data to each second terminal;
the second terminal is configured to receive second data sent by the first terminal, extract a type and each value field in the second data, determine a preset sequence according to the type, and configure a field name for each value field according to the preset sequence to obtain the first data corresponding to the second data.
In order to achieve the above object, the present invention further provides a terminal, including:
the acquisition module is used for acquiring each piece of first data to be transmitted;
the determining module is used for determining the type of each piece of first data and extracting a value field of a field name in each piece of first data;
the arrangement module is used for arranging the type of each piece of first data and each value field in a preset sequence to generate second data corresponding to each piece of first data;
and the broadcasting module is used for sending each piece of second data to the middleware so that the middleware broadcasts each piece of second data to each second terminal.
In order to achieve the above object, the present invention further provides a terminal, including:
the receiving module is used for receiving second data sent by the first terminal;
the extraction module is used for extracting types and each value field from the second data;
and the configuration module is used for determining a preset sequence according to the types, and configuring a field name for each value-taking field according to the preset sequence to obtain the first data corresponding to the second data.
In order to achieve the above object, the present invention further provides a terminal, including: a memory, a processor and a transmission program stored on the memory and executable on the processor, the transmission program, when executed by the processor, implementing the steps of the data transmission method as described above.
To achieve the above object, the present invention also provides a computer-readable storage medium storing a transmission program that implements the steps of the data transmission method as described above when executed by a processor.
The invention provides a terminal, a data transmission method, a system and a computer readable storage medium, wherein the first terminal acquires each piece of first data to be transmitted, determines the type of each piece of first data, extracts the value field of the field name in each piece of first data, arranges each piece of first data type and the value field in a preset sequence to generate second data corresponding to each piece of first data, and finally transmits the second data to the second terminal, so that the terminal recombines each value field based on the type to obtain the first data. The terminal extracts the value field of the field name in the first data to be sent so as to generate the second data with smaller data volume by sequential arrangement according to the type and the value field, thereby greatly reducing the data sending volume of the first terminal, reducing the network overhead of the first terminal and improving the stable operation of the terminal.
Drawings
Fig. 1 is a schematic terminal structure diagram of a hardware operating environment according to an embodiment of the present invention;
FIG. 2 is a flowchart illustrating a data transmission method according to a first embodiment of the present invention;
fig. 3 is a detailed flowchart of step S40 in the second embodiment of the data transmission method according to the present invention;
fig. 4 is a detailed flowchart of step S41 in the third embodiment of the data transmission method according to the present invention;
fig. 5 is a detailed flowchart of step S30 in the fourth embodiment of the data transmission method according to the present invention;
FIG. 6 is a flowchart illustrating a fifth embodiment of a data transmission method according to the present invention;
fig. 7 is a detailed flowchart of step S120 in a sixth embodiment of the data transmission method according to the present invention;
FIG. 8 is a flowchart illustrating a seventh embodiment of a data transmission method according to the present invention;
fig. 9 is a functional block diagram of the terminal according to the first embodiment of the present invention.
The implementation, functional features and advantages of the objects of the present invention will be further explained with reference to the accompanying drawings.
Detailed Description
It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
Referring to fig. 1, fig. 1 is a schematic device structure diagram of a hardware operating environment according to an embodiment of the present invention.
The terminal in the embodiment of the present invention may be a terminal with data transmission capability, and the terminal may be a smart phone, or may be a terminal device such as a PC (Personal Computer), a tablet Computer, or a portable Computer.
As shown in fig. 1, the terminal may include: a processor 1001, such as a CPU, a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. Wherein a communication bus 1002 is used to enable connective communication between these components. The user interface 1003 may include a Display screen (Display), an input unit such as a Keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface, a wireless interface. The network interface 1004 may optionally include a standard wired interface, a wireless interface (e.g., a Wi-Fi interface). The memory 1005 may be a high-speed RAM memory or a non-volatile memory (e.g., a magnetic disk memory). The memory 1005 may alternatively be a storage device separate from the processor 1001.
Those skilled in the art will appreciate that the terminal structure shown in fig. 1 is not intended to be limiting and may include more or fewer components than those shown, or some components may be combined, or a different arrangement of components.
As shown in fig. 1, a memory 1005, which is a kind of computer storage medium, may include therein an operating system, a network communication module, a user interface module, and a transmission program.
In the terminal shown in fig. 1, the network interface 1004 is mainly used for connecting to a backend server and performing data communication with the backend server; the user interface 1003 is mainly used for connecting a client and performing data communication with the client; and the processor 1001 may be configured to call the transfer program stored in the memory 1005 and perform the following operations:
the method comprises the steps that a first terminal obtains various pieces of first data to be sent;
determining the type of each piece of first data, and extracting a value field of a field name in each piece of first data;
arranging the type of each piece of first data and each value field in a preset sequence to generate second data corresponding to each piece of first data;
and sending each piece of the second data to a middleware so that the middleware broadcasts each piece of the second data to each second terminal.
In one embodiment, the processor 1001 may call a transfer program stored in the memory 1005, and further perform the following operations:
putting each piece of second data into each data sending queue, wherein the difference value between the total data volume of the data stored in the data sending queue and a first preset data volume is smaller than a first preset difference value, and the difference value between the first preset data volume and the maximum data volume of the middleware single broadcast is smaller than a second preset difference value;
and sequentially sending the second data in each data sending queue to a middleware.
In one embodiment, the processor 1001 may call a transfer program stored in the memory 1005, and further perform the following operations:
determining a first residual storage amount of a current data sending queue;
when the data volume of the second data to be currently put in is smaller than or equal to the first remaining storage volume, putting the second data to be currently put in into the current data sending queue, and re-determining the second data to be currently put in so as to return to the step of executing the step of determining the first remaining storage volume of the current data sending queue;
when the data volume of the second data is larger than the first residual storage volume, updating a data sending queue without stored data into a current data sending queue, and putting the second data to be put into the current data sending queue after updating;
and re-determining the second data to be put in at present, and returning to the step of determining the first remaining storage capacity of the current data sending queue.
In one embodiment, the processor 1001 may call a transfer program stored in the memory 1005, and further perform the following operations:
configuring a corresponding storage serial number for each piece of second data of the data sending queue, and associating the storage serial number with the second data corresponding to the storage serial number;
and determining the quantity of the second data stored in each data sending queue and the sending sequence number of the data sending queue, and associating the quantity and the sending sequence number with each second data in the data sending queue.
In one embodiment, the processor 1001 may call a transfer program stored in the memory 1005, and further perform the following operations:
receiving a retransmission request sent by the second terminal;
determining second data to be retransmitted according to the storage sequence number and the sending sequence number in the retransmission request;
and sending the second data to be retransmitted to the second terminal.
In one embodiment, the processor 1001 may call a transfer program stored in the memory 1005, and further perform the following operations:
determining a first version number corresponding to each piece of the first data;
and arranging the type, the first version number and the value field of each piece of first data in a preset sequence to generate second data corresponding to each piece of first data.
In one embodiment, the processor 1001 may call a transfer program stored in the memory 1005, and further perform the following operations:
the second terminal receives second data sent by the first terminal;
extracting types and each value field from the second data;
and determining a preset sequence according to the types, and configuring a field name for each value-taking field according to the preset sequence to obtain the first data corresponding to the second data.
In one embodiment, the processor 1001 may call a transfer program stored in the memory 1005, and further perform the following operations:
determining a data template corresponding to the second data according to the type, wherein the field names in the data template are arranged according to a preset sequence;
and according to the extraction sequence of each value field, sequentially filling each value field into the field name in the data template to obtain the first data corresponding to the second data.
In one embodiment, the processor 1001 may call a transfer program stored in the memory 1005, and further perform the following operations:
determining a device identifier of the first terminal;
and determining a data template corresponding to the second data according to the equipment identification and the type.
In an embodiment, after the step of determining a preset order according to the type and configuring a field name for each value-taking field according to the preset order to obtain the first data corresponding to the second data, the method further includes:
acquiring a storage serial number, a sending serial number and the quantity associated with the second data;
when the quantity of the received second data with the same sending sequence number is smaller than the obtained quantity, determining the storage sequence number corresponding to the second data which is not received according to each storage sequence number;
generating a retransmission request according to the storage sequence number of the second data which is not received and the sending sequence number, and sending the retransmission request to the first terminal;
and receiving second data fed back by the first terminal based on the retransmission request.
In one embodiment, the processor 1001 may call a transfer program stored in the memory 1005, and further perform the following operations:
extracting the type and the first version number in the second data;
determining a second version number of third data corresponding to the type stored by the second terminal;
and when the second version number is smaller than the first version number, executing the step of extracting the type and each value field in the second data.
In one embodiment, the processor 1001 may call a transfer program stored in the memory 1005, and further perform the following operations:
determining a data amount of the first data;
determining a target storage space corresponding to the first data according to the data volume of the first data, wherein when the data volume of the first data is smaller than a second preset data volume, the first storage space for storing small data is determined as the target storage space, and when the data volume of the second data volume is larger than or equal to the second preset data volume, the second storage space for storing large data is determined as the target storage space;
and storing the first data to the target storage space.
In one embodiment, the processor 1001 may call a transfer program stored in the memory 1005, and further perform the following operations:
determining a second residual storage amount and a preset residual storage amount of the target storage space;
and when the second residual storage amount is smaller than the preset residual storage amount of the target storage space, deleting the expired data in the target storage space, wherein the expired data comprises data with the access frequency smaller than the preset frequency.
In one embodiment, the processor 1001 may call a transfer program stored in the memory 1005, and further perform the following operations:
acquiring a first access frequency corresponding to each fourth data in a first storage space and a second access frequency corresponding to each fifth data in a second storage space, wherein the data volume of the fourth data is smaller than a preset threshold value, and the data volume of the fifth data is larger than or equal to the preset threshold value;
determining a first reference value corresponding to each fourth data according to the data volume of each fourth data and the first access frequency;
determining a second reference value corresponding to each fifth data according to the data volume of each fifth data and the second access frequency;
and setting a preset residual storage amount of the first storage space and a preset residual storage amount of the second storage space according to the sum of the first reference values, the sum of the second reference values and the storage data amount of the second terminal.
Based on the above hardware structure, embodiments of the data transmission method of the present invention are provided.
The invention provides a data transmission method.
Referring to fig. 2, fig. 2 is a first embodiment of a data transmission method of the present invention, where the data transmission method includes:
step S10, the first terminal obtains each piece of first data to be sent;
in this embodiment, the first terminal and the second terminal form a cluster, the first terminal and the second terminal may be servers, and the first terminal and the second terminal are nodes in the cluster. When a user performs a service operation, the user may invoke an interface of a terminal in the cluster to send data to the terminal, where it can be understood that a terminal where data update occurs is a first terminal, and a terminal that receives updated data sent by the first terminal is a second terminal. In addition, the first terminal can also send the updated data to a database outside the cluster for storage, and the database can also be regarded as a second terminal. The first terminal acquires each updated first data after detecting the data update, and the updated first data is the first data to be sent.
Step S20, determining the type of each piece of first data, and extracting the value field of the field name in each piece of first data;
step S30, arranging the type of each piece of the first data and each value field in a preset order to generate second data corresponding to each piece of the first data;
the first data is service data, the service data has different types, and the field names of the same position in the first data of the same type are the same. For example, the first data is data of a student class, and the first field names and the second field names of all the first data of the student class are names and ages. In contrast, the first terminal determines the type of each piece of first data, and extracts the value field from the field name in the first data, so as to generate second data corresponding to the first data according to the type of the first data and each value field.
It should be noted that the order of extraction of the extracted value field is related to the position of the field name in the first data. For example, the first data is: 1000, name: zhangsan, age: han, the type of the first data is student class, the extracted value fields are 1000, zhangsan, 23 and han respectively, and the second data is: student, 1000, zhangsan, 23, han. The second data is smaller in field name than the first data, and thus the data amount of the second data is much smaller than that of the first data. The extraction sequence of the value-taking fields is related to the arrangement sequence of the value fields and the types in the second data, the arrangement sequence is a preset sequence, and the preset sequence is stored in the first terminal. The first terminal arranges the types and the value fields based on a preset sequence. For example, if the preset sequence is type, ID, name, age, and name, the arrangement of the type and each value field is: student, 1000, zhangsan, 23, han.
Step S40, sending each piece of the second data to a middleware, so that the middleware broadcasts each piece of the second data to each second terminal.
After obtaining the second data, the first terminal may send the second data to each second terminal. Specifically, the first terminal and each second terminal form a cluster, and the cluster has a corresponding middleware, and the middleware has a data broadcasting function. The first terminal sends each piece of second data to the middleware, and the middleware broadcasts each piece of second data to each second terminal.
The second terminal stores the recombination rule sent by the first terminal, so that the field names of the value fields are configured according to the recombination rule and the type in the second data, and the first data corresponding to the second data are obtained. Specifically, the second terminal disassembles the second data to sequentially extract the type of the first data and the value field corresponding to each field name in the first data, determines the recombination rule of each value field based on the type, assigns the corresponding field name to each value field according to the recombination rule, and recombines the field name and the value field corresponding to the field name, thereby obtaining the first data corresponding to the second data. For example, the second data is: student, 1000, zhangsan, 23, han. The second terminal determines a recombination rule according to the student, the recombination rule is determined according to a preset sequence, the preset sequence is type, ID, name, age and substitution, the second terminal ignores the type, a first value field is given with a field name ID, a second value field is given with a field name, a third value field is given with a field name and a fourth value field is given with a field name substitution, and therefore a first data ID of 1000, name: zhangsan, age: and 23, nation: han.
In the technical scheme provided by this embodiment, a first terminal acquires each piece of first data to be transmitted, determines a type of each piece of first data, extracts a value field of a field name in each piece of first data, arranges each piece of first data type and the value field in a preset sequence to generate second data corresponding to each piece of first data, and finally transmits the second data to a second terminal, so that the terminal recombines each value field based on the type to obtain the first data. The terminal extracts the value field of the field name in the first data to be sent so as to generate the second data with smaller data volume by sequential arrangement according to the type and the value field, thereby greatly reducing the data sending volume of the first terminal, reducing the network overhead of the first terminal and improving the stable operation of the terminal.
Referring to fig. 3, fig. 3 is a second embodiment of the data transmission method of the present invention, and based on the first embodiment, the step S40 includes:
step S41, putting each piece of the second data into each data sending queue, where a difference between a total data amount of data stored in the data sending queue and a first preset data amount is smaller than a first preset difference, and a difference between the first preset data amount and a maximum data amount of a single broadcast of the middleware is smaller than a second preset difference;
step S42, sequentially sending the second data in each data sending queue to the middleware.
In this embodiment, the first terminal broadcasts each piece of second data to each second terminal through the middleware. The data volume of the single broadcast of the middleware has an upper limit value, and if the total data volume of each piece of second data is large, the first terminal needs to send the second data to the middleware in batches. In this regard, the first terminal needs to segment each piece of second data so that the data amount of the middle single broadcast does not exceed the upper limit value. The first terminal is provided with a plurality of data sending queues, the difference value between the total data quantity of the data which can be stored in each data sending queue and a first preset data quantity is smaller than a first preset difference value, and the difference value between the first preset data quantity and the maximum data quantity of the single broadcast of the middleware is smaller than a second preset difference value. The first preset difference and the second preset difference can be zero or a smaller value, so that the first preset data amount is slightly smaller than or equal to the maximum data amount of the single broadcast of the middleware, and the total data amount of the data sending queue is slightly smaller than or equal to the maximum data amount of the single broadcast of the middleware, so that the middleware can broadcast each piece of second data to the maximum extent.
The first terminal determines a first residual storage amount of a current data sending queue, the current data sending queue is a data sending queue needing to put in second data, and the first residual storage amount is a residual storable amount of the current data sending queue. Then, the first terminal determines whether the data volume of the second data to be currently put in is less than or equal to the first remaining storage volume, if so, the second data to be currently put in is put in the current data sending queue, and the second data to be currently put in is re-determined from the second data not put in, so that the step of determining the first remaining data volume of the current data sending queue is returned to. If the data volume of the second data volume to be currently put in is larger than the first remaining storage volume, it indicates that the current data sending queue cannot store the second data, at this time, the first terminal updates the data sending queue which does not store the data into the current data sending queue, then puts the second data to be currently put in into the current data sending queue until the second data to be currently put in is updated into the current data sending queue, and the first terminal re-determines the second data to be currently put in so as to return to execute the step of determining the first remaining storage volume of the current data sending queue. It can be understood that, before the second data is placed into the data transmission queue each time, the first terminal needs to determine whether the remaining storage capacity of the data transmission queue is sufficient to store the second data, if so, the second data is placed into the data transmission queue, and if not, the second data is placed into a new data transmission queue, so that the total data volume of the data stored in each data transmission queue is less than or equal to the maximum data volume of the middle single broadcast.
In the technical scheme provided by this embodiment, the first terminal places each piece of second data into each data transmission queue, so that the first terminal transmits each piece of second data to the middleware in a form based on the data transmission queue in a time-sharing manner, thereby ensuring that the middleware can broadcast the second data of the maximum data in a single broadcast to the maximum extent, and further improving the caching efficiency of each second terminal.
Referring to fig. 4, fig. 4 is a third embodiment of the data transmission method of the present invention, and based on the second embodiment, the step S41 includes:
step S411, configuring a corresponding storage serial number for each piece of second data put into the data sending queue, and associating the storage serial number with the second data corresponding to the storage serial number;
step S412, determining the number of the second data stored in each data sending queue and the sending sequence number of the data sending queue, and associating the number and the sending sequence number with each second data in the data sending queue.
In this embodiment, due to the instability of the network, for a large amount of data transmission, the transmission data may be lost, that is, all the second data may not be updated by the second terminal. In this regard, when placing each piece of second data into the data transmission queue, the first terminal configures a corresponding storage sequence number for the second data according to the order of the second data placed into the data transmission queue, and places the storage sequence number and the second data into the data transmission queue in an associated manner.
In addition, the first terminal determines the transmission sequence number of the data transmission queue and the quantity of the second data in the data transmission queue, so as to associate the transmission sequence number and the quantity with each piece of the second data in the data transmission queue. It can be understood that the data sending queue includes a plurality of pieces of second data, a storage sequence number corresponding to each piece of second data, a sending sequence number of the data sending queue, and a number of pieces of second data in the data sending queue.
And when receiving each piece of second data in the data sending queue, the second terminal extracts the storage sequence number of each piece of data, the sending sequence number corresponding to each piece of second data and the quantity of the second data. If the number of the received second data with the same sending sequence number is smaller than the obtained number, it can be determined that the second terminal does not completely receive all the second data in the data sending queue. The storage sequence numbers of the second data are numbered according to the storage sequence, so that the storage sequence numbers of the second data in the same data sending queue are continuous, the second terminal determines the largest storage sequence number according to the extracted number, and the storage sequence number of the second data which is not received is determined according to the largest storage sequence number and the continuity of the storage sequence numbers. For example, if the extracted number is 5, the maximum storage serial number is also 5, and the second terminal receives the second data with the storage serial numbers 1, 2, 4, and 5, it may be determined that the second terminal does not receive the second data with the storage serial number 3.
And the second terminal generates a retransmission request according to the storage sequence number and the sending sequence number of the second data which are not received, and sends the retransmission request to the first terminal. The first terminal analyzes the retransmission request to obtain a storage sequence number and a transmission sequence number of the second data to be retransmitted, determines a corresponding data transmission queue based on the transmission sequence number, extracts the second data corresponding to the storage sequence number from the data transmission queue as the second data to be retransmitted, and transmits the second data to be retransmitted to the second terminal.
It should be noted that the second terminal may determine, after receiving all the second data, the second data that is not received in a centralized manner, so as to generate the retransmission request according to the storage sequence number and the transmission sequence number corresponding to each piece of the second data that is not received. And the first terminal finds all the second data to be retransmitted according to the retransmission request and sends all the second data to be retransmitted to the second terminal together.
In the technical solution provided in this embodiment, the first terminal configures a corresponding storage sequence number for each piece of second data placed in the data sending queue, associates the storage sequence number with the second data corresponding to the storage sequence number, and determines the number of the second data stored in the data sending queue and the sending sequence number of the data sending queue, so as to associate the number and the sending sequence number with each piece of second data in the data sending queue, so that the second terminal determines that the second data is not received.
Referring to fig. 5, fig. 5 is a fourth embodiment of the data transmission method of the present invention, and based on any one of the first to third embodiments, the step S30 includes:
step S31, determining a first version number corresponding to each piece of the first data;
step S32, arranging the type, the first version number, and each value field of each piece of the first data in a preset order, and generating second data corresponding to each piece of the first data.
In this embodiment, the first terminal sends each piece of second data to the middleware, and the middleware broadcasts the second data to each second terminal, so that each second terminal receives the second data. Two versions of the same second data are broadcast and sent to the same second terminal through the message middleware, and as the middleware indiscriminately broadcasts the second data, the second data of an old version is sent first, and then the data of the other terminal is updated and is synchronized to the second terminal again. But there is uncertainty in the network and the new version of data may be received by the second terminal first. If the local cache is updated according to the sequence of the received messages, the cached data record is updated to the new version and then updated to the old version again, so that the second terminal caches the stored data which is not the latest data.
For example, there are two versions of data C, server A, upon receiving a user request, updates data C to the first version C1, while synchronizing C1 to servers B and C, and then server B also receives a user request to update data C to the C2 version and synchronizes to servers A and C. Server C receives two versions of two data C. And the server C receives the C2 version first, updates the local cache and then receives the low version C1, which results in the updated data of the server being historical data.
In this regard, the first terminal adds the version number in the second data, that is, the first terminal determines the first version number corresponding to each piece of the first data, and then arranges the type, the first version number, and each value field of each piece of the first data in a preset order to generate the second data corresponding to each piece of the first data. The first terminal may determine the first version number of the updated first data according to the version number of the first data before the update, and the type of the first data after the update is the same as the type of the first data after the update. For example, if the version number of the first data that is not updated is 5.0, the version number of the first data that is updated may be 5.1.
The second data comprises a type, each value field and a first version number. And after receiving the second data, the second terminal extracts the first version number and the type from the second data, so that third data corresponding to the type is obtained from the second terminal, and a second version number of the third data is further determined. It can be understood that when the second terminal recombines each value field, the version number is retained, so that the first data obtained by recombination is associated with the version number, and all data stored in the second terminal have corresponding version numbers. And the second terminal determines whether the second version number is smaller than the first version number, if so, the second data is the latest data, and the second terminal needs to recombine the second data to obtain the first data for storage and delete the first data with the second version number.
In the technical scheme provided in this embodiment, the first terminal determines a first version number corresponding to each piece of first data, and arranges the type of each piece of first data, the first version number, and each value field in a preset order to generate second data corresponding to each piece of first data, so that the second terminal determines whether the second data needs to be saved based on a version number in the second data, so as to ensure that the second terminal updates the latest second data.
The invention also provides a data transmission method.
Referring to fig. 6, fig. 6 is a fifth embodiment of the data transmission method of the present invention, which includes:
step S100, a second terminal receives second data sent by a first terminal;
step S110, extracting types and each value field from the second data;
step S120, determining a preset sequence according to the type, and configuring a field name for each value-taking field according to the preset sequence to obtain the first data corresponding to the second data.
In this embodiment, the execution main body is the second terminal, and the second terminal is the terminal that receives the second data. And after receiving the second data each time, the second terminal extracts the type and each value field corresponding to the first data from the second data. The second terminal stores the preset sequence sent by other terminals and the type of the associated data in the sent preset sequence. The second terminal can determine a preset sequence according to the extracted type, wherein the preset sequence is the type and the arrangement sequence of each value field. And the second terminal configures corresponding field names for the value fields according to a preset sequence to obtain first data corresponding to the second data. The specific process of the second terminal obtaining the first data according to the second data refers to the above description, and is not repeated herein.
In the technical solution provided in this embodiment, the second terminal receives the second data sent by the first terminal, and extracts the type and each value field from the second data, so as to determine a preset sequence according to the type, configure the field name for the value field according to the preset sequence, and generate the first data corresponding to the second data. According to the invention, the terminal is restored into the first data with large data volume according to the second data with small data volume, so that the data sending volume of the first terminal is reduced, the network overhead of the first terminal is reduced, and the stable operation performance of the terminal is improved.
Referring to fig. 7, fig. 7 is a sixth embodiment of the data transmission method of the present invention, and based on the fifth embodiment, the step S120 includes:
s121, determining a data template corresponding to the second data according to the type, wherein the field names in the data template are arranged according to a preset sequence;
and S122, according to the extraction sequence of each value field, sequentially filling each value field into the field name in the data template to obtain the first data corresponding to the second data.
In this embodiment, the second terminal stores a data template corresponding to various types of data, the data template includes field names corresponding to the data, but does not include value fields corresponding to the field names, and the field names in the data template are arranged according to a preset sequence.
The data module is stored in association with the type. After receiving the second data, the device extracts the type from the second data, so as to determine a data template corresponding to the second data according to the type, and then sequentially fills each value field extracted from the second data into the field name of the data template according to the position of the value field in the second data, thereby obtaining the first data corresponding to the second data.
It should be noted that, the extraction order of each terminal in the cluster for each value field in the first data may be all different, and therefore, the terminal may generate a data template corresponding to various types of data, and broadcast the type and the template data corresponding to the type to other terminals in the cluster. It can be understood that the second terminal stores therein data templates corresponding to other terminals, that is, the device identifier, the type, and the data template of the terminal are stored in association. In this regard, after receiving the second data, the second terminal determines the device identifier of the first terminal, so as to determine the data template corresponding to the second data according to the device identifier and the type.
In the technical scheme provided by this embodiment, the second terminal determines a data template corresponding to the second data according to the type, and sequentially fills each value field into a field name in the data template according to the position of each value field in the second data, thereby quickly obtaining the first data corresponding to the second data.
In an embodiment, after step S120, the method further includes:
acquiring a storage serial number, a sending serial number and the quantity associated with the second data;
when the quantity of the received second data with the same sending sequence number is smaller than the obtained quantity, determining the storage sequence number corresponding to the second data which is not received according to each storage sequence number;
generating a retransmission request according to the storage sequence number of the second data which is not received and the sending sequence number, and sending the retransmission request to the first terminal;
and receiving second data fed back by the first terminal based on the retransmission request.
In this embodiment, the second data is associated with a storage sequence number, a transmission sequence number, and a number, and the second terminal determines whether there is second data that is not received based on the storage sequence number, the transmission sequence number, and the number, so as to send a retransmission request to the first terminal, so as to obtain the second data that is not received. For a specific process, reference is made to the above description, which is not repeated herein.
In an embodiment, after step S100, the method further includes:
extracting the type and the first version number in the second data;
determining a second version number of third data corresponding to the type in the second terminal;
and when the second version number is smaller than the first version number, executing the step of extracting the type and each value field corresponding to the second data from each piece of second data.
In this embodiment, the second data includes a first version number of the first data. The second terminal can determine whether the second data is the latest version of data or not based on the first version number, and if the second data is the latest version of data, the second terminal conducts value field recombination on the second data to obtain the first data. For the specific process of the second terminal determining whether the second data is the latest version of data according to the first version number, reference is made to the above description, and details are not repeated here.
Referring to fig. 8, fig. 8 is a seventh embodiment of the data transmission method of the present invention, and based on the fifth or sixth embodiment, after step S120, the method further includes:
step S130, determining the data volume of the first data;
step S140, determining a target storage space corresponding to the first data according to the data amount of the first data, wherein when the data amount of the first data is smaller than a second preset data amount, the first storage space storing the small data is determined as the target storage space, and when the data amount of the second data is greater than or equal to the second preset data amount, the second storage space storing the large data is determined as the target storage space;
step S150, storing the first data to the target storage space.
In this embodiment, the second terminal is provided with a first storage space and a second storage space. The first storage space stores small data and the second storage space stores large data. The data amount of the data smaller than the second preset data amount is defined as small data, and the data amount of the data larger than or equal to the second preset data amount is defined as large data. The second predetermined amount of data may be any suitable value. And the second terminal receives the second data again, obtains first data corresponding to the second data, and then determines the data volume of the first data, so that the target storage space is determined in the first storage space and the second storage space according to the data volume of the first data. Namely the data volume of the first data is smaller than the second preset data volume, and the first storage space for storing the small data is determined as a target storage space; and if the data volume of the first data is larger than or equal to a second preset data volume, determining a second storage space for storing the big data as a target storage space.
And after the target storage space is determined, the second terminal can store the first data into the target storage space.
Since the data storage amount of the target storage space in the terminal is constant, after the first data is stored in the target storage space, whether the remaining storage amount of the target storage space is sufficient needs to be determined, and if the remaining storage amount of the target storage space is insufficient, the past data in the target storage space is cleared.
＝
And the storage amount M of the first storage spaceSmallMemory space M with second memory spaceBig (a)The two are not equally divided, and are determined according to the access frequency of the data of the user. In particular, MSmallAnd MBig (a)The relationship between them can be referred to the following formula:
Figure BDA0002560460440000201
pi is a first access frequency of fourth data in the first storage space, and Pj is a second access frequency of fifth data in the second storage space. The frequency may be a frequency or a number of times.
Based on the above formula, M isSmallAnd MBig (a)And then setting M according to the ratio K and the total data storage capacity of the second terminalSmallAnd MBig (a)The specific data storage amount of (2). The preset residual memory space of the first memory space is according to MSmallDetermining, and the preset residual storage capacity of the second storage space is according to MBig (a)And (4) determining. And the second terminal determines the second residual storage amount and the preset residual storage amount of the target storage space, and determines whether the second residual storage amount is smaller than the preset residual storage amount of the target storage space.
It can be understood that, after the second terminal determines the remaining storage amount of each storage space, the second terminal obtains a first access frequency corresponding to each fourth data in the first storage space and a second access frequency corresponding to each fifth data in the second storage space, where a data amount of the fourth data is smaller than a preset threshold, and a data amount of the fifth data is greater than or equal to the preset threshold. The second terminal determines a first reference value (i.e., Pi × Mi) corresponding to each fourth data according to the data amount of each fourth data and the first access frequency, and determines a second reference value (i.e., Pj × Mj) corresponding to each fifth data according to the data amount of each fifth data and the second access frequency. And finally, the second terminal sets the preset residual storage capacity of the first storage space and the preset residual storage capacity of the second storage space according to the sum of the first reference values, the sum of the second reference values and the storage data volume of the second terminal.
And when the second terminal determines that the second residual storage amount of the target storage space is less than the preset storage amount of the target storage space, deleting the expired data in the target storage space, wherein the expired data comprises data with the access frequency less than the preset frequency.
It should be noted that, after the first terminal finishes updating the data, the scheme of this embodiment may also be adopted to delete the expired data from each storage space.
In the technical solution provided in this embodiment, the second terminal determines the data size of the first data, and determines the target storage space according to the data size, so as to store the first data in the corresponding target storage space.
The invention also provides a terminal.
Referring to fig. 9, fig. 9 is a functional module diagram of the terminal according to the first embodiment of the present invention.
As shown in fig. 9, the terminal includes:
an obtaining module 10, configured to obtain, by a first terminal, each piece of first data to be sent;
a determining module 20, configured to determine a type of each piece of the first data, and extract a value field of a field name in each piece of the first data;
the generating module 30 is configured to arrange the type of each piece of the first data and each value field in a preset order to generate second data corresponding to each piece of the first data;
a sending module 40, configured to send each piece of the second data to a middleware, so that the middleware broadcasts each piece of the second data to each second terminal.
In an embodiment, the terminal further includes a placing module:
the putting-in module is used for putting each piece of second data into each data sending queue, the difference value between the total data volume of the data stored in the data sending queues and a first preset data volume is smaller than a first preset difference value, and the difference value between the first preset data volume and the maximum data volume of the middleware single broadcast is smaller than a second preset difference value;
the sending module 40 is further configured to send the second data in each data sending queue to a middleware in sequence.
In an embodiment, the terminal further includes an update module:
a determining module 20, configured to determine a first remaining storage amount of a current data sending queue;
the putting-in module is used for putting the second data to be put in the current data sending queue when the data volume of the second data to be put in at present is less than or equal to the first residual storage volume, and re-determining the second data to be put in at present so as to return to the step of determining the first residual storage volume of the current data sending queue;
the updating module is used for updating a data sending queue which does not store data into a current data sending queue when the data volume of the second data is larger than the first residual storage volume, and putting the second data to be put into the current data sending queue after updating;
and re-determining the second data to be put in at present, and returning to the step of determining the first remaining storage capacity of the current data sending queue.
In an embodiment, the terminal further includes a configuration module:
the configuration module is used for configuring a corresponding storage serial number for each piece of second data placed in the data sending queue and associating the storage serial number with the second data corresponding to the storage serial number;
the determining module 20 is further configured to determine the number of the second data stored in each data sending queue and the sending sequence number of the data sending queue, and associate the number and the sending sequence number with each second data in the data sending queue.
In an embodiment, the terminal further includes a receiving module:
the receiving module is configured to receive a retransmission request sent by the second terminal;
the determining module 20 is further configured to determine second data to be retransmitted according to the storage sequence number and the sending sequence number in the retransmission request;
the sending module 40 is further configured to send the second data to be retransmitted to the second terminal.
In an embodiment, the determining module 20 is further configured to determine a first version number corresponding to each piece of the first data;
the generating module 30 is further configured to arrange the type, the first version number, and each value field of each piece of the first data in a preset order, and generate second data corresponding to each piece of the first data.
In an embodiment, the terminal further includes an extraction module:
the receiving module is further configured to receive second data sent by the first terminal;
the extraction module is used for extracting types and each value field from the second data;
the configuration module is further configured to determine a preset sequence according to the type, and configure a field name for each value field according to the preset sequence, so as to obtain the first data corresponding to the second data.
In an embodiment, the terminal further includes a fill-in module:
the determining module 20 is further configured to determine a data template corresponding to the second data according to the type, where field names in the data template are arranged according to a preset sequence;
and the filling module is used for sequentially filling the value-taking fields into the field names in the data template according to the extraction sequence of each value-taking field to obtain the first data corresponding to the second data.
In an embodiment, the determining module 20 is further configured to determine a device identifier of the first terminal;
the determining module 20 is further configured to determine a data template corresponding to the second data according to the device identifier and the type.
In an embodiment, the obtaining module 10 is further configured to obtain a storage sequence number, a sending sequence number, and a quantity associated with the second data;
the determining module 20 is further configured to determine, when the number of the received second data with the same sending sequence number is smaller than the acquired number, a storage sequence number corresponding to the second data that is not received according to each storage sequence number;
the generating module 30 is further configured to generate a retransmission request according to the storage sequence number of the second data that is not received and the sending sequence number, and send the retransmission request to the first terminal;
the receiving module is further configured to receive second data fed back by the first terminal based on the retransmission request.
In an embodiment, the terminal further includes an execution module:
the extracting module is further configured to extract the type and the first version number in the second data;
the determining module 20 is further configured to determine a second version number of third data corresponding to the type stored by the second terminal;
and the execution module is used for executing the step of extracting the type and each value field from the second data when the second version number is smaller than the first version number.
In one embodiment, the terminal further includes a storage module:
a determining module for determining a data amount of the first data;
the determining module is used for determining a target storage space corresponding to the first data according to the data volume of the first data, wherein when the data volume of the first data is smaller than a second preset data volume, the first storage space for storing small data is determined as the target storage space, and when the data volume of the second data volume is larger than or equal to the second preset data volume, the second storage space for storing large data is determined as the target storage space;
and the storage module is used for storing the first data to the target storage space.
In an embodiment, the terminal further includes a deletion module:
the determining module 20 is further configured to determine a second remaining storage amount of the target storage space and a preset remaining storage amount;
and the deleting module is used for deleting the expired data in the target storage space when the second remaining storage amount is less than the preset remaining storage amount of the target storage space, wherein the expired data comprises data with the access frequency less than the preset frequency.
In an embodiment, the obtaining module 10 is further configured to obtain a first access frequency corresponding to each fourth data in the first storage space and a second access frequency corresponding to each fifth data in the second storage space, where a data amount of the fourth data is smaller than a preset threshold, and a data amount of the fifth data is greater than or equal to the preset threshold;
the determining module 20 is further configured to determine a first reference value corresponding to each fourth data according to the data amount of each fourth data and the first access frequency;
the determining module 20 is further configured to determine a second reference value corresponding to each fifth data according to the data amount of each fifth data and the second access frequency;
the determining module 20 is further configured to set a preset remaining storage amount of the first storage space and a preset remaining storage amount of the second storage space according to a sum of the first reference values, a sum of the second reference values, and a storage data amount of the second terminal.
The function implementation of each module in the terminal corresponds to each step in the data transmission method embodiment, and the function and implementation process thereof are not described in detail here.
The present invention also provides a computer-readable storage medium having stored thereon a transmission program which, when executed by a processor, implements the steps of the data transmission method according to any one of the above embodiments.
The specific embodiment of the readable storage medium of the present invention is substantially the same as the embodiments of the data transmission method, and is not described herein again.
The invention also provides a data transmission system, which comprises a first terminal and a second terminal which is connected with the first terminal in a communication way, wherein,
the first terminal is used for acquiring each piece of first data to be sent, determining the type of each piece of first data, extracting a value field of a field name in each piece of first data, generating second data corresponding to each piece of first data according to the type of each piece of first data and each value field, and sending each piece of second data to a second terminal;
the second terminal is configured to receive second data sent by the first terminal, extract a type and each value field corresponding to the first data from the second data, and configure a field name for each value field according to the type to obtain first data corresponding to the second data.
It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or system. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method, article, or system that comprises the element.
The above-mentioned serial numbers of the embodiments of the present invention are merely for description and do not represent the merits of the embodiments.
Through the above description of the embodiments, those skilled in the art will clearly understand that the method of the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but in many cases, the former is a better implementation manner. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium (e.g., ROM/RAM, magnetic disk, optical disk) as described above and includes instructions for enabling a terminal device (e.g., a mobile phone, a computer, a server, an air conditioner, or a network device) to execute the method according to the embodiments of the present invention.
The above description is only a preferred embodiment of the present invention, and not intended to limit the scope of the present invention, and all modifications of equivalent structures and equivalent processes, which are made by using the contents of the present specification and the accompanying drawings, or directly or indirectly applied to other related technical fields, are included in the scope of the present invention.

Claims (19)

1. A data transmission method, characterized in that the data transmission method comprises:
the method comprises the steps that a first terminal obtains various pieces of first data to be sent;
determining the type of each piece of first data, and extracting a value field of a field name in each piece of first data;
arranging the type of each piece of first data and each value field in a preset sequence to generate second data corresponding to each piece of first data;
and sending each piece of the second data to a middleware so that the middleware broadcasts each piece of the second data to each second terminal.
2. The data transmission method according to claim 1, wherein the step of sending each piece of the second data to a middleware comprises:
putting each piece of second data into each data sending queue, wherein the difference value between the total data volume of the data stored in the data sending queue and a first preset data volume is smaller than a first preset difference value, and the difference value between the first preset data volume and the maximum data volume of the middleware single broadcast is smaller than a second preset difference value;
and sequentially sending the second data in each data sending queue to a middleware.
3. The data transmission method according to claim 2, wherein the step of placing each piece of the second data into each data sending queue comprises:
determining a first residual storage amount of a current data sending queue;
when the data volume of the second data to be currently put in is smaller than or equal to the first remaining storage volume, putting the second data to be currently put in into the current data sending queue, and re-determining the second data to be currently put in so as to return to the step of executing the step of determining the first remaining storage volume of the current data sending queue;
when the data volume of the second data is larger than the first residual storage volume, updating a data sending queue without stored data into a current data sending queue, and putting the second data to be put into the current data sending queue after updating;
and re-determining the second data to be put in at present, and returning to the step of determining the first remaining storage capacity of the current data sending queue.
4. The data transmission method according to claim 2, wherein the step of placing each piece of the second data into each data sending queue comprises:
configuring a corresponding storage serial number for each piece of second data of the data sending queue, and associating the storage serial number with the second data corresponding to the storage serial number;
and determining the quantity of the second data stored in each data sending queue and the sending sequence number of the data sending queue, and associating the quantity and the sending sequence number with each second data in the data sending queue.
5. The data transmission method according to claim 4, wherein after the step of sending each piece of the second data to the middleware so that the middleware broadcasts each piece of the second data to each second terminal, the method further comprises:
receiving a retransmission request sent by the second terminal;
determining second data to be retransmitted according to the storage sequence number and the sending sequence number in the retransmission request;
and sending the second data to be retransmitted to the second terminal.
6. The data transmission method according to any one of claims 1 to 5, wherein the step of arranging the type of each piece of the first data and each of the value-taking fields in a preset order to generate second data corresponding to each piece of the first data includes:
determining a first version number corresponding to each piece of the first data;
and arranging the type, the first version number and the value field of each piece of first data in a preset sequence to generate second data corresponding to each piece of first data.
7. A data transmission method, characterized in that the data transmission method comprises:
the second terminal receives second data sent by the first terminal;
extracting types and each value field from the second data;
and determining a preset sequence according to the types, and configuring a field name for each value-taking field according to the preset sequence to obtain the first data corresponding to the second data.
8. The data transmission method according to claim 7, wherein the step of determining a preset sequence according to the type, and configuring a field name for each value field according to the preset sequence to obtain the first data corresponding to the second data comprises:
determining a data template corresponding to the second data according to the type, wherein the field names in the data template are arranged according to a preset sequence;
and according to the extraction sequence of each value field, sequentially filling each value field into the field name in the data template to obtain the first data corresponding to the second data.
9. The data transmission method according to claim 8, wherein the step of determining the data template corresponding to each piece of the second data according to the type comprises:
determining a device identifier of the first terminal;
and determining a data template corresponding to the second data according to the equipment identification and the type.
10. The data transmission method according to claim 7, wherein after the step of determining a preset order according to the type and configuring a field name for each value field according to the preset order to obtain the first data corresponding to the second data, the method further comprises:
acquiring a storage serial number, a sending serial number and the quantity associated with the second data;
when the quantity of the received second data with the same sending sequence number is smaller than the obtained quantity, determining the storage sequence number corresponding to the second data which is not received according to each storage sequence number;
generating a retransmission request according to the storage sequence number of the second data which is not received and the sending sequence number, and sending the retransmission request to the first terminal;
and receiving second data fed back by the first terminal based on the retransmission request.
11. The data transmission method according to claim 7, wherein after the step of the second terminal receiving the second data sent by the first terminal, the method further comprises:
extracting the type and the first version number in the second data;
determining a second version number of third data corresponding to the type stored by the second terminal;
and when the second version number is smaller than the first version number, executing the step of extracting the type and each value field in the second data.
12. The data transmission method according to any one of claims 7 to 11, wherein after the step of determining a preset order according to the type and configuring a field name for each value field according to the preset order to obtain the first data corresponding to the second data, the method further comprises:
determining a data amount of the first data;
determining a target storage space corresponding to the first data according to the data volume of the first data, wherein when the data volume of the first data is smaller than a second preset data volume, the first storage space for storing small data is determined as the target storage space, and when the data volume of the second data volume is larger than or equal to the second preset data volume, the second storage space for storing large data is determined as the target storage space;
and storing the first data to the target storage space.
13. The data transmission method according to claim 12, wherein the step of storing the first data in the target storage space is followed by further comprising:
determining a second residual storage amount and a preset residual storage amount of the target storage space;
and when the second residual storage amount is smaller than the preset residual storage amount of the target storage space, deleting the expired data in the target storage space, wherein the expired data comprises data with the access frequency smaller than the preset frequency.
14. The data transmission method of claim 13, wherein the step of determining the remaining storage amount of the target storage space and presetting the remaining storage amount is preceded by the steps of:
acquiring a first access frequency corresponding to each fourth data in a first storage space and a second access frequency corresponding to each fifth data in a second storage space, wherein the data volume of the fourth data is smaller than a preset threshold value, and the data volume of the fifth data is larger than or equal to the preset threshold value;
determining a first reference value corresponding to each fourth data according to the data volume of each fourth data and the first access frequency;
determining a second reference value corresponding to each fifth data according to the data volume of each fifth data and the second access frequency;
and setting a preset residual storage amount of the first storage space and a preset residual storage amount of the second storage space according to the sum of the first reference values, the sum of the second reference values and the storage data amount of the second terminal.
15. A data transmission system, characterized in that the data transmission system comprises a first terminal and a second terminal communicatively connected to the first terminal, wherein,
the first terminal is configured to acquire each piece of first data to be sent, determine a type of each piece of first data, extract a value field of a field name in each piece of first data, arrange the type of each piece of first data and each value field in a preset order to generate second data corresponding to each piece of first data, and send each piece of second data to a middleware, so that the middleware broadcasts each piece of second data to each second terminal;
the second terminal is configured to receive second data sent by the first terminal, extract a type and each value field in the second data, determine a preset sequence according to the type, and configure a field name for each value field according to the preset sequence to obtain the first data corresponding to the second data.
16. A terminal, characterized in that the terminal comprises:
the acquisition module is used for acquiring each piece of first data to be transmitted;
the determining module is used for determining the type of each piece of first data and extracting a value field of a field name in each piece of first data;
the arrangement module is used for arranging the type of each piece of first data and each value field in a preset sequence to generate second data corresponding to each piece of first data;
and the broadcasting module is used for sending each piece of second data to the middleware so that the middleware broadcasts each piece of second data to each second terminal.
17. A terminal, characterized in that the terminal comprises:
the receiving module is used for receiving second data sent by the first terminal;
the extraction module is used for extracting types and each value field from the second data;
and the configuration module is used for determining a preset sequence according to the types, and configuring a field name for each value-taking field according to the preset sequence to obtain the first data corresponding to the second data.
18. A terminal, characterized in that the terminal comprises: memory, processor and transmission program stored on the memory and executable on the processor, which when executed by the processor implements the steps of the data transmission method according to any one of claims 1 to 14.
19. A computer-readable storage medium, characterized in that the computer-readable storage medium stores a transmission program which, when executed by a processor, implements the steps of the data transmission method according to any one of claims 1 to 14.
CN202010604477.6A 2020-06-29 2020-06-29 Terminal, data transmission method, system, and computer-readable storage medium Active CN111711639B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010604477.6A CN111711639B (en) 2020-06-29 2020-06-29 Terminal, data transmission method, system, and computer-readable storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010604477.6A CN111711639B (en) 2020-06-29 2020-06-29 Terminal, data transmission method, system, and computer-readable storage medium

Publications (2)

Publication Number Publication Date
CN111711639A true CN111711639A (en) 2020-09-25
CN111711639B CN111711639B (en) 2023-04-18

Family

ID=72544411

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010604477.6A Active CN111711639B (en) 2020-06-29 2020-06-29 Terminal, data transmission method, system, and computer-readable storage medium

Country Status (1)

Country Link
CN (1) CN111711639B (en)

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20100036872A1 (en) * 2008-08-08 2010-02-11 Hitachi, Ltd. Data management method
CN105847378A (en) * 2016-04-13 2016-08-10 北京思特奇信息技术股份有限公司 Big data synchronizing method and system
CN105868364A (en) * 2016-03-29 2016-08-17 中国电子科技集团公司第二十八研究所 Structuralized data representation method based on byte stream
CN106777292A (en) * 2016-12-29 2017-05-31 北京神州绿盟信息安全科技股份有限公司 A kind of Data Serialization method and device
CN109167750A (en) * 2018-07-06 2019-01-08 北京金山安全软件有限公司 Data packet transmission method and device, electronic equipment and storage medium
CN109840251A (en) * 2018-12-29 2019-06-04 北京奥鹏远程教育中心有限公司 A kind of big data aggregate query method

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20100036872A1 (en) * 2008-08-08 2010-02-11 Hitachi, Ltd. Data management method
CN105868364A (en) * 2016-03-29 2016-08-17 中国电子科技集团公司第二十八研究所 Structuralized data representation method based on byte stream
CN105847378A (en) * 2016-04-13 2016-08-10 北京思特奇信息技术股份有限公司 Big data synchronizing method and system
CN106777292A (en) * 2016-12-29 2017-05-31 北京神州绿盟信息安全科技股份有限公司 A kind of Data Serialization method and device
CN109167750A (en) * 2018-07-06 2019-01-08 北京金山安全软件有限公司 Data packet transmission method and device, electronic equipment and storage medium
CN109840251A (en) * 2018-12-29 2019-06-04 北京奥鹏远程教育中心有限公司 A kind of big data aggregate query method

Also Published As

Publication number Publication date
CN111711639B (en) 2023-04-18

Similar Documents

Publication Publication Date Title
CN108510389B (en) Intelligent contract calling method and device based on block chain and readable storage medium
CN102289480B (en) Method and system for updating data
CN109408132A (en) A kind of configuration file processing method and server, user terminal, storage medium
CN111221469B (en) Method, device and system for synchronizing cache data
WO2021051747A1 (en) Data update method, system and device, electronic device, and computer storage medium
CN109547524B (en) User behavior storage method, device, equipment and storage medium based on Internet of things
CN102006568A (en) Method, equipment and system for providing portrait service
US11212342B2 (en) Merge trees for collaboration
CN113691635B (en) Method and device for calling microservice, electronic equipment and readable storage medium
CN108076081B (en) Method, device and system for synchronizing service data
CN108959278B (en) Data synchronization method and device, electronic equipment and storage medium
US9723480B2 (en) Information processing device, server device, data communication system, data communication method, and computer-readable storage medium storing data communication program
CN111901157A (en) Service deployment method, device, equipment and medium based on k8s
CN110417876B (en) Session method, node server in distributed system and master control equipment
CN111176706A (en) Terminal upgrading method and device, electronic equipment and storage medium
US11128622B2 (en) Method for processing data request and system therefor, access device, and storage device
CN111061745A (en) Data synchronization system and method
CN107741994B (en) Data updating method and device
CN109391658B (en) Account data synchronization method and equipment, storage medium and terminal thereof
CN110798349B (en) Configuration distribution and receiving method, equipment and computer readable storage medium
CN111711639B (en) Terminal, data transmission method, system, and computer-readable storage medium
CN107786661B (en) Information synchronization method
CN107563942B (en) Logistics data batch processing method, logistics processing system and processing device
CN114422591B (en) Point-to-point communication method, data communication system, computer device, and storage medium
CN113190565B (en) Data updating method and device, storage medium and electronic device

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant