CN113497783B - Method and device for processing data - Google Patents
Method and device for processing data Download PDFInfo
- Publication number
- CN113497783B CN113497783B CN202010197209.7A CN202010197209A CN113497783B CN 113497783 B CN113497783 B CN 113497783B CN 202010197209 A CN202010197209 A CN 202010197209A CN 113497783 B CN113497783 B CN 113497783B
- Authority
- CN
- China
- Prior art keywords
- data
- receiving end
- length
- processing
- type
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000012545 processing Methods 0.000 title claims abstract description 78
- 238000000034 method Methods 0.000 title claims abstract description 45
- 230000005540 biological transmission Effects 0.000 claims description 19
- 238000004590 computer program Methods 0.000 claims description 9
- 238000010586 diagram Methods 0.000 description 11
- 230000008569 process Effects 0.000 description 10
- 238000004891 communication Methods 0.000 description 7
- 230000006870 function Effects 0.000 description 6
- 230000003287 optical effect Effects 0.000 description 4
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 230000009471 action Effects 0.000 description 2
- 230000008901 benefit Effects 0.000 description 2
- 238000006243 chemical reaction Methods 0.000 description 2
- 239000000835 fiber Substances 0.000 description 2
- 230000000644 propagated effect Effects 0.000 description 2
- 239000004065 semiconductor Substances 0.000 description 2
- 238000009825 accumulation Methods 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000007405 data analysis Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L69/00—Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass
- H04L69/18—Multiprotocol handlers, e.g. single devices capable of handling multiple protocols
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/56—Provisioning of proxy services
- H04L67/565—Conversion or adaptation of application format or content
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L69/00—Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass
- H04L69/08—Protocols for interworking; Protocol conversion
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Computer Security & Cryptography (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Information Transfer Between Computers (AREA)
Abstract
The invention discloses a method and a device for processing data, and relates to the technical field of computers. One embodiment of the method comprises the following steps: acquiring a first data set, wherein the first data set comprises at least two first data; determining the content, the length and the type of each first data in the first data set, and respectively generating each second data according to the content, the length and the type of each first data; and assembling the second data together into data to be processed according to a first arrangement sequence of the first data in the first data set, and sending the data to be processed to a data processing end. This embodiment relieves the burden on the data provider.
Description
Technical Field
The present invention relates to the field of computer technologies, and in particular, to a method and an apparatus for processing data.
Background
Existing methods of processing data include: the data providing end obtains the data set and transmits partial data in the data set to the data receiving end according to the requirement of the data receiving end.
In the process of implementing the present invention, the inventor finds that at least the following problems exist in the prior art:
the data acquisition and transmission are all responsible for the data providing end, so that the data providing end is overloaded.
Disclosure of Invention
In view of this, the embodiments of the present invention provide a method and apparatus for processing data, which can reduce the burden of a data providing end.
To achieve the above object, according to a first aspect of an embodiment of the present invention, there is provided a method of processing data.
The method for processing data, which is provided by the embodiment of the invention, is applied to a data providing end and comprises the following steps:
acquiring a first data set, wherein the first data set comprises at least two first data;
determining the content, the length and the type of each first data in the first data set, and respectively generating each second data according to the content, the length and the type of each first data;
and assembling the second data together into data to be processed according to a first arrangement sequence of the first data in the first data set, and sending the data to be processed to a data processing end.
In one embodiment, for any one of the first data in the first data set, the following method is adopted to generate second data corresponding to the first data, including:
determining a second arrangement sequence of the length, the type and the content of the first data;
and splicing the length, the type and the content of the first data into second data corresponding to the first data according to a second arrangement sequence of the length, the type and the content of the first data.
In one embodiment, the sending the data to be processed to the data processing end includes:
and transmitting the data to be processed to the data processing end through a connectionless transmission protocol.
To achieve the above object, according to a second aspect of an embodiment of the present invention, there is provided a method of processing data.
The method for processing data, which is provided by the embodiment of the invention, is applied to a data processing end and comprises the following steps:
receiving data to be processed sent by a data providing end, and analyzing a second data set from the data to be processed;
selecting data corresponding to a data receiving end from the second data set;
converting the data corresponding to the data receiving end into third data, and transmitting the third data to the data receiving end;
the data format of the third data is the same as the data format adopted by the data receiving end.
In one embodiment, parsing the second data set from the data to be processed includes:
and analyzing the second data set from the data to be processed according to the length and the type of the data.
In one embodiment, selecting data corresponding to a data receiving end from the second data set includes:
and selecting the data with the same arrangement sequence as the first arrangement sequence of the data required by the data receiving end from the second data set as the data corresponding to the data receiving end.
To achieve the above object, according to a third aspect of an embodiment of the present invention, there is provided an apparatus for processing data.
The device for processing data of the embodiment of the invention is applied to a data providing end and comprises the following components:
an acquisition unit configured to acquire a first data set including at least two first data;
the first processing unit is used for determining the content, the length and the type of each first data in the first data set and respectively generating each second data according to the content, the length and the type of each first data;
and the second processing unit is used for assembling the second data together into data to be processed according to the first arrangement sequence of the first data in the first data set, and sending the data to be processed to the data processing end.
In one embodiment, the first processing unit is configured to:
determining a second arrangement sequence of the length, the type and the content of the first data;
and splicing the length, the type and the content of the first data into second data corresponding to the first data according to a second arrangement sequence of the length, the type and the content of the first data.
In one embodiment, the second processing unit is configured to:
and transmitting the data to be processed to the data processing end through a connectionless transmission protocol.
To achieve the above object, according to a fourth aspect of an embodiment of the present invention, there is provided an apparatus for processing data.
The device for processing data of the embodiment of the invention is applied to a data processing end and comprises the following components:
the receiving unit is used for receiving the data to be processed sent by the data providing end and analyzing a second data set from the data to be processed;
the selecting unit is used for selecting data corresponding to the data receiving end from the second data set;
the third processing unit is used for converting the data corresponding to the data receiving end into third data and transmitting the third data to the data receiving end; the data format of the third data is the same as the data format adopted by the data receiving end.
In one embodiment, the receiving unit is configured to:
and analyzing the second data set from the data to be processed according to the length and the type of the data.
In one embodiment, the selection unit is for:
and selecting the data with the same arrangement sequence as the first arrangement sequence of the data required by the data receiving end from the second data set as the data corresponding to the data receiving end.
To achieve the above object, according to a fifth aspect of an embodiment of the present invention, there is provided an electronic apparatus.
An electronic device according to an embodiment of the present invention includes: one or more processors; and the storage device is used for storing one or more programs, and when the one or more programs are executed by the one or more processors, the one or more processors are enabled to realize the method for processing data provided by the embodiment of the invention.
To achieve the above object, according to a sixth aspect of the embodiments of the present invention, a computer-readable medium is provided.
A computer readable medium of an embodiment of the present invention has stored thereon a computer program which, when executed by a processor, implements a method for processing data provided by the embodiment of the present invention.
One embodiment of the above invention has the following advantages or benefits: the data providing end obtains a first data set, generates data to be processed according to the first data set and a first arrangement sequence of the first data, sends the data to the data processing end, and the data processing end obtains data corresponding to the data receiving end from the data to be processed, converts the data and transmits the data to the data receiving end. Therefore, the data transmission is decoupled from the data providing end, and the burden of the data providing end is reduced. In addition, only the content, the length and the type of the first data are transmitted, so that the bandwidth occupied by the data transmitting end for transmitting the data is reduced.
Further effects of the above-described non-conventional alternatives are described below in connection with the embodiments.
Drawings
The drawings are included to provide a better understanding of the invention and are not to be construed as unduly limiting the invention. Wherein:
FIG. 1 is a schematic diagram of the main flow of a method of processing data according to an embodiment of the invention;
FIG. 2 is a schematic diagram of the main flow of a method of processing data according to another embodiment of the invention;
FIG. 3 is an application scenario of a method of processing data according to yet another embodiment of the present invention;
FIG. 4 is a schematic diagram of the main units of an apparatus for processing data according to an embodiment of the invention;
FIG. 5 is a schematic diagram of the main units of an apparatus for processing data according to another embodiment of the invention;
FIG. 6 is an exemplary system architecture diagram in which embodiments of the present invention may be applied;
fig. 7 is a schematic diagram of a computer system suitable for use in implementing an embodiment of the invention.
Detailed Description
Exemplary embodiments of the present invention will now be described with reference to the accompanying drawings, in which various details of the embodiments of the present invention are included to facilitate understanding, and are to be considered merely exemplary. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the invention. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.
It is noted that embodiments of the invention and features of the embodiments may be combined with each other without conflict.
In the prior art, partial data in a data set is transmitted to a data receiving end through an http protocol or a tcp long connection. The http protocol is used for transmission, and the data providing end and the data receiving end are required to frequently establish connection (three-way handshake connection establishment), disconnection (four-way handshake disconnection) and the like, so that the problems of more occupied resources (the resources comprise a CPU, a memory, the number of file handles and the like, and the occupied resources are the resources of the data providing end and the resources of the data receiving end) and low transmission efficiency exist. And transmitting through tcp long connection, if the connection establishment between the data providing end and the data receiving end fails, the data providing end is failed to start, and if the data receiving end fails, the data is possibly accumulated on the data providing end continuously, so that the memory leaks.
The following describes a specific example of data provider start-up failure: when the data providing end is started, tcp long connection needs to be established with the data receiving end, and the data providing end can be caused to be started to fail due to the failure of establishing the connection. If the data providing end is a server for user login and registration, operations such as user login and registration are not available.
In order to solve the problems in the prior art, an embodiment of the present invention provides a method for processing data. As shown in fig. 1, the method is applied to a data providing end, and includes:
step S101, acquiring a first data set, where the first data set includes at least two first data.
It should be noted that, the first data is data collected by the data providing terminal.
Step S102, determining the content, the length and the type of each first data in the first data set, and generating each second data according to the content, the length and the type of each first data.
In this step, when embodied, the length of the first data and the type of the first data are determined according to the content of the first data using existing methods. The content of the first data is acquired by the data providing end.
This step is described below with a specific example: the first data set includes data one (i.e., zhang three) and data two (i.e., 30 years old), length1 represents the length of data one, type1 represents the type of data one, value1 represents the content of data one (i.e., zhang three), length2 represents the length of data two, type2 represents the type of data two, and value2 represents the content of data two (i.e., 30 years old).
Generating second data corresponding to the first data according to the length1, the type1 and the value 1; and generating second data corresponding to the second data according to the length2, the type2 and the value2.
Step 103, assembling the second data together into a data to be processed according to a first arrangement sequence of the first data in the first data set, and sending the data to be processed to a data processing end.
In this step, it should be noted that, for each first data in the first data set, the arrangement order of the first data in the first data set may be the same as the first arrangement order of the first data, or may be different from the first arrangement order of the first data. In addition, the first arrangement order of the first data may be preset. Furthermore, the arrangement order of the first data in the second data set is necessarily the same as the first arrangement order of the first data, and if the arrangement order of the first data in the first data set is different from the first arrangement order of the first data, the arrangement order of the first data in the first data set is different from the arrangement order of the first data in the second data set.
The following describes a first arrangement order of the first data in the first data set with a specific example: the first data set includes data one, data two, and data three. Wherein, the first arrangement order of the first data is 3, the first arrangement order of the second data is 2, and the first arrangement order of the third data is 1.
The data provider may be a business system.
In an embodiment of the present invention, for any one first data in the first data set, generating second data corresponding to the first data by using the following method includes:
determining a second arrangement sequence of the length, the type and the content of the first data;
and splicing the length, the type and the content of the first data into second data corresponding to the first data according to a second arrangement sequence of the length, the type and the content of the first data.
In this embodiment, the second arrangement order of the lengths of the first data, the second arrangement order of the types of the first data, and the second arrangement order of the contents of the first data may be predetermined at the time of implementation.
It should be understood that the length, type and second order of arrangement of the content of each first data are the same.
The embodiment of the invention adopts 13 bits to store the length of the first data (the length of the first data is the byte number occupied by the content of the first data), and 3 bits to store the type of the first data.
The embodiment is described below with a specific example:
the second arrangement order of the length, type and content of the first data is shown in table 1:
TABLE 1
According to the second arrangement order shown in table 1, the length of the first data, the type of the first data, and the content of the first data are assembled into second data corresponding to the first data.
In an embodiment of the present invention, sending the data to be processed to a data processing end includes:
and transmitting the data to be processed to the data processing end through a connectionless transmission protocol.
In this embodiment, the connectionless transport protocol (User Datagram Protocol, UDP for short), also known as the user datagram protocol. UDP provides a method for applications to send encapsulated IP datagrams without establishing a connection and is a connectionless transport layer protocol in the OSI (Open System Inter connection, open systems interconnection) reference model.
The maximum data value of UDP single packet transmission is 65507, which satisfies the data transmission requirement of most scenes.
In this embodiment, since connection is not required to be established before data is sent, no connection state is required to be relied on and maintained, resource occupation is saved, transmission efficiency is improved, only uplink traffic is generated in the sending process, downlink traffic is not generated, load of a network card is reduced, occupation of network bandwidth is reduced, start failure of a data providing end is not caused, and continuous accumulation of data at the data providing end is not caused.
In order to solve the problems of the prior art, another embodiment of the present invention provides a method for processing data. As shown in fig. 2, the method is applied to a data processing end, and includes:
step 201, receiving data to be processed sent by a data providing end, and analyzing a second data set from the data to be processed.
It should be noted that, the detailed description of this step is described in detail below, and will not be repeated here.
Step S202, selecting data corresponding to the data receiving end from the second data set.
It should be noted that, the detailed description of this step is described in detail below, and will not be repeated here.
Step 203, converting the data corresponding to the data receiving end into third data, and transmitting the third data to the data receiving end; the data format of the third data is the same as the data format adopted by the data receiving end.
In this step, when it is implemented, the third data is transmitted to the data receiving end through a network transmission protocol corresponding to the data receiving end. In addition, if the number of data corresponding to the data receiving end is multiple, each data corresponding to the data receiving end is respectively converted into third data, and the third data is transmitted to the data receiving end.
The data receiving end may be a message queue such as Kafka (Kafka is a high throughput distributed publish-subscribe message system that can process all action stream data of the consumer in the website).
The data corresponding to the data receiving end can be UUID, and the data format of the third data can be SQL statement format.
It should be understood that the data corresponding to the data receiving end is converted into third data, that is, the data corresponding to the data receiving end is serialized. Serialization (Serialization) is a process of converting the state information of an object into a form that can be stored or transmitted. Serialization can be performed using an open source component such as json, msgpack, protobuf.
In this embodiment, it should be noted that there may be a plurality of data receiving terminals, and the data corresponding to each data receiving terminal may be the same or different, but for each data receiving terminal, the corresponding third data generated and transmitted by the data processing terminal will be received. And data is sent to different data receiving ends, so that diversified requirements are met.
In an embodiment of the present invention, parsing a second data set from the data to be processed includes:
and analyzing the second data set from the data to be processed according to the length and the type of the data.
In this embodiment, on the basis of the example of step S102, this embodiment is described with a specific example: the data to be processed comprises: length1, type1, value1, length2, type2, and value2. And resolving value1 from the data to be processed according to length1 and type1, and resolving value2 from the data to be processed according to length2 and type 2. Since the first arrangement order of the second data is before and the arrangement order of the first data is after, the second data set includes: value2, value1.
In the embodiment of the present invention, selecting data corresponding to a data receiving end from the second data set includes:
and selecting the data with the same arrangement sequence as the first arrangement sequence of the data required by the data receiving end from the second data set as the data corresponding to the data receiving end.
In this embodiment, the embodiment is described below with a specific example:
the first data set includes: data three, data one, and data two.
The second data set includes: data one, data two, and data three.
The arrangement order of the first data in the first data set is 2;
the first permutation of data one is 1;
the arrangement order of the data I in the second data set is 1;
the arrangement order of the second data in the first data set is 3;
the first permutation order of the data two is 2;
the arrangement order of the data two in the second data set is 2;
the arrangement order of the data three in the first data set is 1;
the first permutation of data three is 3;
the arrangement order of the data three in the second data set is 3;
the first data set includes first data and second data, the first arrangement order of the first data set is 1, and the first arrangement order of the second data set is 2, so that the first data set with the arrangement order of 1 in the second data set is selected as the data corresponding to the first data set, and the second data set with the arrangement order of 2 in the second data set is also selected as the data corresponding to the first data set.
The data required by the data receiving end II comprises data I and data III, the first arrangement sequence of the data I is 1, and the first arrangement sequence of the data III is 3, so that the data I with the arrangement sequence of 1 in the second data set is selected from the second data set as the data corresponding to the data receiving end II, and the data III with the arrangement sequence of 3 in the second data set is also selected as the data corresponding to the data receiving end II.
In this embodiment, the data in the second data set, the arrangement sequence of which is the same as the first arrangement sequence of the data required by the data receiving end, is selected as the data corresponding to the data receiving end, so that the data transmission is performed according to the requirement of the data receiving end, the diversified requirement is satisfied, and the burden of the data providing end is further reduced.
Today, where information is rapidly developed, the amount of data is increasing. In particular, in the field of mobile electronic commerce, massive data of a service system needs to be collected for daily monitoring and analysis, so that a user is better served. The data acquisition comprises data acquisition and data transmission, and the data acquisition is a first link of big data analysis and is also an extremely important link. The embodiment of the invention can be applied to the above scene.
In order to solve the problems of the prior art, a further embodiment of the present invention provides a method for processing data.
As shown in fig. 3, the data provider includes a server a (i.e., service a, which may be a server for user login registration) and a server B (i.e., service B). The data processing end comprises a data collection center (the data collection center can be a server). The data receiving end comprises a Target A, a Target B, kafka, mq (a message queue) and mysql (a storage medium).
The configuration center may store the address of the data receiving end and a network transmission protocol corresponding to the data receiving end, for example 11.24.218.134. Therefore, the data processing end can transmit the third data to the data receiving end according to the address of the data receiving end and the network transmission protocol corresponding to the data receiving end. The configuration center may store a first arrangement order of data required by the data receiving end, so that the data processing end may select data corresponding to the data receiving end. The configuration center may store a conversion method corresponding to the data receiving end, so that the data processing end converts the data corresponding to the data receiving end into the third data according to the conversion method corresponding to the data receiving end.
The process of transmitting the third data is illustrated below: the data collection center transmits third data to the Target A through an http protocol; transmitting the third data to Target B via jsf protocol; support for transmitting third data to kafka, mq or mysql. Meets the diversified demands.
The process of obtaining the respective data to be processed by the server a and the server B is the same as the process described in the embodiment shown in fig. 1, and will not be described here again.
The second data set analyzed by the data collection center is a full amount of data, for example, registration data of a user, including: zhangsan, XX cell, 23 years old, 20191226 and 22.22.232.222. Where zhangsan is the user name (username), XX cell is the user address (address), 23 years old is the user age (age), 20191226 is the date (time), 22.22.232.222 is the user terminal address (ip).
the data required by targetA includes zhangsan and 23 years old, and the data format adopted by targetA is comma separated, then the third data transmitted to targetA: [ zhangsan ], [23 years ].
The data required for TargetB includes zhangsan, 20191226 and 111.111.111.111. The data format adopted by TargetB is JSON. Then, the third data transmitted to TargetB: { "username": "zhangsan", "time": "20191226", "ip": "111.111.111.111" }.
Address of TargetB: http://192.168.1.1/store.
According to http://192.168.1.1/store, { "username": "zhangsan", "time": "20191226", "ip": "111.111.111.111" } is transmitted to TargetB.
In order to solve the problems in the prior art, an embodiment of the present invention provides an apparatus for processing data. As shown in fig. 4, the apparatus, applied to a data providing terminal, includes:
the obtaining unit 401 is configured to obtain a first data set, where the first data set includes at least two first data.
The first processing unit 402 is configured to determine content, length, and type of each first data in the first data set, and generate each second data according to the content, length, and type of each first data.
The second processing unit 403 is configured to assemble the second data together into a data to be processed according to a first arrangement sequence of the first data in the first data set, and send the data to be processed to a data processing end.
In an embodiment of the present invention, the first processing unit 402 is configured to:
determining a second arrangement sequence of the length, the type and the content of the first data;
and splicing the length, the type and the content of the first data into second data corresponding to the first data according to a second arrangement sequence of the length, the type and the content of the first data.
In the embodiment of the present invention, the second processing unit 403 is configured to:
and transmitting the data to be processed to the data processing end through a connectionless transmission protocol.
It should be understood that the specific implementation of the embodiment of the present invention is the same as that of the embodiment shown in fig. 1, and will not be described herein.
In order to solve the problems of the prior art, another embodiment of the present invention provides an apparatus for processing data. As shown in fig. 5, the device is applied to a data processing end, and includes:
the receiving unit 501 is configured to receive data to be processed sent by the data providing end, and parse the second data set from the data to be processed.
And a selecting unit 502, configured to select data corresponding to a data receiving end from the second data set.
A third processing unit 503, configured to convert the data corresponding to the data receiving end into third data, and transmit the third data to the data receiving end; the data format of the third data is the same as the data format adopted by the data receiving end.
In the embodiment of the present invention, the receiving unit 501 is configured to:
and analyzing the second data set from the data to be processed according to the length and the type of the data.
In the embodiment of the present invention, the selecting unit 502 is configured to:
and selecting the data with the same arrangement sequence as the first arrangement sequence of the data required by the data receiving end from the second data set as the data corresponding to the data receiving end.
It should be understood that the specific implementation of the embodiment of the present invention is the same as that of the embodiment shown in fig. 2, and will not be described herein.
Fig. 6 illustrates an exemplary system architecture 600 to which a method of processing data or an apparatus of processing data of an embodiment of the present invention may be applied.
As shown in fig. 6, the system architecture 600 may include terminal devices 601, 602, 603, a network 604, and a server 605. The network 604 is used as a medium to provide communication links between the terminal devices 601, 602, 603 and the server 605. The network 604 may include various connection types, such as wired, wireless communication links, or fiber optic cables, among others.
A user may interact with the server 605 via the network 604 using the terminal devices 601, 602, 603 to receive or send messages, etc. Various communication client applications such as shopping class applications, web browser applications, search class applications, instant messaging tools, mailbox clients, social platform software, etc. (by way of example only) may be installed on the terminal devices 601, 602, 603.
The terminal devices 601, 602, 603 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smartphones, tablets, laptop and desktop computers, and the like.
The server 605 may be a server providing various services, such as a background management server (by way of example only) providing support for shopping-type websites browsed by users using terminal devices 601, 602, 603. The background management server may analyze and process the received data such as the product information query request, and feedback the processing result (e.g., the target push information, the product information—only an example) to the terminal device.
It should be noted that, the method for processing data provided in the embodiment of the present invention is generally executed by the server 605, and accordingly, the device for processing data is generally disposed in the server 605.
It should be understood that the number of terminal devices, networks and servers in fig. 6 is merely illustrative. There may be any number of terminal devices, networks, and servers, as desired for implementation.
Referring now to FIG. 7, there is illustrated a schematic diagram of a computer system 700 suitable for use in implementing an embodiment of the present invention. The terminal device shown in fig. 7 is only an example, and should not impose any limitation on the functions and the scope of use of the embodiment of the present invention.
As shown in fig. 7, the computer system 700 includes a Central Processing Unit (CPU) 701, which can perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM) 702 or a program loaded from a storage section 708 into a Random Access Memory (RAM) 703. In the RAM 703, various programs and data required for the operation of the system 700 are also stored. The CPU 701, ROM702, and RAM 703 are connected to each other through a bus 704. An input/output (I/O) interface 705 is also connected to bus 704.
The following components are connected to the I/O interface 705: an input section 706 including a keyboard, a mouse, and the like; an output portion 707 including a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and the like, a speaker, and the like; a storage section 708 including a hard disk or the like; and a communication section 709 including a network interface card such as a LAN card, a modem, or the like. The communication section 709 performs communication processing via a network such as the internet. The drive 710 is also connected to the I/O interface 705 as needed. A removable medium 711 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 710 as necessary, so that a computer program read therefrom is mounted into the storage section 708 as necessary.
In particular, according to embodiments of the present disclosure, the processes described above with reference to flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method shown in the flow chart. In such an embodiment, the computer program may be downloaded and installed from a network via the communication portion 709, and/or installed from the removable medium 711. The above-described functions defined in the system of the present invention are performed when the computer program is executed by a Central Processing Unit (CPU) 701.
The computer readable medium shown in the present invention may be a computer readable signal medium or a computer readable storage medium, or any combination of the two. The computer readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples of the computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this document, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In the present invention, however, the computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, with the computer-readable program code embodied therein. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination of the foregoing. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, fiber optic cable, RF, etc., or any suitable combination of the foregoing.
The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a unit, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams or flowchart illustration, and combinations of blocks in the block diagrams or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The units involved in the embodiments of the present invention may be implemented in software or in hardware. The described units may also be provided in a processor, for example, described as: a processor includes an acquisition unit, a first processing unit, and a second processing unit. The names of these units do not in any way limit the unit itself, for example, the first processing unit may also be described as "a unit for determining the content, the length and the type of each first data in the first data set, and generating each second data according to the content, the length and the type of each first data".
As another aspect, the present invention also provides a computer-readable medium that may be contained in the apparatus described in the above embodiments; or may be present alone without being fitted into the device. The computer readable medium carries one or more programs which, when executed by a device, cause the device to include: acquiring a first data set, wherein the first data set comprises at least two first data; determining the content, the length and the type of each first data in the first data set, and respectively generating each second data according to the content, the length and the type of each first data; and assembling the second data together into data to be processed according to a first arrangement sequence of the first data in the first data set, and sending the data to be processed to a data processing end.
The units involved in the embodiments of the present invention may be implemented in software or in hardware. The described units may also be provided in a processor, for example, described as: a processor includes a receiving unit, a selecting unit, and a third processing unit. The names of these units do not in any way limit the unit itself, for example, the receiving unit may also be described as "a unit that receives data to be processed sent by the data provider and parses the second data set from the data to be processed".
As another aspect, the present invention also provides a computer-readable medium that may be contained in the apparatus described in the above embodiments; or may be present alone without being fitted into the device. The computer readable medium carries one or more programs which, when executed by a device, cause the device to include: receiving data to be processed sent by a data providing end, and analyzing a second data set from the data to be processed; selecting data corresponding to a data receiving end from the second data set; converting the data corresponding to the data receiving end into third data, and transmitting the third data to the data receiving end; the data format of the third data is the same as the data format adopted by the data receiving end.
According to the technical scheme of the embodiment of the invention, the data providing end obtains the first data set, generates data to be processed according to the first data set and the first arrangement sequence of the first data, sends the data to the data processing end, and the data processing end obtains data corresponding to the data receiving end from the data to be processed, converts the data and transmits the data to the data receiving end. Therefore, the data transmission is decoupled from the data providing end, and the burden of the data providing end is reduced. In addition, only the content, the length and the type of the first data are transmitted, so that the bandwidth occupied by the data transmitting end for transmitting the data is reduced.
The above embodiments do not limit the scope of the present invention. It will be apparent to those skilled in the art that various modifications, combinations, sub-combinations and alternatives can occur depending upon design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the scope of the present invention.
Claims (5)
1. A method for processing data, applied to a data processing end, comprising:
receiving data to be processed sent by a data providing end through a connectionless transmission protocol, and analyzing a second data set from the data to be processed according to the length and the type of the data; the data to be processed is formed by the data providing end assembling second data together according to a first arrangement sequence of first data in a first data set which is obtained in advance, and the second data is generated by the data providing end according to the content, the length and the type of the first data which are determined in advance; the first data set includes at least two first data;
selecting data with the same arrangement sequence as the first arrangement sequence of the data required by the data receiving end from the second data set as data corresponding to the data receiving end;
converting the data corresponding to the data receiving end into third data, and transmitting the third data to the data receiving end; the data format of the third data is the same as the data format adopted by the data receiving end.
2. The method of claim 1, wherein for any one of the first data sets, generating second data comprises:
determining a second arrangement sequence of the length, the type and the content of the first data;
and splicing the length, the type and the content of the first data into second data corresponding to the first data according to a second arrangement sequence of the length, the type and the content of the first data.
3. An apparatus for processing data, applied to a data processing end, comprising:
the receiving unit is used for receiving the data to be processed sent by the data providing end through a connectionless transmission protocol, and analyzing a second data set from the data to be processed according to the length and the type of the data; the data to be processed is formed by the data providing end assembling second data together according to a first arrangement sequence of first data in a first data set which is obtained in advance, and the second data is generated by the data providing end according to the content, the length and the type of the first data which are determined in advance; the first data set includes at least two first data;
a selecting unit, configured to select, from the second data set, data in the second data set, where the data has an arrangement sequence that is the same as the first arrangement sequence of the data required by the data receiving end, as data corresponding to the data receiving end;
the third processing unit is used for converting the data corresponding to the data receiving end into third data and transmitting the third data to the data receiving end; the data format of the third data is the same as the data format adopted by the data receiving end.
4. An electronic device, comprising:
one or more processors;
storage means for storing one or more programs,
the one or more programs, when executed by the one or more processors, cause the one or more processors to implement the method of claim 1 or 2.
5. A computer readable medium on which a computer program is stored, characterized in that the program, when being executed by a processor, implements the method according to claim 1 or 2.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010197209.7A CN113497783B (en) | 2020-03-19 | 2020-03-19 | Method and device for processing data |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010197209.7A CN113497783B (en) | 2020-03-19 | 2020-03-19 | Method and device for processing data |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113497783A CN113497783A (en) | 2021-10-12 |
| CN113497783B true CN113497783B (en) | 2024-01-16 |
Family
ID=77993521
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010197209.7A Active CN113497783B (en) | 2020-03-19 | 2020-03-19 | Method and device for processing data |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113497783B (en) |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104468648A (en) * | 2013-09-13 | 2015-03-25 | 腾讯科技（深圳）有限公司 | Data processing system and method |
| CN105379208A (en) * | 2013-06-27 | 2016-03-02 | 徐正焕 | Multi-connection system and method for service using internet protocol |
| WO2017041398A1 (en) * | 2015-09-08 | 2017-03-16 | 安一恒通（北京）科技有限公司 | Data transmission method and device |
| CN106649342A (en) * | 2015-10-30 | 2017-05-10 | 北京国双科技有限公司 | Data processing method and apparatus in data acquisition platform |
| CN107612841A (en) * | 2017-08-21 | 2018-01-19 | 武汉斗鱼网络科技有限公司 | A kind of method, apparatus and computer equipment for transmitting data |
| CN108076058A (en) * | 2017-12-14 | 2018-05-25 | 北京博大光通物联科技股份有限公司 | Heterogeneous network communications protocol method for amalgamation processing based on self-adapting changeable long codes |
| CN110049010A (en) * | 2019-03-12 | 2019-07-23 | 广州市保伦电子有限公司 | A kind of method and system of distributed platform terminal device connection server |
| CN110492984A (en) * | 2019-07-29 | 2019-11-22 | 南方电网数字电网研究院有限公司 | Data transmission method, device, Data Analysis Services system and computer equipment |
-
2020
- 2020-03-19 CN CN202010197209.7A patent/CN113497783B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105379208A (en) * | 2013-06-27 | 2016-03-02 | 徐正焕 | Multi-connection system and method for service using internet protocol |
| CN104468648A (en) * | 2013-09-13 | 2015-03-25 | 腾讯科技（深圳）有限公司 | Data processing system and method |
| WO2017041398A1 (en) * | 2015-09-08 | 2017-03-16 | 安一恒通（北京）科技有限公司 | Data transmission method and device |
| CN106649342A (en) * | 2015-10-30 | 2017-05-10 | 北京国双科技有限公司 | Data processing method and apparatus in data acquisition platform |
| CN107612841A (en) * | 2017-08-21 | 2018-01-19 | 武汉斗鱼网络科技有限公司 | A kind of method, apparatus and computer equipment for transmitting data |
| CN108076058A (en) * | 2017-12-14 | 2018-05-25 | 北京博大光通物联科技股份有限公司 | Heterogeneous network communications protocol method for amalgamation processing based on self-adapting changeable long codes |
| CN110049010A (en) * | 2019-03-12 | 2019-07-23 | 广州市保伦电子有限公司 | A kind of method and system of distributed platform terminal device connection server |
| CN110492984A (en) * | 2019-07-29 | 2019-11-22 | 南方电网数字电网研究院有限公司 | Data transmission method, device, Data Analysis Services system and computer equipment |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113497783A (en) | 2021-10-12 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN102546794B (en) | Method for directly communicating browser client with back-end server as well as gateway and communication system | |
| WO2017174026A1 (en) | Client connection method and system | |
| CN111786939B (en) | Method, device and system for testing management platform of Internet of things | |
| US10367894B2 (en) | Information processing apparatus, method for controlling the same, non-transitory computer-readable storage medium, and information processing system | |
| US20100146112A1 (en) | Efficient communication techniques | |
| CN113382062A (en) | Data transmission method, device and system | |
| CN114979295B (en) | Gateway management method and device | |
| CN111726400A (en) | Reverse connection method, device and server-side system | |
| CN111800223B (en) | Method, device and system for generating sending message and processing receiving message | |
| CN112104679B (en) | Method, apparatus, device and medium for processing hypertext transfer protocol request | |
| CN112084042A (en) | Message processing method and device | |
| CN113778499B (en) | Method, apparatus, device and computer readable medium for publishing services | |
| CN112015383A (en) | Login method and device | |
| CN103873443A (en) | Information processing method, local proxy server and network proxy server | |
| CN113497783B (en) | Method and device for processing data | |
| CN108924773B (en) | Message processing method and device | |
| US11153399B2 (en) | Facilitating inter-proxy communication via an existing protocol | |
| CN108259576B (en) | Software and hardware real-time information transmission system and method | |
| CN112202780B (en) | Data transmission method, device, equipment and medium based on double sockets | |
| JP6212944B2 (en) | Communication system, proxy server, communication method and program | |
| CN113992641A (en) | Data processing method, device, equipment and storage medium | |
| CN113746851A (en) | Agent system and method for supporting real-time analysis of GRPC request | |
| CN112306791B (en) | Performance monitoring method and device | |
| US20230008728A1 (en) | Method and Network Node for Communication with a Non-IP Device | |
| CN112929453A (en) | Method and device for sharing session data |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |