CN104780222A - General data exchange method based on HTTP and object serialization - Google Patents
General data exchange method based on HTTP and object serialization Download PDFInfo
- Publication number
- CN104780222A CN104780222A CN201510213732.3A CN201510213732A CN104780222A CN 104780222 A CN104780222 A CN 104780222A CN 201510213732 A CN201510213732 A CN 201510213732A CN 104780222 A CN104780222 A CN 104780222A
- Authority
- CN
- China
- Prior art keywords
- data
- http
- client
- service end
- compression algorithm
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Landscapes
- Information Transfer Between Computers (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a general data exchange method based on HTTP and object serialization. The general data exchange method includes the steps that (1) a data record is extracted from a data source, or the data record is pushed by the data source; (2) the data record is serialized into a data stream which is packaged into an HTTP request, and data stream element information is recoded in an HTTP request header; (3) a data processing procedure receives the HTTP request, and deserialization is carried out on the record; (4) the data record is processed through the data processing procedure. The cross-system cross-platform data exchange method is provided, and isomerous data can be conveniently exchanged by different data processing procedures based on the general data exchange method.
Description
Technical field
The present invention relates to the method for a kind of multiple system by network exchange data, specifically a kind of universal data switching method based on HTTP and object serialization.
Background technology
Exchanges data is the important foundation technology building data warehouse.Under large data age background, the generation of data and analyzing and processing are normally carried out in different geographical position.By Data Interchange Technology, the isomeric data produced everywhere can be collected and be loaded into data warehouse, become the basis of data processing.Participate in each side of exchanges data, i.e. data producer and data consumer, the technology platform relied on and first floor system different.A kind of general data pipe is set up significant between exchanges data each side.
The present invention proposes a kind of method for interchanging data of the serializing technology based on HTTP and object, exchanges sequencing method and compression algorithm that both sides determine object through consultation.Adopt the present invention, data producer and the data consumer the other side that need not consider to the dependence of certain particular memory technology or system platform, and only need be concerned about data content.
Summary of the invention
The object of this invention is to provide the method for interchanging data that a kind of cross-system is cross-platform, the data processor different based on the present invention can exchange isomeric data easily.
The present invention is achieved through the following technical solutions: the described universal data switching method based on HTTP and object serialization, and adopt http protocol message pass-along message, transport layer, based on Transmission Control Protocol, comprises the following steps:
(1) extract a data record from data source or data source pushes a data record, this data record is called as object;
(2) according to the structure of object, object is turned to data flow by sequence, is encapsulated in the message body of a HTTP request message; Wherein, record data stream metamessage in HTTP request head Headers, comprises sequencing method and object size; Territory, the path Path specific data of HTTP request message exchanges object handling program; The method territory Method specific data swap operation method of HTTP request message;
(3) target processing system of exchanges data receives HTTP request, and according to the data element information that message carries, adopt corresponding unserializing method, converting objects is the data of native system compatible format;
(4) data processor processes data subsequently, and the mode with http response after having processed returns result.
The method of object serialization is consulted by a HTTP request-response process when setting up TCP and connecting; In a data exchange process, the side initiating TCP connection in two sides of participation exchanges data is called client, and the opposing party is called service end, and the detailed process of negotiation sequence method is as follows:
(1) after setting up TCP connection, client sends a HTTP request, and in request header, Accept field carries the sequencing method list that client is supported, multiple sequencing methods of client support use branches to separate;
(2), after service end receives request, read the sequencing method list that Accept field obtains client support, then compare with the sequencing method of native system support, choose the sequencing method that one or more both sides support;
(3) service end returns a http response, and in head response, Accept field carries the sequencing method chosen;
(4) client knows the sequencing method of negotiation after receiving the response that service end beams back, follow-up of adopting with the data exchange process of service end in the sequencing method consulted.
The object serialization method that the both sides participating in exchanges data support at least comprises JSON serializing.Make both sides at least can negotiate a kind of sequencing method.
Described service end and client are transmitted after specific compression algorithm can both be selected to compress serialized data stream again, namely deposit the data after compression in message body.
Described client and service end consult the compression algorithm of optional data flow, and the process of consulting compression algorithm is as follows:
(1) after setting up TCP connection, client sends a HTTP request, and in request header, Accept-Encoding field carries the compression algorithm list that client is supported, multiple compression algorithms of client support use branches to separate;
(2), after service end receives request, read the compression algorithm list that Accept-Encoding field obtains client support, then compare with the compression algorithm of native system support, choose the compression algorithm that one or more both sides support;
(3) service end returns a http response, and in head response, Accept-Encoding field carries available compression algorithm;
(4) client knows the compression algorithm of negotiation after receiving the response that service end beams back, follow-up with the data exchange process of service end in adopt the compression algorithm of consulting to compress when needing to compress data.
The negotiation of compression algorithm and the negotiation of sequencing method are incorporated in once request and realize.
Data compression is optional, and service end or client can not support any data compression algorithm.In a data exchange process, when client-side/server-side have employed compression algorithm, Content-Encoding field in HTTP request/response message, is used to indicate adopted compression algorithm.Client or service end carry out unserializing again according to Content-Encoding field decompressed data.
When service end returns result, in http response message, adopt the method swap data consistent with HTTP request message.
Advantage of the present invention is: object to be exchanged by with a kind of method sequence of consult to decide, makes object can seamless circulation between multiple exchanges data participant.One side does not need the concrete storage organization paying close attention to the opposing party.And based on the light-weight transport protocol of HTTP, this switching method and platform are had nothing to do, and can use by the platform of all support http protocol stacks.The equipment of a large amount of existing support http protocol and program can use we brightly to carry out exchanges data easily.
Accompanying drawing explanation
Fig. 1 is exchanges data flow chart of the present invention.
Embodiment
The present invention proposes a kind of universal data switching method based on HTTP and object serialization, adopt http protocol message pass-along message when data exchange to the opposing party from a side, transport layer is based on Transmission Control Protocol, and as shown in Figure 1, it comprises the following steps:
(1) (Extract) data record (passive request exchange) or data source propelling movement (Push) data record (active request exchange) is extracted from data source.This data record is called as object.
(2) according to the structure of object, object is turned to data flow by sequence, is encapsulated in the message body (Message Body) of a HTTP request message.Record data stream metamessage (Metadata) in HTTP request head (Headers), comprises sequencing method and object size.Territory, path (Path) specific data of HTTP request message exchanges object handling program.Method territory (Method) the specific data swap operation method of HTTP request message.
(3) target processing system of exchanges data receives HTTP request, and according to the data element information that message carries, adopt corresponding unserializing method, converting objects is the data of native system compatible format.
(4) data processor processes data subsequently.Mode with http response after having processed returns result.
In above-mentioned steps, the method that object serialization adopts is determined through consultation.Data after serializing can be compressed, and compression algorithm is also determined through consultation.In a data exchange process, the side initiating TCP connection in two sides of participation exchanges data is called client, and the opposing party is called service end.Sequencing method negotiations process is as follows:
(1) after setting up TCP connection, client sends a HTTP request, and in request header, Accept field carries the sequencing method list that client is supported.Multiple sequencing methods of client support use branch to separate.
(2), after service end receives request, read the sequencing method list that Accept field obtains client support, then compare with the sequencing method of native system support, choose the sequencing method that one or more both sides support.
(3) service end returns a http response, and in head response, Accept field carries the sequencing method chosen.
(4) client knows the sequencing method of negotiation after receiving the response that service end beams back.It is follow-up that to adopt with the data exchange process of service end in the sequencing method consulted some.
The process of consulting the process of compression algorithm and negotiation sequence method is similar, and detailed process is as follows:
(1) after setting up TCP connection, client sends a HTTP request, and in request header, Accept-Encoding field carries the compression algorithm list that client is supported.Multiple compression algorithms of client support use branch to separate.
(2), after service end receives request, read the compression algorithm list that Accept-Encoding field obtains client support, then compare with the compression algorithm of native system support, choose the compression algorithm that one or more both sides support.
(3) service end returns a http response, and in head response, Accept-Encoding field carries available compression algorithm.
(4) client knows the compression algorithm of negotiation after receiving the response that service end beams back.Follow-up with the data exchange process of service end in adopt the compression algorithm of consulting to compress when needing to compress data.
Sequencing method is necessary.The each side participating in exchanges data at least must realize the JSON sequencing method of object.Data compression algorithm is optional, and each side participating in exchanges data can not support compression.
Service end processes data exchange request (can be the read-write of a secondary data, deletion action), then returns result in the mode of http response.
The present invention includes a client-side program and a serve end program, initiate a side of data exchange request as client, deal with data exchanges a side of request as service end.Program in fact can be service end is also client.Communication between client with service end adopts TCP to be connected, and the message of exchange defers to http protocol.The concrete steps of exchanges data are as follows:
Below in conjunction with drawings and Examples, the invention will be further described.
Step 1, client prepare a data record to be exchanged (being called object), then set up TCP with service end and are connected.TCP between client and service end is connected to after an exchanges data completes and can keeps.Be connected if therefore existed between client with service end, the connection that can multiplexingly exist.
After step 2, connection establishment, first client sends a conversation initialization request, for negotiation sequence method and compression algorithm to service end.Client inserts field Accept and the Accept-Encoding that two are respectively used to sequencing method and the compression algorithm of specifying this client to support in the request header (Headers) of request message.Multiple sequencing method, compression algorithm directly adopt branch to separate.
Following conversation initialization message is pointed out, client supports JSON serializing and xml serializing two kinds of serializing modes, supports gzip and snappy compression algorithm simultaneously.
HEAD/HTTP/1.1
Accept：application/json；application/xml
Accept-Encoding：gzip；snappy
After step 3, service end receive conversation initialization request, the sequencing method of the sequencing method of client support, compression algorithm and the support of this service end, compression algorithm are contrasted.In http response message, use Accept and Accept-Encoding field to return sequencing method and the compression algorithm of negotiation.
Following conversation initialization response message, negotiation result is JSON serializing and xml serializing two kinds of methods, and gzip compression algorithm.
HTTP/1.1 200 OK
Accept：application/json；application/xml
Accept-Encoding：gzip
Server：des
Step 4, client send a data exchange request to service end, object serialization to be operated, compression are encapsulated in the message body of HTTP request message.Sequencing method and the compression algorithm of the use of this clearing house is indicated at request message head.
Following data exchange request message represents: client adopts JSON sequencing method (PUT) data record to stock_price write.Do not compress after Data Serialization.
PUT/stock_price HTTP/1.1
Content-Type：application/json
Content-Length：77
{″name″：″APPL″，″price″：″126.37″，"source"：″NASDAQ″，″date″："2015-03-30″}
Step 5, service end process request after receiving client-requested.The operation of specifying by client, destination path process, and then return a http response message.
Below respond expression: data exchange operation success, and service end there is no data to return.
HTTP/1.1 200 OK
Server：des。
Claims (7)
1. based on the universal data switching method of HTTP and object serialization, it is characterized in that, described method adopts http protocol message pass-along message, and transport layer, based on Transmission Control Protocol, comprises the following steps:
(1) extract a data record from data source or data source pushes a data record, this data record is called as object;
(2) according to the structure of object, object is turned to data flow by sequence, is encapsulated in the message body of a HTTP request message; Wherein, record data stream metamessage in HTTP request head Headers, comprises sequencing method and object size; Territory, the path Path specific data of HTTP request message exchanges object handling program; The method territory Method specific data swap operation method of HTTP request message;
(3) target processing system of exchanges data receives HTTP request, and according to the data element information that message carries, adopt corresponding unserializing method, converting objects is the data of native system compatible format;
(4) data processor processes data subsequently, and the mode with http response after having processed returns result.
2. the universal data switching method based on HTTP and object serialization according to claim 1, is characterized in that: the method for object serialization is consulted by a HTTP request-response process when setting up TCP and connecting; In a data exchange process, the side initiating TCP connection in two sides of participation exchanges data is called client, and the opposing party is called service end, and the detailed process of negotiation sequence method is as follows:
(1) after setting up TCP connection, client sends a HTTP request, and in request header, Accept field carries the sequencing method list that client is supported, multiple sequencing methods of client support use branches to separate;
(2), after service end receives request, read the sequencing method list that Accept field obtains client support, then compare with the sequencing method of native system support, choose the sequencing method that one or more both sides support;
(3) service end returns a http response, and in head response, Accept field carries the sequencing method chosen;
(4) client knows the sequencing method of negotiation after receiving the response that service end beams back, follow-up of adopting with the data exchange process of service end in the sequencing method consulted.
3. the universal data switching method based on HTTP and object serialization according to claim 2, is characterized in that: include JSON serializing in the object serialization method that the both sides participating in exchanges data support.
4. the universal data switching method based on HTTP and object serialization according to claim 2, it is characterized in that: described service end and client are transmitted after specific compression algorithm can both be selected to compress serialized data stream again, namely deposit the data after compression in message body.
5. the universal data switching method based on HTTP and object serialization according to claim 4, is characterized in that: described client and service end consult the compression algorithm of optional data flow, and the process of consulting compression algorithm is as follows:
(1) after setting up TCP connection, client sends a HTTP request, and in request header, Accept-Encoding field carries the compression algorithm list that client is supported, multiple compression algorithms of client support use branches to separate;
(2), after service end receives request, read the compression algorithm list that Accept-Encoding field obtains client support, then compare with the compression algorithm of native system support, choose the compression algorithm that one or more both sides support;
(3) service end returns a http response, and in head response, Accept-Encoding field carries available compression algorithm;
(4) client knows the compression algorithm of negotiation after receiving the response that service end beams back, follow-up with the data exchange process of service end in adopt the compression algorithm of consulting to compress when needing to compress data.
6. the universal data switching method based on HTTP and object serialization according to claim 5, it is characterized in that: in a data exchange process, when client or service end have employed compression algorithm, use Content-Encoding field to indicate adopted compression algorithm in HTTP request or response message, client or service end carry out unserializing again according to Content-Encoding field decompressed data.
7. the universal data switching method based on HTTP and object serialization according to claim 2, is characterized in that: when service end returns result, adopts the method swap data consistent with HTTP request message in http response message.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510213732.3A CN104780222B (en) | 2015-04-29 | 2015-04-29 | Universal data switching method based on HTTP and object serialization |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510213732.3A CN104780222B (en) | 2015-04-29 | 2015-04-29 | Universal data switching method based on HTTP and object serialization |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN104780222A true CN104780222A (en) | 2015-07-15 |
| CN104780222B CN104780222B (en) | 2019-03-12 |
Family
ID=53621456
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201510213732.3A Active CN104780222B (en) | 2015-04-29 | 2015-04-29 | Universal data switching method based on HTTP and object serialization |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN104780222B (en) |
Cited By (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106648565A (en) * | 2015-10-30 | 2017-05-10 | 北京国双科技有限公司 | Request sending method and apparatus |
| CN107040539A (en) * | 2017-04-20 | 2017-08-11 | 广州华多网络科技有限公司 | A kind of protocol data bag construction method, device and computer system |
| CN107229670A (en) * | 2017-04-10 | 2017-10-03 | 中国科学院信息工程研究所 | Conventional data serializing and unserializing method based on Avro |
| CN107332854A (en) * | 2017-07-17 | 2017-11-07 | 联动优势科技有限公司 | A kind of sequence of message machinery of consultation and service providing device |
| CN109039542A (en) * | 2018-08-16 | 2018-12-18 | 深圳市共进电子股份有限公司 | A kind of transfer approach and device of interactive signaling |
| CN109753270A (en) * | 2017-11-01 | 2019-05-14 | 中国石油化工股份有限公司 | A kind of expansible drilling well business datum exchange system and method |
| CN109862095A (en) * | 2019-01-30 | 2019-06-07 | 新华三大数据技术有限公司 | Data processing method and device based on Web service |
| CN110365690A (en) * | 2019-07-19 | 2019-10-22 | 迈普通信技术股份有限公司 | Flow collection method, apparatus and storage medium |
| CN110912803A (en) * | 2019-11-14 | 2020-03-24 | 富途网络科技(深圳)有限公司 | IM message transmission method and terminal |
| WO2023084418A3 (en) * | 2021-11-12 | 2023-07-13 | Alpha Sanatorium Technologies Inc. | Method and system for optimizing transmission of serialized data using dynamic, adaptive slicing and reduction of serialized data |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20050108627A1 (en) * | 2003-11-13 | 2005-05-19 | International Business Machines Corporation | Serialization and preservation of objects |
| CN101895520A (en) * | 2009-05-20 | 2010-11-24 | 华为技术有限公司 | Widget system data sharing method, server and data sharing system |
| CN102111437A (en) * | 2010-12-30 | 2011-06-29 | 浪潮集团山东通用软件有限公司 | Simple object access protocol (SOAP) extension-based WEB service transmission compression method |
-
2015
- 2015-04-29 CN CN201510213732.3A patent/CN104780222B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20050108627A1 (en) * | 2003-11-13 | 2005-05-19 | International Business Machines Corporation | Serialization and preservation of objects |
| CN101895520A (en) * | 2009-05-20 | 2010-11-24 | 华为技术有限公司 | Widget system data sharing method, server and data sharing system |
| CN102111437A (en) * | 2010-12-30 | 2011-06-29 | 浪潮集团山东通用软件有限公司 | Simple object access protocol (SOAP) extension-based WEB service transmission compression method |
Non-Patent Citations (2)
| Title |
|---|
| MIKE WASSON: "Content Negotiation", 《HTTP://WWW.ASP.NET/WEB-API/OVERVIEW/FORMATS-AND-MODEL-BINDING/CONTENT-NEGOTIATION》 * |
| 毛国勇等: "基于序列化及压缩的网格数据传输性能改进", 《计算机工程》 * |
Cited By (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106648565A (en) * | 2015-10-30 | 2017-05-10 | 北京国双科技有限公司 | Request sending method and apparatus |
| CN107229670A (en) * | 2017-04-10 | 2017-10-03 | 中国科学院信息工程研究所 | Conventional data serializing and unserializing method based on Avro |
| CN107040539A (en) * | 2017-04-20 | 2017-08-11 | 广州华多网络科技有限公司 | A kind of protocol data bag construction method, device and computer system |
| CN107332854A (en) * | 2017-07-17 | 2017-11-07 | 联动优势科技有限公司 | A kind of sequence of message machinery of consultation and service providing device |
| CN109753270A (en) * | 2017-11-01 | 2019-05-14 | 中国石油化工股份有限公司 | A kind of expansible drilling well business datum exchange system and method |
| CN109039542A (en) * | 2018-08-16 | 2018-12-18 | 深圳市共进电子股份有限公司 | A kind of transfer approach and device of interactive signaling |
| CN109862095A (en) * | 2019-01-30 | 2019-06-07 | 新华三大数据技术有限公司 | Data processing method and device based on Web service |
| CN110365690A (en) * | 2019-07-19 | 2019-10-22 | 迈普通信技术股份有限公司 | Flow collection method, apparatus and storage medium |
| CN110912803A (en) * | 2019-11-14 | 2020-03-24 | 富途网络科技(深圳)有限公司 | IM message transmission method and terminal |
| CN110912803B (en) * | 2019-11-14 | 2022-03-08 | 富途网络科技(深圳)有限公司 | IM message transmission method and terminal |
| WO2023084418A3 (en) * | 2021-11-12 | 2023-07-13 | Alpha Sanatorium Technologies Inc. | Method and system for optimizing transmission of serialized data using dynamic, adaptive slicing and reduction of serialized data |
Also Published As
| Publication number | Publication date |
|---|---|
| CN104780222B (en) | 2019-03-12 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN104780222A (en) | General data exchange method based on HTTP and object serialization | |
| US9531802B2 (en) | Data transmission and reception system | |
| KR101859235B1 (en) | System and method of multi-media conferencing between universal plug and play (upnp) enabled telephony devices and wireless area network (wan) devices | |
| CN105531979B (en) | It is a kind of for exchanging the method and server of data between a client and a server | |
| WO2015165395A1 (en) | Video playback method and apparatus | |
| EP3907736A3 (en) | Disaggregated data storage and retrieval system | |
| KR101663009B1 (en) | Server, apparatus and method for managing storing of messages in communication network | |
| CA3116817A1 (en) | Method of delivering advertisement information, method of implementing_electronic voucher, and system | |
| CN102457573A (en) | Data communication method and information processing device | |
| CN103546771B (en) | A kind of TV programme comment processing method and system based on intelligent terminal | |
| US20210258355A1 (en) | Data communication system and method | |
| CN110706824A (en) | Interactive method, apparatus, server, system and medium for remote consultation | |
| US20210067472A1 (en) | Transcription bot for group communications | |
| CN115473876A (en) | Method, device and system for transmitting real-time streaming media data and storage medium | |
| CN114650178A (en) | Big data exchange method and system | |
| US20150026123A1 (en) | Size-based data synchronization | |
| CN100446567C (en) | Apparatus and method for realizing P2P stream broadcasting in information family appliances | |
| US20130191384A1 (en) | System and method for decoding and/or encoding a text message or instant message sent by a wireless device and transmitting audio and/or video content to the recipient of the text message or instant message based on key words in the text message | |
| CN103580951A (en) | Method of comparing outputs in a plurality of information systems | |
| CN106612307B (en) | A kind of implementation method and device of always online business | |
| CN114301880B (en) | Three-dimensional data transmission method, electronic equipment and signaling server | |
| CN105429981A (en) | Game voice transmission method, terminal, voice service module and game system | |
| CN106874278A (en) | A kind of online method for previewing, device, system and cloud storage platform | |
| CN104954368A (en) | File processing method and system in direct recording and broadcasting interactive system | |
| CN108111575A (en) | A kind of expansible client computer and virtual-machine data interactive system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| EXSB | Decision made by sipo to initiate substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CP01 | Change in the name or title of a patent holder | ||
| CP01 | Change in the name or title of a patent holder |
Address after: 214135 Block C, International Innovation Park of China Sensor Network, 200 Linghu Avenue, Wuxi New District, Jiangsu Province Patentee after: JIANGSU R & D CENTER FOR INTERNET OF THINGS Patentee after: Beijing Kedian High-Tech Co.,Ltd. Address before: 214135 Block C, International Innovation Park of China Sensor Network, 200 Linghu Avenue, Wuxi New District, Jiangsu Province Patentee before: JIANGSU R & D CENTER FOR INTERNET OF THINGS Patentee before: HIGH TECH Co. IEAS BEIJING CHINA |