CN113114606A

CN113114606A - Distribution data acquisition method and device

Info

Publication number: CN113114606A
Application number: CN202010023186.8A
Authority: CN
Inventors: 虞海明
Original assignee: Beijing Jingdong Zhenshi Information Technology Co Ltd
Current assignee: Beijing Jingdong Zhenshi Information Technology Co Ltd
Priority date: 2020-01-09
Filing date: 2020-01-09
Publication date: 2021-07-13
Anticipated expiration: 2040-01-09
Also published as: CN113114606B

Abstract

The invention discloses a distribution data acquisition method and device, and relates to the technical field of logistics. One embodiment of the method comprises: receiving a distribution data message sent by a terminal based on a UDP protocol; parsing a delivery data packet from said delivery data packet; and caching the distribution data packet to a data buffer window. On one hand, the embodiment can avoid frequent connection establishment, greatly reduce the message length, reduce the network resource consumption and improve the transmission speed; in addition, based on a data buffering mechanism, the reliability of the protocol can be ensured.

Description

Distribution data acquisition method and device

Technical Field

The invention relates to the technical field of logistics, in particular to a distribution data acquisition method and device.

Background

And the distribution terminal all-in-one machine (POS machine) collects data of the position of one equipment terminal every second, and the data is uploaded to the server as a coordinate point of the trajectory of a distributor. The client requests incremental or full amount of trajectory data for a period of time for presentation to the client to display the current dispenser location and the delivery trajectory.

In the prior art, a POS machine uploads acquired coordinate data in real time by using a short connection of an HTTP (Hyper Text Transport Protocol), and a server stores and provides a relevant data interface to finally display a trajectory of a distributor at a client.

In the process of implementing the invention, the inventor finds that at least the following problems exist in the prior art:

(1) since the HTTP short connection must contain header information, the data packet length is large, consuming a large amount of network resources.

(2) The HTTP protocol is slow to transmit.

Disclosure of Invention

In view of this, embodiments of the present invention provide a method and an apparatus for acquiring distribution data, which on one hand can avoid frequently creating a connection, and greatly reduce the message length, reduce network resource consumption, and improve the transmission speed; in addition, based on a data buffering mechanism, the reliability of the protocol can be ensured.

To achieve the above object, according to an aspect of an embodiment of the present invention, there is provided a method of distributing data collection, including:

receiving a distribution data message sent by a terminal based on a UDP protocol;

parsing a delivery data packet from said delivery data packet;

and caching the distribution data packet to a data buffer window.

Optionally, after buffering the delivery data packet into a data buffer window, the method further includes:

judging whether the number of the distributed data packages in the data buffer window in the preset time meets a first preset condition or not;

if so, controlling the terminal to adopt a UDP protocol to send a distribution data packet cached to the next data buffer window;

otherwise, updating the retransmission times of the terminal, and controlling the terminal to retransmit the distribution data packet corresponding to the data buffer window.

Optionally, before controlling the terminal to retransmit the delivery data packet corresponding to the data buffer window, the method further includes:

and determining the UDP availability according to the updated retransmission times, and if the UDP availability is less than a preset first threshold, adjusting the message protocol of the terminal to a TCP protocol.

Optionally, after the message protocol of the terminal is adjusted to the TCP protocol, the method further includes:

and detecting the UDP availability periodically, and adjusting the message protocol of the terminal to the UDP protocol when the UDP availability is larger than a preset second threshold.

Optionally, the delivery data packet includes the following fields: a terminal identification field, a check field, a dispatcher identification field, a longitude field and a latitude field; where each field takes four bytes.

According to a second aspect of the embodiments of the present invention, there is provided an apparatus for delivery data collection, including:

the receiving unit is used for receiving a distribution data message sent by a terminal based on a UDP protocol;

an analysis unit configured to analyze the delivery packet from the delivery data packet;

and the buffer unit is used for buffering the distribution data packet to a data buffer window.

Optionally, the apparatus further comprises: the switching unit is used for judging whether the number of the distributed data packages in the data buffer window meets a first preset condition within a preset time after the distributed data packages are cached to the data buffer window;

Optionally, the switching unit is further configured to: before controlling the terminal to retransmit the distribution data packet corresponding to the data buffer window, determining the UDP availability according to the updated retransmission times, and if the UDP availability is smaller than a preset first threshold, adjusting the message protocol of the terminal to a TCP protocol.

Optionally, the switching unit is further configured to: and after the message protocol of the terminal is adjusted to a TCP protocol, periodically detecting the UDP availability, and when the UDP availability is greater than a preset second threshold, adjusting the message protocol of the terminal to the UDP protocol.

According to a third aspect of the embodiments of the present invention, there is provided an electronic device for delivery data collection, including:

one or more processors;

a storage device for storing one or more programs,

when the one or more programs are executed by the one or more processors, the one or more processors are caused to implement the method provided by the first aspect of the embodiments of the present invention.

According to a fourth aspect of embodiments of the present invention, there is provided a computer readable medium, on which a computer program is stored, which when executed by a processor, implements the method provided by the first aspect of embodiments of the present invention.

One embodiment of the above invention has the following advantages or benefits: by sending the message based on the UDP protocol, on one hand, frequent connection establishment can be avoided, the message length is greatly reduced, the network resource consumption is reduced, and the transmission speed is improved; on the other hand, the hardware resource consumption of the receiving end for analyzing the message can be reduced, and the system stability is improved. In addition, based on a data buffering mechanism, the reliability of the protocol can be ensured. Based on the availability and the periodic detection mechanism, specifically, when the UDP availability is smaller than a preset first threshold, the message protocol of the terminal is adjusted to the TCP protocol, and when the UDP availability is larger than a preset second threshold, the message protocol of the terminal is adjusted to the UDP protocol, so that the transfer protocol switching can be realized, and the high availability of the protocol is ensured.

Further effects of the above-mentioned non-conventional alternatives will be described below in connection with the embodiments.

Drawings

The drawings are included to provide a better understanding of the invention and are not to be construed as unduly limiting the invention. Wherein:

FIG. 1 is a schematic diagram of a main flow of a method of distributed data acquisition according to an embodiment of the present invention;

FIG. 2 is a schematic diagram of network protocol conversion in an embodiment of the invention;

fig. 3 is a schematic diagram of a UDP protocol packet in an embodiment of the present invention;

FIG. 4 is a schematic diagram of the principal components of an apparatus for distribution data acquisition in an embodiment of the present invention;

FIG. 5 is an exemplary system architecture diagram in which embodiments of the present invention may be employed;

fig. 6 is a schematic block diagram of a computer system suitable for use in implementing a terminal device or server of an embodiment of the invention.

Detailed Description

Exemplary embodiments of the present invention are described below with reference to the accompanying drawings, in which various details of embodiments of the invention are included to assist understanding, and which are to be considered as merely exemplary. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the invention. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.

In the prior art, a POS machine uploads collected coordinate data in real time by adopting HTTP short connection, and the following problems exist: (1) because the HTTP short connection must contain the header information, the data message length is large, and a large amount of network resources are consumed; (2) the HTTP protocol is slow to transmit. In the invention, the terminal defaults to adopt a UDP (User Datagram Protocol) Protocol to send the message.

According to one aspect of an embodiment of the present invention, a method of delivery data collection is provided.

Fig. 1 is a schematic diagram of a main flow of a method for collecting distribution data according to an embodiment of the present invention, and as shown in fig. 1, the method for collecting distribution data includes:

step S101, receiving a distribution data message sent by a terminal based on a UDP protocol;

step S102, analyzing a distribution data packet from the distribution data message;

step S103, caching the distribution data packet to a data buffer window.

By sending the message based on the UDP protocol, on one hand, frequent connection establishment can be avoided, the message length is greatly reduced, the network resource consumption is reduced, and the transmission speed is improved; on the other hand, the hardware resource consumption of the receiving end for analyzing the message can be reduced, and the system stability is improved. In addition, based on a data buffering mechanism, the reliability of the protocol can be ensured.

Optionally, after buffering the delivery data packet into a data buffer window, the method further includes: judging whether the number of the distributed data packages in the data buffer window in the preset time meets a first preset condition or not; if so, controlling the terminal to adopt a UDP protocol to send a distribution data packet cached to the next data buffer window; otherwise, updating the retransmission times of the terminal, and controlling the terminal to retransmit the distribution data packet corresponding to the data buffer window.

The first preset condition can be selectively set according to actual conditions. In some embodiments, the first predetermined condition is that the number of delivered packets in the data buffer window within the predetermined time is equal to the buffer threshold of the data buffer window, for example, the buffer threshold of the data buffer window is N, and the number of delivered packets in the data buffer window within the predetermined time is also equal to N. In other embodiments, the first predetermined condition is that a ratio between the number of delivered packets in the data buffer window and a buffer threshold of the data buffer window within a predetermined time is greater than or equal to a predetermined ratio threshold, for example, the buffer threshold of the data buffer window is N, and the number of delivered packets in the data buffer window within the predetermined time is also equal to 90% N.

And when the number of the distribution data packages in the data buffer window meets a first preset condition within a preset time, the current data buffer window is considered to be completely acquired, and the next data buffer window is slid to continue to acquire distribution data.

When the number of the distributed data packages in the data buffer window does not meet a first preset condition within a preset time, updating the retransmission times of the terminal, so that the UDP availability is determined conveniently according to the retransmission times; and controlling the terminal to retransmit the distribution data packet corresponding to the data buffer window, so that the distribution data can be collected to the maximum extent, and the problem of data loss caused by the failure of receiving part of the distribution data packets is prevented.

Optionally, before controlling the terminal to retransmit the delivery data packet corresponding to the data buffer window, the method further includes: and determining the UDP availability according to the updated retransmission times, and if the UDP availability is less than a preset first threshold, adjusting the message protocol of the terminal to a TCP protocol, as shown in FIG. 2. When the UDP availability is smaller than the preset first threshold, the fact that the UDP protocol is not suitable for sending the distribution data is indicated, for example, the transmission speed is low or the transmission fails, the message protocol of the terminal is adjusted to the TCP protocol, the terminal can send the distribution data through the TCP protocol, the transmission speed is improved, and the reliability and high availability of the protocol are guaranteed.

Optionally, after the message protocol of the terminal is adjusted to the TCP protocol, the method further includes: and detecting the UDP availability periodically, and adjusting the message protocol of the terminal to the UDP protocol when the UDP availability is larger than a preset second threshold. By regular detection, when the network environment is suitable for transmitting and distributing data by adopting a UDP (user Datagram protocol), the message protocol of the terminal can be timely adjusted to the UDP, on one hand, frequent connection establishment caused by adopting a TCP (transmission control protocol) protocol is avoided, the message length is greatly reduced, the network resource consumption is reduced, and the transmission speed is improved; on the other hand, the hardware resource consumption of the receiving end for analyzing the message can be reduced, and the system stability is improved.

It should be noted that the first threshold and the second threshold mentioned in the present invention may be selectively set according to actual situations. Typically, the first threshold is less than the second threshold. In an optional embodiment of controlling the transmission protocol switching by simultaneously adopting the first threshold and the second threshold, assuming that the current UDP availability is T, the UDP protocol is currently used to transmit the message, and T > the second threshold > the first threshold, if T is decreased to T ', the second threshold is greater than or equal to T ' ≧ the first threshold, at this time, the UDP protocol is still used to transmit the message, and until T ' < the first threshold, the TCP protocol is used to transmit the message instead. Assuming that the current UDP availability is T, the TCP protocol is currently used for transmitting the message, and the second threshold is greater than the first threshold and greater than the first threshold, if the T is increased to T ', the second threshold is greater than T ' and is greater than or equal to the first threshold, the TCP protocol is still used for transmitting the message at the moment, and the UDP protocol is used for transmitting the message until T ' is greater than the second threshold.

The invention can realize the switching of the transmission protocol based on the availability ratio and the periodic detection mechanism, and ensures the high availability of the protocol.

Optionally, the delivery data packet includes the following fields: a terminal identification field, a check field, a dispatcher identification field, a longitude field and a latitude field; where each field takes four bytes. Fig. 3 is a schematic diagram of a UDP protocol packet according to an embodiment of the present invention, and as shown in fig. 3, the first 4 bytes of the data packet are distributed to store CIDs; bytes 5-8 store the RSN; bytes 9-12 store the operatorNo; bytes 13-116 store longituude; bytes 17-20 store latitude. The CID, i.e., the client ID, represents a unique identifier of the terminal. RSN (Robust Security Network) is used to check the user's validity. The RSN is an authentication identification issued after a distributor logs in the system, the identification is carried by data in the logging time limit to indicate that a user is legal, if the identification is inconsistent with the server, the user needs to log in again, and the RSN is obtained again. In fig. 3, operator no represents the card dealer identification, longitude represents the accuracy, and latitude represents the latitude.

By adopting the UDP protocol message, the message length can be greatly reduced, the network resource consumption is reduced, and the transmission speed is improved; hardware resource consumption of message analysis of the receiving end is reduced, and system stability is improved.

According to a second aspect of the embodiments of the present invention, there is provided an apparatus for implementing the above method.

Fig. 4 is a schematic diagram illustrating the main components of an apparatus for collecting distribution data according to an embodiment of the present invention, and as shown in fig. 4, an apparatus 400 for collecting distribution data includes:

a receiving unit 401, configured to receive a distribution data packet sent by a terminal based on a UDP protocol;

an analyzing unit 402 configured to analyze the delivery packet from the delivery data packet;

the buffer unit 403 buffers the delivery packet in a data buffer window.

one or more processors;

a storage device for storing one or more programs,

Fig. 5 illustrates an exemplary system architecture 500 to which the method of distribution data collection or the apparatus of distribution data collection of embodiments of the present invention may be applied.

As shown in fig. 5, the system architecture 500 may include

terminal devices

501, 502, 503, a network 504, and a server 505. The network 504 serves to provide a medium for communication links between the

terminal devices

501, 502, 503 and the server 505. Network 504 may include various connection types, such as wired, wireless communication links, or fiber optic cables, to name a few.

The user may use the

terminal devices

501, 502, 503 to interact with the server 505 over the network 504 to receive or send messages (e.g., delivery data upload requests, delivery trajectory requests), and the like. The

terminal devices

501, 502, 503 may have installed thereon various communication client applications, such as shopping-like applications, web browser applications, search-like applications, instant messaging tools, mailbox clients, social platform software, etc. (by way of example only).

The

terminal devices

501, 502, 503 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop portable computers, desktop computers, and the like.

The server 505 may be a server providing various services, such as a background management server (for example only) providing support for shopping websites browsed by users using the

terminal devices

501, 502, 503. The backend management server may analyze and otherwise process data such as the received delivery trajectory query request, and feed back a processing result (for example, delivery trajectory information — just an example) to the terminal device.

It should be noted that the method for collecting the distribution data provided by the embodiment of the present invention is generally executed by the server 505, and accordingly, the device for collecting the distribution data is generally disposed in the server 505.

It should be understood that the number of terminal devices, networks, and servers in fig. 5 is merely illustrative. There may be any number of terminal devices, networks, and servers, as desired for implementation.

Referring now to FIG. 6, a block diagram of a computer system 600 suitable for use with a terminal device implementing an embodiment of the invention is shown. The terminal device shown in fig. 6 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiments of the present invention.

As shown in fig. 6, the computer system 600 includes a Central Processing Unit (CPU)601 that can perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM)602 or a program loaded from a storage section 608 into a Random Access Memory (RAM) 603. In the RAM 603, various programs and data necessary for the operation of the system 600 are also stored. The CPU 601, ROM 602, and RAM 603 are connected to each other via a bus 604. An input/output (I/O) interface 605 is also connected to bus 604.

The following components are connected to the I/O interface 605: an input portion 606 including a keyboard, a mouse, and the like; an output portion 607 including a display such as a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and the like, and a speaker; a storage section 608 including a hard disk and the like; and a communication section 609 including a network interface card such as a LAN card, a modem, or the like. The communication section 609 performs communication processing via a network such as the internet. The driver 610 is also connected to the I/O interface 605 as needed. A removable medium 611 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 610 as necessary, so that a computer program read out therefrom is mounted in the storage section 608 as necessary.

In particular, according to the embodiments of the present disclosure, the processes described above with reference to the flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method illustrated in the flow chart. In such an embodiment, the computer program may be downloaded and installed from a network through the communication section 609, and/or installed from the removable medium 611. The computer program performs the above-described functions defined in the system of the present invention when executed by the Central Processing Unit (CPU) 601.

It should be noted that the computer readable medium shown in the present invention can be a computer readable signal medium or a computer readable storage medium or any combination of the two. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present invention, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In the present invention, however, a computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, fiber optic cable, RF, etc., or any suitable combination of the foregoing.

The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams or flowchart illustration, and combinations of blocks in the block diagrams or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

The units described in the embodiments of the present invention may be implemented by software or hardware. The described units may also be provided in a processor, and may be described as: a processor comprising: the receiving unit is used for receiving a distribution data message sent by a terminal based on a UDP protocol; an analysis unit configured to analyze the delivery packet from the delivery data packet; and the buffer unit is used for buffering the distribution data packet to a data buffer window. The names of these units do not form a limitation to the unit itself in some cases, and for example, the receiving unit may be further described as a "unit that buffers the delivery packet to the data buffer window".

As another aspect, the present invention also provides a computer-readable medium that may be contained in the apparatus described in the above embodiments; or may be separate and not incorporated into the device. The computer readable medium carries one or more programs which, when executed by a device, cause the device to comprise: receiving a distribution data message sent by a terminal based on a UDP protocol; parsing a delivery data packet from said delivery data packet; and caching the distribution data packet to a data buffer window.

According to the technical scheme of the embodiment of the invention, the method has the following advantages or beneficial effects: by sending the message based on the UDP protocol, on one hand, frequent connection establishment can be avoided, the message length is greatly reduced, the network resource consumption is reduced, and the transmission speed is improved; on the other hand, the hardware resource consumption of the receiving end for analyzing the message can be reduced, and the system stability is improved. In addition, based on a data buffering mechanism, the reliability of the protocol can be ensured. Based on the availability and the periodic detection mechanism, specifically, when the UDP availability is smaller than a preset first threshold, the message protocol of the terminal is adjusted to the TCP protocol, and when the UDP availability is larger than a preset second threshold, the message protocol of the terminal is adjusted to the UDP protocol, so that the transfer protocol switching can be realized, and the high availability of the protocol is ensured.

The above-described embodiments should not be construed as limiting the scope of the invention. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions can occur, depending on design requirements and other factors. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method of distributing data collection, comprising:

receiving a distribution data message sent by a client based on a UDP protocol;

parsing a delivery data packet from said delivery data packet;

and caching the distribution data packet to a data buffer window.

2. The method of claim 1, wherein buffering the delivery packet into a data buffer window, further comprises:

3. The method of claim 1, wherein before controlling the terminal to retransmit the delivery data packet corresponding to the data buffer window, further comprising:

4. The method of claim 3, wherein after adjusting the messaging protocol of the terminal to the TCP protocol, further comprising:

5. The method according to any of claims 1-4, wherein the delivery data message comprises the following fields: a terminal identification field, a check field, a dispatcher identification field, a longitude field and a latitude field; where each field takes four bytes.

6. An apparatus for distribution data collection, comprising:

7. The apparatus of claim 6, further comprising: the switching unit is used for judging whether the number of the distributed data packages in the data buffer window meets a first preset condition within a preset time after the distributed data packages are cached to the data buffer window;

8. The apparatus of claim 6, wherein the switching unit is further to: before controlling the terminal to retransmit the distribution data packet corresponding to the data buffer window, determining the UDP availability according to the updated retransmission times, and if the UDP availability is smaller than a preset first threshold, adjusting the message protocol of the terminal to a TCP protocol.

9. An electronic device for delivery data collection, comprising:

one or more processors;

a storage device for storing one or more programs,

when executed by the one or more processors, cause the one or more processors to implement the method of any one of claims 1-6.

10. A computer-readable medium, on which a computer program is stored, which, when being executed by a processor, carries out the method according to any one of claims 1-6.