CN106470237B - Asynchronous downloading method and system - Google Patents

Asynchronous downloading method and system Download PDF

Info

Publication number
CN106470237B
CN106470237B CN201510515897.6A CN201510515897A CN106470237B CN 106470237 B CN106470237 B CN 106470237B CN 201510515897 A CN201510515897 A CN 201510515897A CN 106470237 B CN106470237 B CN 106470237B
Authority
CN
China
Prior art keywords
downloading
asynchronous
information
request
packet
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201510515897.6A
Other languages
Chinese (zh)
Other versions
CN106470237A (en
Inventor
赵铭鑫
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Tencent Technology Shenzhen Co Ltd
Original Assignee
Tencent Technology Shenzhen Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Tencent Technology Shenzhen Co Ltd filed Critical Tencent Technology Shenzhen Co Ltd
Priority to CN201510515897.6A priority Critical patent/CN106470237B/en
Publication of CN106470237A publication Critical patent/CN106470237A/en
Application granted granted Critical
Publication of CN106470237B publication Critical patent/CN106470237B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network arrangements or protocols for supporting network services or applications
    • H04L67/01Protocols
    • H04L67/02Protocols based on web technology, e.g. hypertext transfer protocol [HTTP]

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Information Transfer Between Computers (AREA)
  • Data Exchanges In Wide-Area Networks (AREA)

Abstract

According to the asynchronous downloading method and system provided by the invention, the asynchronous downloading of the HTTPS webpage is realized by setting the IP and the port number of the server corresponding to the UR L in the DNS cache, so that the HTTPS webpage downloading function is met, the asynchronous downloading can be carried out, and the downloading efficiency of the HTTPS webpage is improved.

Description

Asynchronous downloading method and system
Technical Field
The invention relates to the field of internet, in particular to an asynchronous downloading method and an asynchronous downloading system.
Background
The HyperText transfer protocol HTTP (chinese: HyperText transfer protocol) protocol is used to transfer information between the Web browser and the Web server. The HTTP protocol transmits contents in a plaintext manner without providing any data encryption manner, and if an attacker intercepts a transmission message between the Web browser and the Web server, the attacker can directly read and understand the information therein, so that the HTTP protocol is not suitable for transmitting some sensitive information, such as a credit card number, a password, and the like.
In order to solve the defect of the HTTP Protocol, another Protocol, namely an HTTPS (Chinese: Secure Socket layer hypertext Transfer Protocol, English: Hyper Text Transfer Protocol over Secure Socket L eye) Protocol is used, an HTTP channel which aims at security is used, for the security of data transmission, the HTTPS is added with an SS L (Chinese: Secure Socket layer, English: Secure Socket L eye) Protocol on the basis of the HTTP, and the SS L verifies the identity of a server by means of a certificate and encrypts the communication between a browser and the server.
The WEB download mode is divided into two types of HTTP and FTP (chinese: File transfer protocol), which are data exchange modes between computers and also two classic download modes, and the download mode principle is very simple, namely that a user contacts with a server providing files and moves the files to the computer of the user, thereby realizing the download function.
The WEB downloading includes two modes of synchronous downloading and asynchronous downloading, the asynchronous downloading means that when a connection request is sent to a target website, the request can be continuously sent without waiting whether the downloading is completed, that is, the request does not need to wait for the data returned by the other website and continuously sends the data to the outside, most of the currently realized modes of asynchronous downloading WEB pages are HTTP WEB pages, and the WEB pages of HTTPS only support synchronous downloading at present, such as the download libraries of python, the get command and the curl command in linux. The Python library can use url 2 or requests (Python is added to openssl when compiling is installed), and the synchronous downloading of HTTPS web pages can be realized by adding some parameters by using wget commands and curl commands.
In the current technical solution, asynchronous downloading of HTTPS web pages cannot be achieved, data congestion is easily caused for downloading services with large data volume, and particularly, the data congestion is limited by the response time of the opposite website, and if a large number of links are overtime, the downloading efficiency is greatly affected.
Disclosure of Invention
In view of this, embodiments of the present invention provide an asynchronous downloading method and system.
One object of the present invention is to provide an asynchronous downloading method, comprising:
receiving a request data packet of a logic control sending module;
acquiring a uniform resource locator link UR L in the request data packet;
when the UR L is an HTTPS request, recording first information of the UR L for information statistics and packed information after completion of downloading, and storing the first information to an asynchronous request queue, where the first information includes the length of the UR L, an IP address obtained by resolution of the UR L domain name, and a logic control sending module ID for sending the UR L;
initializing the transmission request information of the UR L;
configuring a DNS cache and placing the IP and port number pointed to by the UR L into the DNS cache;
adding the sending request information and the first information into the asynchronous request queue to start downloading;
and when the asynchronous request queue is monitored to finish downloading, the downloading data is sent to the logic control sending module according to the first information.
Further, the receiving a request packet of the logic control sending module includes:
monitoring a network port;
receiving an ASN abstract syntax mark format data packet of a logic control sending module;
the obtaining the uniform resource locator link UR L in the request packet includes:
get the uniform resource locator link UR L in the ASN abstract syntax notation format packet.
Further, before acquiring the url link UR L in the ASN abstract syntax notation format packet, the method further includes:
and judging whether the ASN format data packet is a legal request or not, and if so, executing subsequent steps.
Further, the sending request information of the initialization UR L includes:
setting a header of an HTTPS request;
whether UR L hopping is supported;
whether to set an agent;
setting a variable for recording a downloading result; and
whether to verify the certificate of the HTTPS site accessed by the UR L.
Further, when it is monitored that the asynchronous request queue completes downloading, the step of sending the download data to the logic control sending module according to the first information includes:
monitoring the downloading condition of the asynchronous request queue;
when downloading is completed, packaging the downloaded data into a return packet in an ASN format;
and sending the return packet in the ASN format to the logic control sending module according to the content of the first information.
Further, after the asynchronous request queue is monitored to finish downloading, and downloading data is sent to the logic control sending module according to the first information, the method further includes:
and deleting the first information corresponding to the downloading data in the asynchronous request queue.
It is another object of the present invention to provide an asynchronous download system, comprising:
the receiving unit is used for receiving a request data packet of the logic control sending module;
an obtaining unit, configured to obtain a uniform resource locator link UR L in the request packet;
the storage unit is used for recording first information of the UR L used for information statistics and packaging information after downloading is completed when the UR L is an HTTPS request, and storing the first information to an asynchronous request queue, wherein the first information comprises UR L length, UR L domain name resolution to obtain an IP address and a logic control sending module ID for sending UR L;
an initialization unit for initializing the transmission request information of the UR L;
a configuration unit configured to configure a DNS cache and place the IP and port number pointed to by the UR L into the DNS cache;
the downloading unit is used for adding the sending request information and the first information into the asynchronous request queue to start downloading;
and the sending unit is used for sending the download data to the logic control sending module according to the first information when the asynchronous request queue finishes downloading.
Furthermore, the receiving unit is further configured to monitor a network port and receive an ASN abstract syntax notation format data packet of the logic control sending module;
the obtaining unit is also used for obtaining the uniform resource locator link UR L in the ASN abstract syntax notation format packet.
Further, the asynchronous download system further comprises:
and the judging unit is used for judging whether the ASN format data packet is a legal request or not, and informing the acquiring unit if the ASN format data packet is the legal request.
Further, the initialization unit is also used for setting a header of an HTTPS request, whether UR L jump is supported, whether an agent is set, setting a variable recording a download result, and whether a certificate of an HTTPS site accessed by the UR L is verified.
Further, the sending unit is further configured to monitor a downloading condition of the asynchronous request queue, encapsulate the downloaded data into a return packet in an ASN format when the downloading is completed, and send the return packet in the ASN format to the logic control sending module according to the content of the first information.
Further, the asynchronous downloading system further comprises:
and the deleting unit is used for deleting the first information corresponding to the downloading data in the asynchronous request queue.
According to the technical scheme, the embodiment of the invention has the following advantages:
according to the asynchronous downloading method and system provided by the invention, the asynchronous downloading of the HTTPS webpage is realized by setting the IP and the port number of the server corresponding to the UR L in the DNS cache, so that the HTTPS webpage downloading function is met, the asynchronous downloading can be carried out, and the downloading efficiency of the HTTPS webpage is improved.
Drawings
FIG. 1 is a flow chart of one embodiment of an asynchronous download method provided by the present invention;
FIG. 2 is a flow chart of another embodiment of an asynchronous download method provided by the present invention;
fig. 3 is a block diagram of an embodiment of an asynchronous download system provided by the present invention.
Detailed Description
In order to make the technical solutions of the present invention better understood, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The terms "first," "second," "third," "fourth," and the like in the description and in the claims, as well as in the drawings, are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It will be appreciated that the data so used may be interchanged under appropriate circumstances such that the embodiments described herein may be practiced otherwise than as specifically illustrated or described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
Referring to the drawings, an embodiment of an asynchronous downloading method of the present invention includes:
s101, receiving a request data packet of a logic control sending module.
A logic Control sending module (L CS, L g Control Sender), where the request packet may be sent according to a certain format, the request packet includes a url link UR L, the logic Control sending module sends the request packet to a server, the server may parse the request packet, the format of the request packet may be an ASN (Abstract Syntax Notation in chinese), and the ASN is a data format for representing, encoding, transmitting, and decoding data, and those skilled in the art should understand that no specific description is made.
S102, obtaining the uniform resource locator link UR L in the request data packet.
UR L (Chinese: Uniform Resource locator, English: Uniform Resource L ocessor), that is, the web address entered by the user in the browser during normal use, for example, searching "Baidu corporation", can enter "HTTPS:// www.baidu.com/" in the address bar, and this web address is called UR L, after receiving the request packet, it needs to analyze, and then it analyzes UR L in the request packet, that is, the user is to the target location of the downloaded Resource, and then it can obtain the IP (Chinese: interconnection Protocol between networks, English: Internet Protocol) address corresponding to UR L by domain name analysis, and can access by IP address.
S103, when the UR L is an HTTPS request, recording first information of the UR L used for information statistics and packaging information after downloading is completed and storing the first information to an asynchronous request queue, wherein the first information comprises the length of the UR L, the IP address obtained by resolving the domain name of the UR L and the ID (Chinese: identity) of a logic control sending module sending the UR L.
After UR L obtained by analyzing the request data packet, judging the link type of UR L, wherein the link type may include http link, HTTPS link, FTP (Chinese: File Transfer Protocol, English: File Transfer Protocol) link or FTP (multiple Transfer Protocol, equivalent to encrypted FTP) link, when UR L obtained by the analysis is judged to be an HTTPS link, recording the first information of UR L, the first information is used for packaging information after information statistics and downloading are completed, and storing the first information into an asynchronous request queue, the asynchronous request queue is in a list form composed of key-values, namely K-V storage, the key is the address of the UR L initialized data structure, the value is the data structure recorded by various information in the first information, and the first information includes UR L length, UR L IP address, logic identification for sending UR 35identity, and ID (identity control module: ID: sending UR L) of the first information.
And S104, initializing the sending request information of the UR L.
In S104, the initialization operation may adopt a conventional means of a crawler technology, a simulated browser is used to access a network, the initialized content may include whether UR L jump is supported, whether an agent is set, whether a variable for recording a download result is set, and whether a certificate of an HTTPS site accessed by UR L is verified, it is worth mentioning whether the certificate of the HTTPS site accessed by UR L is verified, an HTTPS page accessed in the browser is normally accessed only through certificate verification, and once an untrusted site is encountered, the browser gives an alarm, so that the accessed site is prevented from being an untrusted site, and security is improved.
S105, configuring a DNS cache and placing the IP and the port number pointed by the UR L into the DNS cache.
In the scheme, in order to realize asynchronous downloading, a DNS (Chinese Domain name system, English: Domain name system) cache structure is added, an IP address and a port number corresponding to the UR L are analyzed in advance and then are placed in a DNS cache, the IP and the port number can be analyzed in advance to skip calling of the function getaddrinfo, the function getaddrinfo is also analyzed in advance by utilizing Domain name analysis instead of calling the function getaddrinfo on site, so that color can be avoided, because the Domain name analysis process, namely calling of the function getaddrinfo, is required to be performed in the HTTPS request process, the function getaddrinfo is a congested and non-asynchronous function, the downloading efficiency is seriously influenced, a DNS cache is arranged, the corresponding IP and the port number are directly arranged in the cache of the downloading request, the phenomenon of congestion is avoided, and the real asynchronous downloading is realized.
S106, adding the sending request information and the first information into the asynchronous request queue to start downloading.
The sending request information and the first information are added to the asynchronous request queue to start downloading, and the construction of the corresponding asynchronous request queue is described in the foregoing, which is not described herein again.
And S107, when the asynchronous request queue is monitored to finish downloading, the downloading data is sent to the logic control sending module according to the first information.
During the process of asynchronous request downloading, the downloading condition of an asynchronous request queue needs to be monitored, the downloading condition can be immediately processed when the condition occurs, and the processing efficiency is improved, wherein the downloading condition can comprise two conditions, namely downloading completion and downloading error, the downloading error can comprise various error conditions, such as abnormal connection, acceptance rejection of UR L, connection overtime and the like, each downloading error corresponds to an error code, a person skilled in the art should understand that the method is not limited, the error codes can be returned to a logic control sending module for feedback after the downloading error information is monitored, when the downloading is completed, the address of a request party for downloading can be determined according to first information stored in the asynchronous request queue, namely a logic control sending module ID for sending UR L in the first information, the downloaded data is sent to the logic control sending module after being processed, the asynchronous downloading operation is completed, and the downloaded data can be packaged in an ASN format before being sent to the logic control sending module to obtain a return package in an ASN format, the returned package in the ASN format is sent to the logic control sending module, and other processing modes can be adopted.
According to the asynchronous downloading method provided by the invention, the asynchronous downloading of the HTTPS is realized by adding the DNS cache and pre-placing the IP and the port number corresponding to the UR L, so that the function of downloading the HTTPS webpage is met, the asynchronous downloading can be carried out, the efficiency is greatly improved, and the downloading efficiency is improved.
Referring to fig. 2, the asynchronous downloading method of the present invention further provides an embodiment, including;
s201, monitoring a network port.
By monitoring the network port, the ASN format data packet sent by the logic control sending module can be obtained.
S202, receiving the ASN abstract syntax mark format data packet of the logic control sending module.
The ASN format data packet includes UR L requesting downloading, that is, the location of the downloaded resource, and UR L can be obtained by parsing the ASN format data packet.
S203, judging whether the ASN format data packet is a legal request, if so, executing S204, and if not, executing S201.
In order to improve the security, whether the request in the ASN format data packet is legal or not is judged, only the subsequent operation is needed to be executed for the legal request, if not, the ASN format data packet is ignored, the network port is monitored continuously, and a new ASN format data packet is prepared to be received.
And S204, acquiring a uniform resource locator link UR L in the data packet in the ASN abstract syntax notation format.
After verifying that the request in the data packet in the ASN format is legal, parsing the data packet in the ASN format to obtain UR L corresponding to the download resource.
S205, when the UR L is an HTTPS request, recording first information of the UR L used for information statistics and packaging information after downloading is completed and storing the first information to an asynchronous request queue, wherein the first information comprises UR L length, UR L domain name resolution to obtain an IP address and a logic control sending module ID for sending UR L.
Step 205 is similar to step S103 above, and is not described herein.
S206, initializing the transmission request information of UR L.
The initialization operation may include setting a header of an HTTPS request, whether a UR L jump is supported, whether an agent is set, setting a variable for recording a download result, and whether a certificate of an HTTPS site accessed by the UR L is verified, which may be specifically referred to as S104 and is not described herein.
S207, configuring the DNS cache and placing the IP and port number pointed to by the UR L into the DNS cache.
Step S207 is similar to step S105 above, and is not repeated here to avoid content duplication
S208, adding the sending request information and the first information into the asynchronous request queue to start downloading.
And adding the sending request information and the first information into an asynchronous request queue to start downloading.
S209, monitoring the downloading condition of the asynchronous request queue, if the downloading is finished, executing S210, and if the downloading is in error, executing S213.
In the process of asynchronous request downloading, the downloading condition of an asynchronous request queue needs to be monitored, and the condition can be immediately processed, so that the processing efficiency is improved.
S210, when the downloading is finished, the downloading data is packaged into a return packet in an ASN format;
s211, sending the return packet in the ASN format to the logic control sending module according to the content of the first information.
S212, deleting the first information corresponding to the download data in the asynchronous request queue.
Because the asynchronous request queue needs to receive data continuously and send data continuously when asynchronous downloading is carried out, otherwise the asynchronous request queue can be increased infinitely to cause performance reduction, after the downloaded data package obtained after downloading is called an ASN format return packet and is sent to the logic control sending module, first information of the downloaded data in the asynchronous request queue needs to be deleted, and the space in the asynchronous request queue is ensured to be enough to process a new asynchronous downloading request.
And S213, when the download error occurs, returning an error code to the logic control sending module.
The reason of the downloading error can be known according to the error code, so that the user can conveniently process the operation of the downloading error in time, and it should be understood by those skilled in the art that the details are not limited.
With reference to fig. 3, the above describes an asynchronous downloading method, and correspondingly, an asynchronous downloading system is further provided, which is described in detail below:
an embodiment of an asynchronous download system, comprising:
a receiving unit 301, configured to receive a request packet of a logic control sending module;
an obtaining unit 302, configured to obtain a uniform resource locator link UR L in the request packet;
a storage unit 303, configured to record, when the UR L is an HTTPS request, first information of the UR L, which is used for information statistics and information packing after completion of downloading, and store the first information in an asynchronous request queue, where the first information includes a UR L length, a DNS address obtained through resolution, and a UR L identification bit;
an initialization unit 304, configured to initialize the sending request information of the UR L;
a configuration unit 305, configured to configure a DNS cache and place the IP and port number pointed to by the UR L into the DNS cache;
a downloading unit 306, configured to add the sending request information and the first information into the asynchronous request queue to start downloading;
a sending unit 307, configured to send the download data to the logic control sending module according to the first information when it is monitored that the asynchronous request queue completes downloading.
Further, the receiving unit 301 is further configured to monitor a network port and receive an ASN abstract syntax notation format data packet of the logic control sending module;
the obtaining unit 302 is further configured to obtain the uniform resource locator link UR L in the ASN abstract syntax notation format packet.
Further, the asynchronous download system further comprises:
the determining unit 308 is configured to determine whether the ASN format data packet is a legal request, and notify the acquiring unit if the ASN format data packet is a legal request.
Further, the initialization unit 304 is also used to set the header of the HTTPS request, whether UR L jump is supported, whether an agent is set, set a variable recording the download result, and whether to verify the certificate of the HTTPS site accessed by the UR L.
Further, the sending unit 307 is further configured to monitor a downloading condition of the asynchronous request queue, encapsulate the downloaded data into a return packet in an ASN format when the downloading is completed, and send the return packet in the ASN format to the logic control sending module according to the content of the first information.
Further, the asynchronous download system further comprises:
a deleting unit 309, configured to delete the first information corresponding to the download data in the asynchronous request queue.
According to the asynchronous downloading method and system provided by the invention, the asynchronous downloading of the HTTPS webpage is realized by setting the IP and the port number of the server corresponding to the UR L in the DNS cache, so that the HTTPS webpage downloading function is met, the asynchronous downloading can be carried out, and the downloading efficiency of the HTTPS webpage is improved.
It is clear to those skilled in the art that, for convenience and brevity of description, the specific working processes of the above-described systems, apparatuses and units may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.
In the several embodiments provided in the present application, it should be understood that the disclosed system, apparatus and method may be implemented in other manners. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the units is only one logical division, and other divisions may be realized in practice, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.
Those skilled in the art will appreciate that all or part of the steps in the methods of the above embodiments may be implemented by associated hardware instructed by a program, which may be stored in a computer-readable storage medium, and the storage medium may include: a Read Only Memory (ROM), a Random Access Memory (RAM), a magnetic or optical disk, or the like.
It will be understood by those skilled in the art that all or part of the steps in the method for implementing the above embodiments may be implemented by hardware that is instructed to implement by a program, and the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a magnetic disk or an optical disk, etc.
While the asynchronous downloading method and system provided by the present invention have been described in detail, those skilled in the art will appreciate that the embodiments of the present invention are not limited to the foregoing description.

Claims (13)

1. An asynchronous download method, comprising:
receiving a request data packet of a logic control sending module;
acquiring a uniform resource locator link UR L in the request data packet;
when the UR L is an HTTPS request, recording first information of the UR L for information statistics and packed information after completion of downloading, and storing the first information to an asynchronous request queue, where the first information includes the length of the UR L, an IP address obtained by resolution of a UR L domain name, and a logic control sending module ID for sending the UR L;
initializing the transmission request information of the UR L;
configuring a DNS cache and placing the IP and port number pointed to by the UR L into the DNS cache;
adding the sending request information and the first information into the asynchronous request queue to start downloading;
and when the asynchronous request queue is monitored to finish downloading, the downloading data is sent to the logic control sending module according to the first information.
2. The asynchronous downloading method of claim 1, wherein the receiving logic controls the request packet of the sending module, comprising:
monitoring a network port;
receiving an ASN abstract syntax mark format data packet of a logic control sending module;
the obtaining the uniform resource locator link UR L in the request packet includes:
get the uniform resource locator link UR L in the ASN abstract syntax notation format packet.
3. The asynchronous downloading method as claimed in claim 2, wherein said obtaining the url link UR L in the ASN abstract syntax notation format packet further comprises:
and judging whether the ASN format data packet is a legal request or not, and if so, executing subsequent steps.
4. The asynchronous downloading method of claim 1, wherein the initializing the request information for sending UR L includes:
setting a header of an HTTPS request;
whether UR L hopping is supported;
whether to set an agent;
setting a variable for recording a downloading result; and
whether to verify the certificate of the HTTPS site accessed by the UR L.
5. The asynchronous downloading method according to claim 1, wherein the sending the download data to the logic control sending module according to the first information when it is monitored that the asynchronous request queue completes downloading comprises:
monitoring the downloading condition of the asynchronous request queue;
when downloading is completed, packaging the downloaded data into a return packet in an ASN format;
and sending the return packet in the ASN format to the logic control sending module according to the content of the first information.
6. The asynchronous downloading method according to claim 1, wherein after the step of sending the download data to the logic control sending module according to the first information when it is monitored that the asynchronous request queue completes downloading, the method further comprises:
and deleting the first information corresponding to the downloading data in the asynchronous request queue.
7. An asynchronous download system, comprising:
the receiving unit is used for receiving a request data packet of the logic control sending module;
an obtaining unit, configured to obtain a uniform resource locator link UR L in the request packet;
a storage unit, configured to record, when the UR L is an HTTPS request, first information of the UR L, which is used for information statistics and is used for packing information after completion of downloading, and store the first information in an asynchronous request queue, where the first information includes a UR L length, an IP address obtained by resolution of a UR L domain name, and a logic control sending module ID for sending the UR L;
an initialization unit for initializing the transmission request information of the UR L;
a configuration unit configured to configure a DNS cache and place the IP and port number pointed to by the UR L into the DNS cache;
the downloading unit is used for adding the sending request information and the first information into the asynchronous request queue to start downloading;
and the sending unit is used for sending the download data to the logic control sending module according to the first information when the asynchronous request queue finishes downloading.
8. The asynchronous download system of claim 7 wherein the receiving unit is further configured to listen to a network port and receive the ASN abstract syntax notation format packet of the logic control sending module;
the obtaining unit is also used for obtaining the uniform resource locator link UR L in the ASN abstract syntax notation format packet.
9. The asynchronous download system of claim 8, further comprising:
and the judging unit is used for judging whether the ASN format data packet is a legal request or not, and informing the acquiring unit if the ASN format data packet is the legal request.
10. The asynchronous download system of claim 7, characterized in that the initialization unit is further configured to set the header of the HTTPS request, whether UR L jump is supported, whether the proxy is set, set the variable to record the download result, and whether the certificate of the HTTPS site accessed by the UR L is verified.
11. The asynchronous download system of claim 7, wherein the sending unit is further configured to monitor the asynchronous request queue for download, encapsulate the download data into the return packet in the ASN format when the download is completed, and send the return packet in the ASN format to the logic control sending module according to the content of the first information.
12. The asynchronous download system of claim 7, further comprising:
and the deleting unit is used for deleting the first information corresponding to the downloading data in the asynchronous request queue.
13. A computer-readable storage medium having stored therein instructions which, when run on a computer, cause the computer to perform the asynchronous download method of any of claims 1-6.
CN201510515897.6A 2015-08-20 2015-08-20 Asynchronous downloading method and system Active CN106470237B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201510515897.6A CN106470237B (en) 2015-08-20 2015-08-20 Asynchronous downloading method and system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201510515897.6A CN106470237B (en) 2015-08-20 2015-08-20 Asynchronous downloading method and system

Publications (2)

Publication Number Publication Date
CN106470237A CN106470237A (en) 2017-03-01
CN106470237B true CN106470237B (en) 2020-07-31

Family

ID=58229160

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201510515897.6A Active CN106470237B (en) 2015-08-20 2015-08-20 Asynchronous downloading method and system

Country Status (1)

Country Link
CN (1) CN106470237B (en)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107872548B (en) * 2017-11-10 2021-01-12 北京知道创宇信息技术股份有限公司 Webpage linking method and device and readable storage medium
CN108881485A (en) * 2018-07-30 2018-11-23 中国石油化工股份有限公司 The method for ensureing the high concurrent system response time under big data packet

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20080250152A1 (en) * 2007-04-03 2008-10-09 Ching-Hung Chou Communication method having firewall network tunnel and location transparency
WO2011038409A1 (en) * 2009-09-28 2011-03-31 Icontrol Networks, Inc. Integrated security system with parallel processing architecture
EP2360583A3 (en) * 2010-02-12 2011-09-28 Samsung Electronics Co., Ltd. Method and system for installing applications
CN104834519A (en) * 2015-04-04 2015-08-12 杭州中焯信息技术股份有限公司 Cross-platform application software development system

Also Published As

Publication number Publication date
CN106470237A (en) 2017-03-01

Similar Documents

Publication Publication Date Title
CN102137059B (en) Method and system for blocking malicious accesses
CN103607385B (en) Method and apparatus for security detection based on browser
CN107026821B (en) Message processing method and device
CN111400722B (en) Method, apparatus, computer device and storage medium for scanning small program
CN110519380B (en) Data access method and device, storage medium and electronic equipment
CN107124430B (en) Page hijacking monitoring method, device, system and storage medium
CN108768960B (en) Virus detection method, device, storage medium and computer equipment
CN107104924B (en) Verification method and device for website backdoor file
CN105635178B (en) Ensure the block type Network Access Method and device of safety
CN103401836A (en) Method and device used for judging whether webpage is hijacked by ISP (internet service provider) or not
CN104283903A (en) Downloading method for files and device thereof
CN109508437B (en) Search website auditing method, system, gateway equipment and storage medium
CN110557358A (en) Honeypot server communication method, SSLStrip man-in-the-middle attack perception method and related device
CN105159992A (en) Method and device for detecting page contents and network behaviors of application program
CN106470237B (en) Asynchronous downloading method and system
CN108259416B (en) Method for detecting malicious webpage and related equipment
CN113596014A (en) Access vulnerability detection method and device and electronic equipment
US10360379B2 (en) Method and apparatus for detecting exploits
CN111225038B (en) Server access method and device
CN111371775A (en) Single sign-on method, device, equipment, system and storage medium
CN107566916B (en) The access process control method and device of Internet video link
CN112202785B (en) Uploaded file processing method, device and equipment and computer storage medium
CN114915565A (en) Method and system for debugging network
CN114553529A (en) Data processing method, device, network equipment and storage medium
CN108985013A (en) Method, apparatus, client and the server-side that detection SWF file is used by third party application

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant