CN106254497B - A method of the black file in black file polling is counted - Google Patents
A method of the black file in black file polling is counted Download PDFInfo
- Publication number
- CN106254497B CN106254497B CN201610695068.5A CN201610695068A CN106254497B CN 106254497 B CN106254497 B CN 106254497B CN 201610695068 A CN201610695068 A CN 201610695068A CN 106254497 B CN106254497 B CN 106254497B
- Authority
- CN
- China
- Prior art keywords
- file
- code
- http response
- black
- target
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/02—Protocols based on web technology, e.g. hypertext transfer protocol [HTTP]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/56—Provisioning of proxy services
Abstract
A method of the black file in black file polling is counted, for being provided with the destination server side of Nginx Reverse Proxy, comprising: hypertext transfer protocol http response entity hook is set in Nginx Reverse Proxy；Using http response entity hook, corresponding http response entity is obtained from the http response message being transferred in destination server；Http response entity is parsed, determines MD5 code and the file type in http response entity；According to file type, MD5 code corresponding with black file is filtered out, and is target MD5 code by MD5 code labeling corresponding with black file；According to all target MD5 codes filtered out in preset time, the black file in black file query requests sent within a preset time to Cloud Server to client is counted.When being counted using this method to the black file in black file polling, statistic processes is simpler efficiently.
Description
Technical field
The invention belongs to network communication technology fields more particularly to a kind of black file in black file polling to count
Method.
Background technique
When client needs inquire black file from Cloud Server, client sends HTTP request, cloud to Cloud Server
After server receives the HTTP request that client is sent to it, if it is confirmed that the request is correct, it can be directed to client transmission
The http response of the HTTP request, if it is desired to be looked by the black file that third-party server sends client to Cloud Server
If black file in inquiry request is counted, needs that Nginx Reverse Proxy is arranged in third-party server, pass through
Nginx Reverse Proxy intercepts and captures http traffic, and the relevant information of black file is extracted from the http traffic of intercepting and capturing,
And then realization is to the statistics of black file.
In general, Nginx Reverse Proxy when intercepting and capturing http traffic, needs first location client end and Nginx anti-
To the TCP connection between proxy server, then response data is recombinated and intercepts and captures, entire treatment process is complex cumbersome, efficiency
It is lower.
So in the method that the existing black file in black file polling is counted, due to intercepting and capturing http traffic
Process it is complex cumbersome, cause entire statistic processes efficiency lower.
Summary of the invention
In view of this, it is an object of the invention to propose a kind of side that the black file in black file polling is counted
Method, to solve in the method that the existing black file in black file polling is counted, due to intercepting and capturing the mistake of http traffic
Journey is complex cumbersome, the problem for causing statistic processes efficiency lower.In order to which some aspects of the embodiment to disclosure have one
Simple summary is shown below in basic understanding.The summarized section is not extensive overview, nor to determine key/critical
Component or the protection scope for describing these embodiments.Its sole purpose is that some concepts are presented with simple form, with this
Preamble as following detailed description.
In some alternative embodiments, this method is used to be provided with the destination server of Nginx Reverse Proxy
Side, comprising: hypertext transfer protocol http response entity hook is set in the Nginx Reverse Proxy；Using institute
Http response entity hook is stated, obtains corresponding http response from the http response message being transferred in the destination server
Entity, the http response message are the message that the HTTP request that Cloud Server is sent for client makes a response；To described
Http response entity is parsed, and determines MD5 code and file type in the http response entity；According to the files classes
Type filters out MD5 code corresponding with black file, and is target MD5 code by MD5 code labeling corresponding with black file；According to default
All target MD5 codes filtered out in time, the black text that client is sent in the preset time to Cloud Server
Black file in part inquiry request is counted.
Further, according to all target MD5 codes filtered out in preset time, to client in the preset time
The process that black file in the interior black file query requests sent to Cloud Server is counted, specifically includes: when statistics is default
The quantity of the interior all target MD5 codes filtered out, to count client in the preset time to cloud service
The total quantity for the black file in black file query requests that device is sent；All target MD5 codes are divided into multiple MD5 codes
Group, wherein the MD5 code in each MD5 code character is identical；The quantity for counting target MD5 code in each MD5 code character, to count
Out client in the preset time to Cloud Server send black file query requests in the MD5 code in the MD5 code character
The number that corresponding black file is queried.
Further, this method further include: during obtaining corresponding http response entity from http response message,
Record the protocol IP or uniform resource position mark URL interconnected between the corresponding network of the http response entity；To described
Http response entity is parsed, during determining MD5 code and the file type in the http response entity, described in record
The corresponding IP or URL of MD5 code；During screening target MD5 code, the corresponding IP of the target MD5 code is recorded
Or URL.
Further, according to all target MD5 codes filtered out in preset time, to client in the preset time
The process that black file in the interior black file query requests sent to Cloud Server is counted, further includes: count each IP
Or the quantity of the corresponding target MD5 code of URL, thus count the corresponding client of the IP or URL in the preset time to
The quantity for the black file in black file query requests that Cloud Server is sent.
Further, this method further include: the Nginx Reverse Proxy is monitored in registration in the destination server
The locality connection event of the http response entity is sent by locality connection and the HTTP is received from the locality connection
Respond the reading event of entity；After obtaining corresponding http response entity in http response message, to the http response
Before entity is parsed, judge whether to listen to locality connection event；If listening to locality connection event, judge again
Whether reading event is listened to；If listening to reading event, the http response entity is received from the locality connection.
Further, this method further include: shared circle queue and the mapping of interim memory are set in the destination server
File.
Further, MD5 code corresponding with black file is being filtered out, and is being target by MD5 code labeling corresponding with black file
After MD5 code, before being counted to black file, this method further include: the target MD5 code is added to the shared annular team
The tail portion of column；From the head of the shared circle queue extraction target MD5 code to the interim Memory Mapping File, and will
It is stored in the storage disk that the interim Memory Mapping File is synchronized in the destination server.
Further, this method further include: synchronous documents timer is set in the destination server and is created in interim
Deposit mapped file timer, wherein the time for creating interim Memory Mapping File timer is fixed greater than the synchronous documents
When device time；The target MD5 code is being extracted to the interim Memory Mapping File from the head of the shared circle queue
At the beginning of, start the synchronous documents timer and described creates interim Memory Mapping File timer；The interim memory is reflected
The process that file synchronization is stored in the storage disk into the destination server is penetrated, is specifically included: if the synchronization
The time of file timer expires, then current time corresponding presently described interim Memory Mapping File is synchronized to the target
It is stored in storage disk in server；Or, if the time for creating interim Memory Mapping File timer expire,
Then by the moment corresponding presently described interim Memory Mapping File be synchronized in the storage disk in the destination server into
Row storage, and create an interim Memory Mapping File, from the head of the shared circle queue extract the target MD5 code to
In newly-built interim Memory Mapping File.
Further, this method further include: registration monitors and exits signal event in the destination server；If listened to
Signal event is exited, then exits and monitors ontology connection event and reading event, stops for the target MD5 code being added to described shared
The tail portion of circle queue and to extract the target MD5 code from the head of the shared circle queue literary to the interim memory mapping
Part, and close the synchronous documents timer and described create interim Memory Mapping File timer.
Compared with prior art, the invention has the benefit that
The present invention provides a kind of method and device that the black file in black file polling is counted, and in this method, leads to
The setting http response entity hook in the Nginx Reverse Proxy of destination server is crossed, can be taken from target is transferred to
It is obtained in the http response message for the response that the HTTP request that Cloud Server in business device is sent for client is made corresponding
Http response entity can determine the corresponding MD5 code of http response entity and file type by analysis http response entity,
Pass through MD5 code and file type, so that it may to black file of the client into the black file query requests that Cloud Server is sent into
Row statistics, entire statistic processes are no longer needed to without the TCP connection between first location client end and Nginx Reverse Proxy
Response data is recombinated and intercepted and captured, the complexity for the treatment of process is greatly reduced, so that the simpler height of entire statistic processes
Effect.
For the above and related purposes, one or more embodiments include being particularly described below and in claim
In the feature that particularly points out.Certain illustrative aspects are described in detail in the following description and the annexed drawings, and its instruction is only
Some modes in the utilizable various modes of the principle of each embodiment.Other benefits and novel features will be under
The detailed description in face is considered in conjunction with the accompanying and becomes obvious, the disclosed embodiments be all such aspects to be included and they
Be equal.
Detailed description of the invention
Fig. 1 is the process for the method that a kind of black file in black file polling provided in an embodiment of the present invention is counted
Schematic diagram；
Fig. 2 is the stream for the method that another kind provided in an embodiment of the present invention counts the black file in black file polling
Journey schematic diagram；
Fig. 3 is the stream of another method counted to the black file in black file polling provided in an embodiment of the present invention
Journey schematic diagram.
Specific embodiment
The following description and drawings fully show specific embodiments of the present invention, to enable those skilled in the art to
Practice them.Embodiment only represents possible variation.Unless explicitly requested, otherwise individual components and functionality is optional, and
And the sequence of operation can change.The part of some embodiments and feature can be included in or replace other embodiments
Part and feature.The range of embodiment of the present invention includes the entire scope of claims and the institute of claims
There is obtainable equivalent.Herein, these embodiments of the invention individually or generally can be used term " invention "
It indicates, this is not meant to automatically limit this just for the sake of convenient, and if in fact disclose the invention more than one
The range of application is any single invention or inventive concept.
It is illustrated presently in connection with attached drawing, shown in fig. 1 is in some optional embodiments to black in black file polling
The flow chart for the method that file is counted；Fig. 2 shows be in some optional embodiments to the black text in black file polling
The flow chart for the method that part is counted；Fig. 3 is shown in some optional embodiments to the black file in black file polling
The flow chart of the method counted.
As shown in Figure 1, in some alternative embodiments, discloses a kind of black file in black file polling and unite
The method of meter, this method are used to be provided with the destination server side of Nginx Reverse Proxy, comprising:
S11, hypertext transfer protocol http response entity hook is set in the Nginx Reverse Proxy；
When client needs to inquire black file from Cloud Server, the HTTP about black file polling can be sent to Cloud Server
Request message first determines whether this after Cloud Server receives the HTTP request message about black file polling of client transmission
Whether HTTP request message is correct, and after determining that the HTTP request message is correct, Cloud Server asks the HTTP to client transmission
The http response message for asking message to be responded.
Third-party server and Cloud Server are connected to the network, herein, third-party server is defined as target
Server, after Nginx Reverse Proxy is arranged in destination server, which can be intercepted and captured
Client sends HTTP request message and Cloud Server to Cloud Server and rings to client transmission to the HTTP request message
The data flow for the http response message answered, and then can be extracted from the data flow of intercepting and capturing in corresponding http response message
Http response entity.
In order to it is simpler efficiently extract that server sends to client the HTTP of client transmission is asked
The http response entity in http response message for asking message to be responded, in method provided in this embodiment, in destination service
Hypertext transfer protocol http response entity hook is set in the Nginx Reverse Proxy in device, utilizes the http response
Entity hook is no longer needed to recombinate and be intercepted and captured without the TCP connection between first location client end and Nginx Reverse Proxy
Response data, can easily obtain corresponding http response entity from http response message, and process is simpler efficiently.
S12, using the http response entity hook, from the http response message being transferred in the destination server
Corresponding http response entity is obtained, the http response message is that the HTTP request that Cloud Server is sent for client is made
The message of response；
Wherein, the http response message being transferred in destination server includes multiple, multiple http response message possibility
It is that HTTP request message acquisition is repeatedly sent from a client to Cloud Server, it is also possible to from multiple client to cloud service
The one or many transmission HTTP request messages of device obtain.
The http response message being transferred in destination server include Cloud Server for client send about the text of an annotated book
The message that makes a response of HTTP request of part inquiry also includes Cloud Server for client send about black file polling
The message that HTTP request makes a response.
Due to including what dialogue file query requests made a response in the http response message that is transferred in destination server
Message and the message made a response to black file polling, therefore, destination server is using the http response entity being disposed therein
Hook, from being transferred in the corresponding http response entity got in http response message therein, part http response is real
Include the information of text of an annotated book part in body, includes the information of black file in the http response entity of part.
S13, the http response entity is parsed, determines the MD5 code and files classes in the http response entity
Type；
If it is intended to passing through black text of the destination server to client into the black file query requests that Cloud Server is sent
Part is counted, and is needed to determine first in the http response entity got, comprising black file in which http response entity
Information, and the http response entity comprising black the file information is filtered out from all http response entities, then it is first right to need
Http response entity is parsed, and after parsing to http response entity, is specified in each http response entity
File type and MD5 code are counted with will pass through this document type and MD5 code to the black file in black file polling.
S14, according to the file type, filter out MD5 code corresponding with black file, and will MD5 corresponding with black file
Code labeling is target MD5 code；
S15, according to all target MD5 codes filtered out in preset time, to client in the preset time
The black file in black file query requests sent to Cloud Server is counted.
The manager of destination server wants in which section time, and the black file polling that client is sent to Cloud Server is asked
Black file in asking is counted, so that it may set the corresponding period for preset time, also, preset time can be set
For multiple periods.
Wherein, according to all target MD5 codes filtered out in preset time, to client in the preset time
The process that the black file in black file query requests sent to Cloud Server is counted, specifically includes:
The quantity of all target MD5 codes filtered out in statistics preset time, to count client described
The total quantity of the black file in black file query requests sent in preset time to Cloud Server；
All target MD5 codes are divided into multiple MD5 code characters, wherein the MD5 code phase in each MD5 code character
Together；
The quantity of target MD5 code in each MD5 code character is counted, to count client in the preset time to cloud
The number that black file corresponding with the MD5 code in the MD5 code character is queried in the black file query requests that server is sent.
In this method, by the way that http response entity hook is arranged in the Nginx Reverse Proxy of destination server,
The HTTP for the response that can be made from the HTTP request that the Cloud Server being transferred in destination server is sent for client is rung
It answers and obtains corresponding http response entity in message, by analyzing http response entity, can determine that http response entity is corresponding
MD5 code and file type, pass through MD5 code and file type, so that it may which the black file sent to client to Cloud Server is looked into
The black file ask in request is counted, entire statistic processes, without first location client end and Nginx Reverse Proxy it
Between TCP connection, no longer need to recombinate and intercept and capture response data, be greatly reduced the complexity for the treatment of process, so that entirely
Statistic processes is simpler efficiently.
As shown in Fig. 2, in some alternative embodiments, disclosing another black file in black file polling and carrying out
The method of statistics, this method are used to be provided with the destination server side of Nginx Reverse Proxy, comprising:
S21, hypertext transfer protocol http response entity hook is set in the Nginx Reverse Proxy,
Registration monitors locality connection event and reads event in the destination server, and shared circle queue and interim memory mapping text is arranged
Part；
Wherein, locality connection event sends the HTTP by locality connection for the Nginx Reverse Proxy and rings
The event of entity is answered, reading event is that the event of the http response entity is received from the locality connection.
When client needs to inquire black file from Cloud Server, the HTTP about black file polling can be sent to Cloud Server
Request message first determines whether this after Cloud Server receives the HTTP request message about black file polling of client transmission
Whether HTTP request message is correct, and after determining that the HTTP request message is correct, Cloud Server asks the HTTP to client transmission
The http response message for asking message to be responded.
Destination server and Cloud Server are connected to the network, Nginx reverse proxy is set in destination server and is taken
It is engaged in after device, which can intercept and capture client and send HTTP request message and cloud service to Cloud Server
Device sends the data flow of the http response message responded to the HTTP request message to client, and then can be from intercepting and capturing
The http response entity in corresponding http response message is extracted in data flow.
In order to it is simpler efficiently extract that server sends to client the HTTP of client transmission is asked
The http response entity in http response message for asking message to be responded, in method provided in this embodiment, in destination service
Hypertext transfer protocol http response entity hook is set in the Nginx Reverse Proxy in device, utilizes the http response
Entity hook is no longer needed to recombinate and be intercepted and captured without the TCP connection between first location client end and Nginx Reverse Proxy
Response data, can easily obtain corresponding http response entity from http response message, and process is simpler efficiently.
S22, using the http response entity hook, from the http response message being transferred in the destination server
Corresponding http response entity is obtained, the http response message is that the HTTP request that Cloud Server is sent for client is made
The message of response；
Wherein, the http response message being transferred in destination server includes multiple, multiple http response message possibility
It is that HTTP request message acquisition is repeatedly sent from a client to Cloud Server, it is also possible to from multiple client to cloud service
The one or many transmission HTTP request messages of device obtain.
The http response message being transferred in destination server include Cloud Server for client send about the text of an annotated book
The message that makes a response of HTTP request of part inquiry also includes Cloud Server for client send about black file polling
The message that HTTP request makes a response.
Due to including what dialogue file query requests made a response in the http response message that is transferred in destination server
Message and the message made a response to black file polling, therefore, destination server is using the http response entity being disposed therein
Hook, from being transferred in the corresponding http response entity got in http response message therein, part http response is real
Include the information of text of an annotated book part in body, includes the information of black file in the http response entity of part.
S23, judge whether to listen to locality connection event；If listening to locality connection event, judge whether again
Listen to reading event；If listening to reading event, the http response entity is received from the locality connection；
In order to guarantee that the Nginx Reverse Proxy in destination server is capable of the operation of stability and high efficiency, this is not influenced
The operational efficiency of Nginx Reverse Proxy can monitor locality connection event by registration and reading event is monitored in registration
The http response entity obtained in Nginx Reverse Proxy is sent in other modules in destination server by mode
Subsequent analysis processing and statistical disposition are carried out, i.e., a module is separately provided in destination server, with the module to HTTP
It responds entity and carries out subsequent processing.
S24, the http response entity is parsed, determines the MD5 code and files classes in the http response entity
Type；
If it is intended to passing through black text of the destination server to client into the black file query requests that Cloud Server is sent
Part is counted, and is needed to determine first in the http response entity got, comprising black file in which http response entity
Information, and the http response entity comprising black the file information is filtered out from all http response entities, then it is first right to need
Http response entity is parsed, and after parsing to http response entity, is specified in each http response entity
File type and MD5 code are counted with will pass through this document type and MD5 code to the black file in black file polling.
S25, according to the file type, filter out MD5 code corresponding with black file, and will MD5 corresponding with black file
Code labeling is target MD5 code；
S26, the tail portion that the target MD5 code is added to the shared circle queue；
S27, the target MD5 code is extracted from the head of the shared circle queue to the interim Memory Mapping File,
And it will be stored in storage disk that the interim Memory Mapping File is synchronized in the destination server；
During extracting target MD5 code to interim Memory Mapping File from the head of the shared circle queue, record
Extract the time of each target MD5 code, so as to it is subsequent target MD5 code is stored in the storage disk into server when, mark
Remember the extraction time of each target MD5 code.
S28, according to all target MD5 codes filtered out in preset time, to client in the preset time
The black file in black file query requests sent to Cloud Server is counted.
The manager of destination server wants in which section time, and the black file polling that client is sent to Cloud Server is asked
Black file in asking is counted, so that it may set the corresponding period for preset time, also, preset time can be set
For multiple periods.According to the extraction time of each target MD5 code, it is capable of determining that all mesh filtered out in preset time
Mark MD5 code.
Wherein, according to all target MD5 codes filtered out in preset time, to client in the preset time
The process that the black file in black file query requests sent to Cloud Server is counted, specifically includes:
The quantity of all target MD5 codes filtered out in statistics preset time, to count client described
The total quantity of the black file in black file query requests sent in preset time to Cloud Server；
All target MD5 codes are divided into multiple MD5 code characters, wherein the MD5 code phase in each MD5 code character
Together；
The quantity of target MD5 code in each MD5 code character is counted, to count client in the preset time to cloud
The number that black file corresponding with the MD5 code in the MD5 code character is queried in the black file query requests that server is sent.
Using method provided in this embodiment, locality connection event can be monitored by registration and reading event is monitored in registration
The http response entity obtained in Nginx Reverse Proxy is sent in other modules in destination server by mode
Subsequent analysis processing and statistical disposition are carried out, i.e., a module is separately provided in destination server, with the module to HTTP
It responds entity and carries out subsequent processing, to ensure that the Nginx Reverse Proxy in destination server can stablize height
The operation of effect does not influence the operational efficiency of the Nginx Reverse Proxy.
As shown in figure 3, in some alternative embodiments, disclosing another and being carried out to the black file in black file polling
The method of statistics, this method are used to be provided with the destination server side of Nginx Reverse Proxy, comprising:
S31, hypertext transfer protocol http response entity hook is set in the Nginx Reverse Proxy,
Registration monitors locality connection event and reads event in the destination server, and shared circle queue and interim memory mapping text is arranged
Part is arranged synchronous documents timer and creates interim Memory Mapping File timer；
Wherein, locality connection event sends the HTTP by locality connection for the Nginx Reverse Proxy and rings
The event of entity is answered, reading event is that the event of the http response entity is received from the locality connection, creates interim memory
The time of mapped file timer is greater than the time of synchronous documents timer.
When client needs to inquire black file from Cloud Server, the HTTP about black file polling can be sent to Cloud Server
Request message first determines whether this after Cloud Server receives the HTTP request message about black file polling of client transmission
Whether HTTP request message is correct, and after determining that the HTTP request message is correct, Cloud Server asks the HTTP to client transmission
The http response message for asking message to be responded.
Destination server and Cloud Server are connected to the network, Nginx reverse proxy is set in destination server and is taken
It is engaged in after device, which can intercept and capture client and send HTTP request message and cloud service to Cloud Server
Device sends the data flow of the http response message responded to the HTTP request message to client, and then can be from intercepting and capturing
The http response entity in corresponding http response message is extracted in data flow.
In order to it is simpler efficiently extract that server sends to client the HTTP of client transmission is asked
The http response entity in http response message for asking message to be responded, in method provided in this embodiment, in destination service
Hypertext transfer protocol http response entity hook is set in the Nginx Reverse Proxy in device, utilizes the http response
Entity hook is no longer needed to recombinate and be intercepted and captured without the TCP connection between first location client end and Nginx Reverse Proxy
Response data, can easily obtain corresponding http response entity from http response message, and process is simpler efficiently.
S32, using the http response entity hook, from the http response message being transferred in the destination server
Corresponding http response entity is obtained, the protocol IP interconnected between the corresponding network of the http response entity or unified money are recorded
Source finger URL URL；
Wherein, http response message is the message that Cloud Server makes a response for the HTTP request of client transmission, is passed
The defeated http response message into destination server include it is multiple, multiple http response message may be from a client to
Cloud Server repeatedly sends HTTP request message acquisition, it is also possible to from multiple client to the one or many transmissions of Cloud Server
HTTP request message obtains.
The http response message being transferred in destination server include Cloud Server for client send about the text of an annotated book
The message that makes a response of HTTP request of part inquiry also includes Cloud Server for client send about black file polling
The message that HTTP request makes a response.
Due to including what dialogue file query requests made a response in the http response message that is transferred in destination server
Message and the message made a response to black file polling, therefore, destination server is using the http response entity being disposed therein
Hook, from being transferred in the corresponding http response entity got in http response message therein, part http response is real
Include the information of text of an annotated book part in body, includes the information of black file in the http response entity of part.
S33, judge whether to listen to locality connection event；If listening to locality connection event, judge whether again
Listen to reading event；If listening to reading event, the http response entity is received from the locality connection；
In order to guarantee that the Nginx Reverse Proxy in destination server is capable of the operation of stability and high efficiency, this is not influenced
The operational efficiency of Nginx Reverse Proxy can monitor locality connection event by registration and reading event is monitored in registration
The http response entity obtained in Nginx Reverse Proxy is sent in other modules in destination server by mode
Subsequent analysis processing and statistical disposition are carried out, i.e., a module is separately provided in destination server, with the module to HTTP
It responds entity and carries out subsequent processing.
S34, the http response entity is parsed, determines the MD5 code and files classes in the http response entity
Type records the corresponding IP or URL of the MD5 code；
If it is intended to passing through black text of the destination server to client into the black file query requests that Cloud Server is sent
Part is counted, and is needed to determine first in the http response entity got, comprising black file in which http response entity
Information, and the http response entity comprising black the file information is filtered out from all http response entities, then it is first right to need
Http response entity is parsed, and after parsing to http response entity, is specified in each http response entity
File type and MD5 code are counted with will pass through this document type and MD5 code to the black file in black file polling.
S35, according to the file type, filter out MD5 code corresponding with black file, and will MD5 corresponding with black file
Code labeling is target MD5 code, records the corresponding IP or URL of the target MD5 code；
S36, the tail portion that the target MD5 code is added to the shared circle queue；
S37, the starting synchronous documents timer and described create interim Memory Mapping File timer；
S38, the target MD5 code is extracted to the interim Memory Mapping File from the head of the shared circle queue；
During extracting target MD5 code to interim Memory Mapping File from the head of the shared circle queue, record
Extract the time of each target MD5 code, so as to it is subsequent target MD5 code is stored in the storage disk into server when, mark
Remember the extraction time of each target MD5 code.
It is if the time of S39, the synchronous documents timer expire, current time is corresponding presently described interim interior
It deposits and is stored in the storage disk that mapped file is synchronized in the destination server；
It is if S40, the time for creating interim Memory Mapping File timer expire, the moment is corresponding current
It is stored in the storage disk that the interim Memory Mapping File is synchronized in the destination server, and creates one temporarily
Memory Mapping File extracts the target MD5 code to newly-built interim memory mapping text from the head of the shared circle queue
In part；
S41, according to all target MD5 codes filtered out in preset time, to client in the preset time
The black file in black file query requests sent to Cloud Server is counted.
The manager of destination server wants in which section time, and the black file polling that client is sent to Cloud Server is asked
Black file in asking is counted, so that it may set the corresponding period for preset time, also, preset time can be set
For multiple periods.According to the extraction time of each target MD5 code, it is capable of determining that all mesh filtered out in preset time
Mark MD5 code.
Wherein, according to all target MD5 codes filtered out in preset time, to client in the preset time
The process that the black file in black file query requests sent to Cloud Server is counted, specifically includes:
The quantity of all target MD5 codes filtered out in statistics preset time, to count client described
The total quantity of the black file in black file query requests sent in preset time to Cloud Server；
All target MD5 codes are divided into multiple MD5 code characters, wherein the MD5 code phase in each MD5 code character
Together；
The quantity of target MD5 code in each MD5 code character is counted, to count client in the preset time to cloud
The number that black file corresponding with the MD5 code in the MD5 code character is queried in the black file query requests that server is sent；
The quantity of the corresponding target MD5 code of each IP or URL is counted, to count the corresponding client of the IP or URL
Hold the quantity of the black file in the black file query requests sent in the preset time to Cloud Server.
Further, if the administrator of destination server wants using the set time as the period, such as with one day for a week
Phase, perhaps using one month for a cycle or using a season as a cycle etc., to client in each period to cloud
The black file in black file query requests that server is sent is counted, i.e., sends to client to Cloud Server within one day black
Black file in file query requests carries out counting primary or a month black file sent to client to Cloud Server is looked into
It askes the black file in request and count black file polling that is primary or sending to client to Cloud Server in a season and ask
Black file in asking carries out counting primary, as long as setting a cycle for the time for creating interim Memory Mapping File timer
Time, later according to the institute in interim Memory Mapping File corresponding with each period in the storage disk in destination server
There is MD5 code, the black file in black file query requests sent within the corresponding time in the period to Cloud Server to client
It is counted, no longer needs to count the MD5 code in preset time according to the extraction time of each MD5 code, count
Journey is simpler efficiently.
Further, this method further include: registration monitors and exits signal event in the destination server；If listened to
Signal event is exited, then exits and monitors ontology connection event and reading event, stops for the target MD5 code being added to described shared
The tail portion of circle queue and to extract the target MD5 code from the head of the shared circle queue literary to the interim memory mapping
Part, and close the synchronous documents timer and described create interim Memory Mapping File timer.
Registration is monitored and is exited after signal event in destination server, can be stopped at any time using in destination server
Other modules in addition to Nginx Reverse Proxy are analyzed and processed and subsequent Statistics Division http response entity
Reason, can guarantee in the case where the module is not normally functioning, and still can use the completion pair of Nginx Reverse Proxy
The analysis of http response entity and subsequent statistic processes will not influence the normal work of Nginx Reverse Proxy, protect
Demonstrate,prove the normal operation of statistical work.
It is right if the administrator of destination server wants using the set time as the period using method provided in this embodiment
Black file of the client into the black file query requests that Cloud Server is sent is counted in each period, as long as will create
The time of interim Memory Mapping File timer is set as the time of a cycle, later according to the storage magnetic in destination server
All MD5 codes in disk in interim Memory Mapping File corresponding with each period, to client in the corresponding time in the period
Black file in the interior black file query requests sent to Cloud Server is counted, and is no longer needed to according to each MD5 code
Extraction time counts the MD5 code in preset time, and statistic processes is simpler efficiently.
In short, the above description is only an embodiment of the present invention, it is not intended to limit the scope of the present invention, and is used for
The bright present invention.All within the spirits and principles of the present invention, any modification, equivalent replacement, improvement and so on should be included in
Within protection scope of the present invention.
Claims (8)
1. a kind of method that the black file in black file polling is counted, which is characterized in that reversed for being provided with Nginx
The destination server side of proxy server, comprising:
Hypertext transfer protocol http response entity hook is set in the Nginx Reverse Proxy；
Using the http response entity hook, obtained from the http response message being transferred in the destination server corresponding
Http response entity, the http response message is the disappearing of making a response of HTTP request that Cloud Server is sent for client
Breath；
The http response entity is parsed, determines the MD5 code and file type in the http response entity；
According to the file type, MD5 code corresponding with black file is filtered out, and be by MD5 code labeling corresponding with black file
Target MD5 code；
According to all target MD5 codes filtered out in preset time, to client to cloud service in the preset time
The black file in black file query requests that device is sent is counted；
According to all target MD5 codes filtered out in preset time, to client to cloud service in the preset time
The process that the black file in black file query requests that device is sent is counted, specifically includes:
The quantity of all target MD5 codes filtered out in statistics preset time, to count client described default
The total quantity of the black file in black file query requests sent in time to Cloud Server；
All target MD5 codes are divided into multiple MD5 code characters, wherein the MD5 code in each MD5 code character is identical；
The quantity of target MD5 code in each MD5 code character is counted, to count client in the preset time to cloud service
The number that black file corresponding with the MD5 code in the MD5 code character is queried in the black file query requests that device is sent.
2. the method according to claim 1, wherein this method further include:
During obtaining corresponding http response entity from http response message, it is corresponding to record the http response entity
Network between the protocol IP or uniform resource position mark URL that interconnect；
It is parsed to the http response entity, determines the mistake of the MD5 code and file type in the http response entity
Cheng Zhong records the corresponding IP or URL of the MD5 code；
During screening target MD5 code, the corresponding IP or URL of the target MD5 code is recorded.
3. according to the method described in claim 2, it is characterized in that, described according to all mesh filtered out in preset time
MD5 code is marked, the black file in black file query requests sent in the preset time to Cloud Server to client carries out
The process of statistics, further includes:
The quantity of the corresponding target MD5 code of each IP or URL is counted, is existed to count the corresponding client of the IP or URL
The quantity of the black file in black file query requests sent in the preset time to Cloud Server.
4. according to claim 1 to method described in 3 any one, which is characterized in that this method further include:
Registration monitors the Nginx Reverse Proxy and sends the HTTP by locality connection in the destination server
It responds the locality connection event of entity and receives the reading event of the http response entity from the locality connection；
After obtaining corresponding http response entity in http response message, the http response entity is carried out parsing it
Before, judge whether to listen to locality connection event；
If listening to locality connection event, judge whether to listen to reading event again；
If listening to reading event, the http response entity is received from the locality connection.
5. according to the method described in claim 4, it is characterized in that, this method further include: be arranged in the destination server
Shared circle queue and interim Memory Mapping File.
6. according to the method described in claim 5, it is characterized in that, filtering out MD5 code corresponding with black file, and will with it is black
The corresponding MD5 code labeling of file be target MD5 code after, before being counted to black file, this method further include:
The target MD5 code is added to the tail portion of the shared circle queue；
From the head of the shared circle queue extraction target MD5 code to the interim Memory Mapping File, and will be described
It is stored in the storage disk that interim Memory Mapping File is synchronized in the destination server.
7. according to the method described in claim 6, it is characterized in that, this method further include:
Synchronous documents timer is set in the destination server and creates interim Memory Mapping File timer, wherein institute
It states and creates time of the time of interim Memory Mapping File timer greater than the synchronous documents timer；
At the beginning of extracting the target MD5 code to the interim Memory Mapping File from the head of the shared circle queue, open
It moves the synchronous documents timer and described creates interim Memory Mapping File timer；
The process that will be stored in storage disk that the interim Memory Mapping File is synchronized in the destination server, tool
Body includes:
If the time of the synchronous documents timer expires, current time corresponding presently described interim memory is mapped into text
It is stored in the storage disk that part is synchronized in the destination server；Or,
It is if the time for creating interim Memory Mapping File timer expires, the moment is corresponding presently described interim
It is stored in the storage disk that Memory Mapping File is synchronized in the destination server, and a newly-built interim memory mapping
File extracts the target MD5 code into newly-built interim Memory Mapping File from the head of the shared circle queue.
8. the method according to the description of claim 7 is characterized in that this method further include: registered in the destination server
Signal event is exited in monitoring；Signal event is exited if listened to, exits and monitors ontology connection event and reading event, stopping will
The target MD5 code is added to the tail portion of the shared circle queue and extracts the mesh from the head of the shared circle queue
MD5 code is marked to the interim Memory Mapping File, and the closing synchronous documents timer and described create interim memory and reflects
Penetrate file timer.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610695068.5A CN106254497B (en) | 2016-08-19 | 2016-08-19 | A method of the black file in black file polling is counted |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610695068.5A CN106254497B (en) | 2016-08-19 | 2016-08-19 | A method of the black file in black file polling is counted |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN106254497A CN106254497A (en) | 2016-12-21 |
| CN106254497B true CN106254497B (en) | 2019-03-26 |
Family
ID=57593199
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610695068.5A Active CN106254497B (en) | 2016-08-19 | 2016-08-19 | A method of the black file in black file polling is counted |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106254497B (en) |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103841156A (en) * | 2012-11-26 | 2014-06-04 | 腾讯科技（深圳）有限公司 | File download protection method, device, and system based on an end-to-end protocol |
| CN104573518A (en) * | 2015-01-23 | 2015-04-29 | 百度在线网络技术（北京）有限公司 | Method, device, server and system for scanning files |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2011002818A1 (en) * | 2009-06-29 | 2011-01-06 | Cyberdefender Corporation | Systems and methods for operating an anti-malware network on a cloud computing platform |
-
2016
- 2016-08-19 CN CN201610695068.5A patent/CN106254497B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103841156A (en) * | 2012-11-26 | 2014-06-04 | 腾讯科技（深圳）有限公司 | File download protection method, device, and system based on an end-to-end protocol |
| CN104573518A (en) * | 2015-01-23 | 2015-04-29 | 百度在线网络技术（北京）有限公司 | Method, device, server and system for scanning files |
Also Published As
| Publication number | Publication date |
|---|---|
| CN106254497A (en) | 2016-12-21 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN105490854B (en) | Real-time logs collection method, system and application server cluster | |
| CN109525558A (en) | Leaking data detection method, system, device and storage medium | |
| US20180212989A1 (en) | System and method for monitoring, capturing and reporting network activity | |
| CN107528766B (en) | Information pushing method, device and system | |
| CN110750497A (en) | Data scheduling system | |
| CN105103496A (en) | System and method for extracting and preserving metadata for analyzing network communications | |
| WO2008000132A1 (en) | A system and method for collecting the entire network signaling information and a system for tracing the entire network signaling | |
| CN106412061A (en) | Linux-based log folder remote transmission system | |
| CN102075450A (en) | Utility method for recording chatting content of instant messaging device | |
| CN109714648A (en) | A kind of video flow load balancing method and device | |
| US20140229617A1 (en) | Server-side web analytics system and method | |
| CN101830240B (en) | Track traffic centralized alarming management system and method thereof | |
| CN110858192A (en) | Log query method and system, log checking system and query terminal | |
| CN103685354A (en) | Method and device for testing based on RMI protocol | |
| CN103886250A (en) | Data processing method, device, controller and system oriented to business support system | |
| CN103176997A (en) | Publishing and receiving system for multi-dimensional information | |
| KR101078375B1 (en) | System for tracing user activity using operating system and method thereof | |
| CN103136288A (en) | Multi-dimensional information system of mobile terminal | |
| CN106254497B (en) | A method of the black file in black file polling is counted | |
| CN110445671A (en) | A kind of network flow monitoring method based on SDN | |
| CN105471676B (en) | A kind of port scan IP network location liveness statistical system and method | |
| CN104283703A (en) | User login reminding method and system | |
| WO2012139462A1 (en) | Method, device, and system that enable signaling tracking | |
| CN106656658A (en) | Substation information pushing system | |
| CN101355727B (en) | System and method for automatically prompting user of full mobile phone EMS memory |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CP03 | Change of name, title or address |
Address after: 100041, room 2, building 3, building 30, Xing Xing street, Shijingshan District, Beijing, Patentee after: Beijing Falcon Safety Technology Co., Ltd Address before: 100041 Beijing city Shijingshan District Street Hing 30 Hospital No. 3 Building 2 layer A-0003 Patentee before: BEIJING KINGSOFT SECURITY MANAGEMENT SYSTEM TECHNOLOGY Co.,Ltd. |
|
| CP03 | Change of name, title or address |