CN105100015B - A kind of method and device for acquiring internet access data - Google Patents
A kind of method and device for acquiring internet access data Download PDFInfo
- Publication number
- CN105100015B CN105100015B CN201410208321.0A CN201410208321A CN105100015B CN 105100015 B CN105100015 B CN 105100015B CN 201410208321 A CN201410208321 A CN 201410208321A CN 105100015 B CN105100015 B CN 105100015B
- Authority
- CN
- China
- Prior art keywords
- daily record
- address
- network access
- application layer
- access identifier
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Landscapes
- Information Transfer Between Computers (AREA)
- Computer And Data Communications (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
An embodiment of the present invention provides a kind of method and devices for acquiring internet access data,Meet setting Web proxy server cluster on the node of preset condition in the IP bearer networks of telecom operators,The content requests message routing that network access Identifier is accessed URL by the forwarding Redirectional system in internet is forwarded to Web proxy server cluster,Web proxy server cluster generates application layer daily record by responding the content requests message,Data collecting system obtains the correspondence between application layer daily record and network access Identifier,And according to the correspondence,The network access Identifier is acquired in preset time period,To the access content of the internet,It can be seen that,The method and device of acquisition internet access data described in the embodiment of the present invention,Participation without DPI equipment,So as to reduce the cost of acquisition internet access content.
Description
Technical field
The present invention relates to the communications field more particularly to a kind of method and devices for acquiring internet access data.
Background technology
As the competition between Internet enterprises gradually aggravates, the content of internet that natural person user accesses is adopted
Collection is just all the more important.
The method of existing acquisition internet access data, it usually needs more in the access path of user to internet
Locate acquisition node setting depth data packet detection (Deep Packet Inspection, DPI) equipment, using DPI equipment to logical
The data packet of letter chain road is unpacked, feature recognition and detection.
And DPI equipment is expensive, it is seen then that the acquisition method of existing internet access data has of high cost lack
Point.
Invention content
An embodiment of the present invention provides a kind of methods for acquiring internet access data, it is therefore intended that solves existing interconnection
The problem of acquisition method of net access data is of high cost.
A kind of method for acquiring internet access content, applied to internet data acquisition system, the method includes：
The correspondence between application layer daily record and network access Identifier is obtained, the application layer daily record is visited by responding terminal
Ask the content requests message generation of uniform resource position mark URL, the content requests message passes through routing forwarding to the Web generations
Server cluster is managed, the Web proxy server cluster is arranged on to meet in the IP bearer networks of the telecom operators and preset
On the node of condition, the terminal uses the network access Identifier；
According to the correspondence, visit of the network access Identifier in preset time period, to the internet is acquired
Ask content.
Optionally, the correspondence obtained between application layer daily record and network access Identifier includes：
Bearing bed daily record is obtained from the aaa server in the internet, the bearing bed daily record is included with lower word
Section：Network access Identifier, this carrying layer conversation described in network access Identifier use IP address, this carrying layer conversation open
The timestamp of the timestamp of beginning and this bearing bed conversation end；
Application layer daily record is obtained from the Web proxy server cluster, the application layer daily record includes following field：This
Timestamp that secondary application-level request terminates, responds what this application-level request used at the IP address for initiating this application-level request
The URL of transport layer flow and this application-level request；
When the IP address for initiating this application-level request and network insertion mark described in this described carrying layer conversation
Know the timestamp that the IP address used matches and this described application-level request terminates and be located at this described bearing bed session start
Timestamp and this bearing bed conversation end timestamp between when, determine the application layer daily record and the network insertion mark
There are correspondences between knowledge.
Optionally, the correspondence obtained between application layer daily record and network access Identifier includes：
Bearing bed daily record is obtained from the aaa server in the internet, the bearing bed daily record is included with lower word
Section：Network access Identifier, this carrying layer conversation described in network access Identifier use original access IP address, this carrying
The timestamp and the timestamp of this bearing bed conversation end that layer conversation starts；
Address conversion daily record is obtained, described address conversion log includes following field：Original access IP address, the original
The corresponding port numbers of the access IP address that begins, the access IP address obtained by the original access IP address conversion, the access IP
The corresponding port numbers in address and address conversion time；
Application layer daily record is obtained from the Web proxy server cluster, the application layer daily record includes following field：This
Timestamp that secondary application-level request terminates, the IP address for initiating this application-level request, the corresponding access port of the IP address
Number, content response success timestamp, respond transport layer flow that this application-level request uses and this application-level request
URL；
When initiating in the IP address of this application-level request and described address conversion log in the application layer daily record
By original obtained the accesss IP address of access IP address conversion is identical, the IP address correspondence in the application layer daily record
Access end slogan it is identical with the corresponding port numbers of the access IP address in described address conversion log and, it is described should
In interval with address conversion time of the content response success timestamp in described address conversion log in layer daily record, then really
The IP address for initiating this application-level request in the fixed application layer daily record and the original access in described address conversion log
IP address corresponds to；
If described carry into the original access that network access Identifier uses described in this carrying layer conversation in daily record
IP address is identical with the original access IP address, obtains in the bearing bed daily record and carries network described in layer conversation with this
The corresponding network access Identifier of original access IP address that access mark uses；
Establish the correspondence of the network access Identifier and the application layer daily record.
Optionally, the correspondence obtained between application layer daily record and network access Identifier includes：
Daily record is applied according to what is obtained, determines the correspondence between the application layer daily record and network access Identifier, it is described
Application layer daily record includes the network access Identifier.
A kind of method for acquiring internet access content, applied to Web proxy server cluster, the web proxy service
Device cluster, which is arranged in the IP bearer networks of the telecom operators, to be met on the node of preset condition, the method includes：
The content requests message of terminal access URL is received, the content requests message passes through routing forwarding to the Web generations
Manage server cluster；
Application layer daily record is generated by responding the content requests message, the application layer daily record is used to obtain the application
Correspondence between layer daily record and network access Identifier, the correspondence are used to acquire the network access Identifier when default
Between in section, to the access content of the internet, the terminal uses the network access Identifier.
Optionally, the Web proxy server cluster has the function of Internet content cache.
Optionally, it further includes：
If carrying the network access Identifier in the content requests message, the network access Identifier is recorded in institute
It states in application layer daily record.
A kind of method for acquiring internet access content, including：
The network access Identifier is accessed the content requests message road of URL by the forwarding Redirectional system in the internet
By being forwarded to Web proxy server cluster, the Web proxy server cluster is arranged on the IP bearer networks of the telecom operators
Meet on the node of preset condition in network；
The content requests message that the Web proxy server cluster accesses URL by responding the network access Identifier is given birth to
Into application layer daily record；
Internet data acquisition system obtains the correspondence between the application layer daily record and network access Identifier, and foundation
The correspondence, acquire the network access Identifier in preset time period, to the access content of the internet.
Optionally, the preset condition includes：
Connect the communication between the Certificate Authority accounting system of internet data acquisition system and telecom operators；
When, there are during the conversion of IP address, being protected on address-translating device between terminal and the Web proxy server cluster
There is the daily record of address conversion；
And it is connected to equipment on the node and synchronizes in time.
A kind of internet data acquisition system, including：
Acquisition module, for obtaining the correspondence between application layer daily record and network access Identifier, the application layer daily record
It is generated by the content requests message for responding terminal access URL, the content requests message passes through routing forwarding to the Web generations
Server cluster is managed, the Web proxy server cluster is arranged on to meet in the IP bearer networks of the telecom operators and preset
On the node of condition, the terminal uses the network access Identifier；
Acquisition module, for according to the correspondence, acquire the network access Identifier in preset time period, to institute
State the access content of internet.
Optionally, the acquisition module includes：
First acquisition unit, for obtaining bearing bed daily record, the bearing bed from the aaa server in the internet
Daily record includes following field：Network access Identifier, this carrying layer conversation described in network access Identifier use IP address,
The timestamp of the timestamp of this bearing bed session start and this bearing bed conversation end；
Second acquisition unit, for obtaining application layer daily record, the application layer daily record from the Web proxy server cluster
Include following field：Timestamp that this application-level request terminates, the IP address for initiating this application-level request, response are this time
The transport layer flow and the URL of this application-level request that application-level request uses；
First determination unit, for working as the IP address for initiating this application-level request and this described carrying layer conversation
Described in the IP address matching that uses of network access Identifier and the timestamp that terminates of this described application-level request be located at described
When between the timestamp of secondary bearing bed session start and the timestamp of this bearing bed conversation end, the application layer daily record is determined
There are correspondences between the network access Identifier.
Optionally, the acquisition module includes：
First acquisition unit, for obtaining bearing bed daily record, the bearing bed from the aaa server in the internet
Daily record includes following field：What network access Identifier used described in network access Identifier, this carrying layer conversation original connects
Enter the timestamp of IP address, the timestamp of this bearing bed session start and this bearing bed conversation end；
Third acquiring unit, for obtaining address conversion daily record, described address conversion log includes following field：It is original
Access IP address, the original access for accessing the corresponding port numbers of IP address, being obtained by the original access IP address conversion
IP address, the corresponding port numbers of the access IP address and address conversion time；
Second acquisition unit, for obtaining application layer daily record, the application layer daily record from the Web proxy server cluster
Include following field：Timestamp that this application-level request terminates, the IP address for initiating this application-level request, the IP
The corresponding access end slogan in location, content response success timestamp respond transport layer flow and sheet that this application-level request uses
The URL of secondary application-level request；
Correspondence relationship establishing unit, for work as in the application layer daily record initiation this application-level request IP address with
In described address conversion log by the original obtained access IP address of access IP address conversion is identical, application layer day
The corresponding access end slogan of IP address end corresponding with the access IP address in described address conversion log in will
Slogan it is identical and, address of content response in the application layer daily record success timestamp in described address conversion log
In the interval of conversion time, it is determined that the IP address and described address of initiating this application-level request in the application layer daily record
Original access IP address in conversion log corresponds to；If described carry into network described in this carrying layer conversation in daily record
The access original access IP address that uses of mark is identical with the original access IP address, obtain in the bearing bed daily record with
The corresponding network access Identifier of original access IP address that network access Identifier described in secondary carrying layer conversation uses, and establish institute
State the correspondence of network access Identifier and the application layer daily record.
Optionally, the acquisition module includes：
Second determination unit for applying daily record according to what is obtained, determines the application layer daily record and network access Identifier
Between correspondence, the application layer daily record includes the network access Identifier.
A kind of Web proxy server cluster, the IP that the Web proxy server cluster is arranged on the telecom operators are held
Meet on the node of preset condition in contained network network, including：
Receiving module, for receiving the content requests message of terminal access URL, the content requests message is turned by routing
It is sent to the Web proxy server cluster；
Generation module responds the content requests message generation application layer daily record for passing through, and the application layer daily record is used
In obtaining the correspondence between the application layer daily record and network access Identifier, the correspondence connects for acquiring the network
Inlet identity in preset time period, to the access content of the internet, the terminal uses the network access Identifier.
Optionally, the Web proxy server cluster has the function of Internet content cache.
Optionally further include：
If for carrying the network access Identifier in the content requests message, the network is connect for logging modle
Inlet identity is recorded in the application layer daily record.
A kind of device for acquiring internet access content, including：
Redirectional system is forwarded, the content requests message routing for the network access Identifier to be accessed to URL is forwarded to
Web proxy server cluster, the Web proxy server cluster, which is arranged in the IP bearer networks of the telecom operators, to be met
On the node of preset condition；
The Web proxy server cluster is used for, and the content requests report of URL is accessed by responding the network access Identifier
Text generation application layer daily record；
Internet data acquisition system, for obtaining the correspondence between the application layer daily record and network access Identifier,
And according to the correspondence, acquire the network access Identifier in preset time period, to the access content of the internet.
The method and device of acquisition internet access data described in the embodiment of the present invention is carried in the IP of telecom operators
Meet on the node of preset condition setting Web proxy server cluster in network, the forwarding Redirectional system in internet is by net
The content requests message routing of network access identification access URL is forwarded to Web proxy server cluster, therefore, Web proxy server
Cluster can generate application layer daily record by responding the content requests message, data collecting system obtain application layer daily record and
Correspondence between network access Identifier, and according to the correspondence, the network access Identifier is acquired in preset time period
Access content interior, to the internet, it can be seen that, the method for the acquisition internet access data described in the embodiment of the present invention
And device, by the cooperation between Web proxy server cluster, forwarding Redirectional system and data collecting system, obtain network
Access mark without the participation of DPI equipment, acquires therefore, it is possible to reduce in internet access the access content of internet
The cost of appearance.
Description of the drawings
In order to illustrate more clearly about the embodiment of the present invention or technical scheme of the prior art, to embodiment or will show below
There is attached drawing needed in technology description to be briefly described, it should be apparent that, the accompanying drawings in the following description is only this
Some embodiments of invention, for those of ordinary skill in the art, without creative efforts, can be with
Other attached drawings are obtained according to these attached drawings.
Fig. 1 is a kind of flow chart of method for acquiring internet access content disclosed by the embodiments of the present invention；
Fig. 2 is a kind of topological structure schematic diagram of application scenarios of present invention method；
Fig. 3 is a kind of structure diagram of device for acquiring internet access content disclosed by the embodiments of the present invention；
Fig. 4 is a kind of structure diagram of internet data acquisition system disclosed by the embodiments of the present invention；
Fig. 5 is a kind of structure diagram of Web proxy server cluster disclosed by the embodiments of the present invention.
Specific embodiment
The embodiment of the present invention is applied to internet data harvester, is included at least in the internet data harvester
Following system：Forwarding Redirectional system, AAA system and Web proxy server in internet data acquisition system, internet
Cluster, Web proxy server cluster, which is arranged in internet, to be met on the node of preset condition.
Wherein, where the node that the web proxy cluster is disposed can be mobile grouping field data gateway
Backbone in node (for acquiring the access content of mobile internet access user) or province's backbone network, local Metropolitan Area Network (MAN)
Layer, convergence-level, access layer nodes at different levels.Most as low as the node with BRAS equipment same layer.(for acquiring fixed linking Internet
The access content of user), optionally, preset condition includes：The node can connect internet data acquisition system and electricity
Believe operator Certificate Authority book keeping operation (Authentication Authorization Accounting, AAA) system (including
Mobile AAA system, fixed network AAA system and fixed estropia AAA system) between communication, when terminal and Web proxy server collection
Group between there are IP address conversion when, in network address translation (Network Address Translation, NAT) equipment
Preserve address conversion daily record and, the Web proxy server cluster on the node, network address translation device
(optional) and the time synchronization of AAA system.In general, AAA system is arranged in the Operation Support System of internet.
Wherein, the node requirements that the internet data acquisition system is disposed meet the following conditions：Interconnection can be connected
The Certificate Authority of network data acquisition system and telecom operators book keeping operation (Authentication Authorization
Accounting, AAA) it is logical between system (including mobile AAA system, fixed network AAA system and fixed estropia AAA system)
Letter and, the communication of internet data acquisition system and web proxy cluster can be connected.
, will be by above triangular cooperation in the embodiment of the present invention, acquisition terminal (network access Identifier) is to interconnection
The access content of net.
Below in conjunction with the attached drawing in the embodiment of the present invention, the technical solution in the embodiment of the present invention is carried out clear, complete
Site preparation describes, it is clear that described embodiment is only part of the embodiment of the present invention, instead of all the embodiments.It is based on
Embodiment in the present invention, those of ordinary skill in the art are obtained every other without making creative work
Embodiment shall fall within the protection scope of the present invention.
A kind of method for acquiring internet access content disclosed by the embodiments of the present invention, as shown in Figure 1, including：
S101：Terminal accesses internet using network access Identifier；
If in general, terminal access to mobile network, network access Identifier is Mobile Directory Number (Mobile
Directory Number, MDN), if terminal access is fixed network, network access Identifier is broadband account, because
For the label that network access Identifier is internet identification terminal, therefore, in the embodiment of the present invention, to simplify the description, by terminal
Behavior regard the behavior of network access Identifier as.
S102：Aaa server distributes IP address for the network identity, while generates the day of a bearing bed session start
Will；
Wherein, IP address is effective for this access of this network identity.
Bearing bed session start daily record include the network access Identifier, this carrying layer conversation described in network insertion
Identify the IP address used and the timestamp of this bearing bed session start.
S103：Forwarding Redirectional system performs redirection；
S104：Application (network access Identifier) in terminal is initiated using the IP address for its distribution for internet content
The content requests message of URL under provider/Internet Service Provider (ICP/ISP), in the weight of forwarding Redirectional system
Under orientation, the content requests message is forwarded to Web proxy server cluster；
Wherein, intelligent domain name system (Domain Name System, DNS) may be used in routing forwarding technology, webpage delays
Deposit communication protocol (Web Cache Communication Protocol, WCCP) or Virtual Private Network (Virtual
Private Network, VPN) one kind in tunneling technique or arbitrary combination.
By taking intelligent DNS domain name mapping technology as an example：The bearing bed access distribution that aaa server is initiated to network identity is special
Local dns address, all domain name requests of user are parsed to the IP address of specific web proxy by it；
Or directly by general Local DNS by the request analysis of the URL to the IP address of specific web proxy, also may be used
The CNAME of its authorized DNS server is directed toward a set of intelligent DNS resolution system as the source station where purpose URL, by intelligence
Specific web proxy IP address is returned to the domain name request of the URL by dns resolution system.The choosing of Web proxy server
The standard of selecting can be the various selectable algorithms such as route distance is nearest, content response is most fast, subregion setting.
S105：Web proxy server cluster responds application (the network insertion mark in terminal as Reverse Proxy
Know) the URL access requests initiated, and after requested content is provided, generate the application layer daily record of this visit；
Following field is included at least in the application layer daily record：Timestamp that this application-level request terminates is initiated this time
The IP address of application-level request responds transport layer flow that this application-level request uses and the URL of this application-level request, removes
It, can also accurately the information of the specific URL of record access or utility cession information in application layer daily record other than this.Optionally, this reality
It can be Greenwich Mean Time to apply the example timestamp.
According to application layer access to content daily record, this URL corresponding content fileinfo, such as file size, file can be provided
Type, filename etc..By utility cession information, the corresponding application type of this visit can be provided, as e-commerce transaction,
Internet financial transaction and the flow of utility cession.
When the application in terminal repeatedly accesses, Web proxy server cluster generates application layer for access every time
Daily record.
Internet content cache webcache functions can be enabled on Web proxy server cluster in the present embodiment,
The high internet content of cache access frequency on server proxy cluster is accelerated and is saved network for internet access
Bandwidth.
In the present embodiment, Web proxy server cluster can be by using Squid, Traffic in PC server
The Open Source Codes such as Server realize above-mentioned function, therefore, realization it is of low cost, and can be carried out with the granularity of " server "
Dilatation, so as to fulfill the flatness of dilatation.
S106：When terminal (network access Identifier) stops accessing internet, aaa server is with withdrawing this bearing bed IP
Location, while a bearing bed conversation end daily record is generated, including：Network access Identifier and this bearing bed conversation end
Timestamp；
So far, complete bearing bed daily record (bearing bed session start daily record and carrying layer conversation knot are generated in aaa server
Shu Zhi), including following field：The IP that network access Identifier uses described in network access Identifier, this carrying layer conversation
The timestamp of location, the timestamp of this bearing bed session start and this bearing bed conversation end.
S107：Internet data acquisition system obtains the bearing bed daily record in preset time period from aaa server；
S108：Internet data acquisition system obtains the application layer day in preset time period from Web proxy server cluster
Will；
S109：Internet data acquisition system matches bearing bed daily record and application layer daily record, determines application layer day
Correspondence between will and network access Identifier：When the IP address for initiating this application-level request and this described bearing bed
The timestamp that the IP address that network access Identifier described in session uses matches and this described application-level request terminates is located at institute
When stating between the timestamp of this bearing bed session start and the timestamp of this bearing bed conversation end, the application layer is determined
There are correspondences between daily record and the network access Identifier.
S110：Internet data acquisition system acquires the network access Identifier when default according to the correspondence
Between in section, to the access content of the internet.
In general, application layer daily record includes the access content of internet, therefore, according to correspondence, net can be collected
Network access is identified in preset time period, to the access content of the internet.
S109 and S110 can realize the acquisition to the access content of the internet of network access Identifier, can acquire a certain
The access content of network access Identifier can also acquire the access content of multiple network access Identifier.
Based on network access Identifier in preset time period, to the access content of the internet, user couple can be counted
The statistical data that internet accesses, for example, the visit capacity of content URL provided for certain specific ICP can be provided, and or
Person is the visit capacity for certain certain types of content (such as video) flow.It, can be real based on this as a result, coordinating with charge system
Realize very fine granularity, and abundant content charging.For example, in addition to the flow to different definition applies mechanically different charging policy,
It also is able to apply mechanically content different in same flow different charging policy；In addition to that can realize to different ICP contents
Different charge strategy, additionally it is possible to realize inside same ICP, the different charge strategy of different content type；In addition to can to figure
The different contents such as piece and music type carries out carrying out charging to this termination number using different unit prices, it might even be possible to for similary
Two different contents (such as two head are similarly the music content of mp3 forms) of content type, are counted using different unit prices
Take.When being got through by the user data in network access Identifier and operator's crm system, it can be estimated that on internet in certain
The access object crowd characteristic of appearance/certain application, is the positioning of content/application, and the dispensing of embedded advertisement provides targeting data.
The method this implementation Suo Shu is illustrated below：With mobile internet access, terminal and web proxy service
For the scene for not including NAT device between device, the topological diagram of network with network access gateway as shown in Fig. 2, wherein, have
Deployment Web proxy server cluster in the network subnet of identical IP address space, for performing application proxy, file cache etc.
Function.Aaa server, dns server and crm system, crm system energy are at least deployed with inside the IT Operation Support System of ISP
The correspondence of network access Identifier and user's real information is enough provided.
Based on the topological structure described in Fig. 2, acquisition network access Identifier includes the process of the access content of internet：
1st, terminal (network access Identifier) is to network access gateway (fixed network BRAS, mobile network PDSN/GGSN/SAE
GW access request) is sent；
2nd, network access gateway as Authentication Client to certificate server transmit network access Identifier, certificate server to
Network access gateway issues the access IP address for distributing to terminal；Or by network access gateway for certification by terminal distribution
IP address.No matter which kind of mode, network access gateway can all be uploaded in the request message that charging starts network access Identifier and
Its used IP address, network insertion initial time give access authentication server；At the end of network insertion, network insertion net
The request message that will terminate to certificate server transmission charging is closed, uploads its network access Identifier, IP address, network insertion terminates
Time gives access authentication server.In this step, a bearing bed network insertion daily record is generated on access authentication server,
Record network access Identifier, this carrying layer conversation described in network access Identifier use IP address, this bearing bed meeting
Greenwich Mean Time when words start and Greenwich Mean Time during this bearing bed conversation end；
3rd, it is successfully accessed the terminal of network and initiates internet content access request to distribute to its access IP, the request
Destination address is adjusted to the IP address of web proxy cluster by means such as DNS intelligently parsings.Web proxy server
After cluster finishes Web content/application access request of a terminal being successfully processed, an application layer access to content day is generated
Will records access to content success timestamp：Time at the end of the response of this application-level request, form can be Greenwich
Time initiates client ip address, the transport layer flow of this application-level request：This time application layer responds corresponding TCP/IP stacks
The byte of transmission, URL addresses：The information such as the Internet resources mark URL of this access response；
4th, internet data acquisition system gets bearing bed daily record at aaa server, from web proxy cluster
Place gets application layer daily record, is stored in log server, as described in above-mentioned S109 and S110, carries out bearing bed daily record and application layer
The association of daily record, acquire the network access Identifier in preset time period, to the access content of the internet.
Method described in the present embodiment, by the way that content access request message routing is forwarded to Web proxy server cluster,
By Web proxy server cluster by response contents access request message, and application layer daily record is generated, and utilize aaa server
Application layer daily record and bearing bed daily record are carried out the matching of field, so that it is determined that network access Identifier by the bearing bed daily record of generation
Correspondence between application layer daily record further acquires visit of the network access Identifier in preset time period, to internet
Ask content.As it can be seen that the method described in the present embodiment does not need to carry out the deep analysis of data packet, set it is therefore not necessary to rely on DPI
It is standby, so cost can be reduced.
In addition to this, the present embodiment has further the advantage that：
1st, the present embodiment carries out routing forwarding using DNS technologies, because technology does not need to the terminal actively similar WAP of progress thus
The configuration of gateway address, therefore, terminal-pair routing forwarding is noninductive, i.e., routing forwarding process does not interfere with the normal communication stream of terminal
Journey.Meanwhile need to interrupt existing business with the existing method using DPI equipment come compared with setting DPI equipment collection point, this reality
Applying the method described in example will not interrupt existing business that DPI collection points and under each DPI equipment are configured on physical link
Identification feature code is sent out, only need to change dns resolution entry, does not need to issue that corresponding certain is specific in advance again to each DPI collecting devices
The identification feature code of business.
2nd, Web proxy server cluster generates application layer daily record by responding access request message, therefore, for internet
For topology, additional overhead will not be generated, so as to improve efficiency.
3rd, the precision of the method described in the present embodiment is high.
By taking a daily record of Squid softwares as an example：
1066037222.3521947527.34.49.248TCP_MISS/20012387
GET http://espanol.geocities.com/lebastias/divulgacion/budismo-
tarot.html
-
DIRECT/209.1.225.139text/html
It is the concrete meaning of journal entry in bracket：
(access response is spent (1066037222.352 the timestamp that access to content starts) 19475
Time millisecond number)
(27.34.49.248 accessing the client ip address initiated)
(result code of access/conditional code TCP_MISS represents to access to fail slow TCP_MISS/200
Middle hit is deposited, 200 represent that this access finally normally completes)
12387 (byte number of this content in ICP/IP protocol stack can be used to calculate the flow of content)
GET (request method of access to content) http://espanol.geocities.com/lebastias/
Divulgacion/budismo-tarot.html (the complete URL detail fields of the content accessed)-(client identity word
Section, it is empty in this example)
DIRECT/209.1.225.139 (opposite end coding/and to end main frame, opposite end code field indicates how to select next
It jumps, opposite end host field is the IP address of next-hop)
Text/html (content type field identifies which kind of classification is this section of content be)
It can thus be seen that access the fine of content (bold Italic part) using WEB server proxy clusters are collected
Information is spent, the IP five-tuples that be significantly larger than DPI modes (typically refer to source IP address, source port, purpose IP address, destination
Mouth and transport layer protocol number), flowing of access, type of service can be reached, access acts, and accesses the precise acquisition of content.Simultaneously
The log field of WEB proxy servers can also be extended by way of custom programming, collecting an internet content please
The full content asked.Such as present open source community has had plug-in code to be able to record that client ip makes when initiating this request
Port numbers.
4th, WEB proxy servers are deployed on the node for approaching terminal network access point, and access to content life is performed in agency
Into while accessing data, moreover it is possible to the function of similar CDN edges distribution node is played by opening caching function, it is mutual to accelerate user
Networking accesses experience.
In the embodiment shown in fig. 1, it other than S107 to S109, optionally, obtains application layer daily record and network connects
Correspondence between inlet identity can also use in the following manner：According to obtain apply daily record, determine the application layer daily record and
Correspondence between network access Identifier.
That is, for intelligent terminal (such as smart mobile phone or intelligent router), it can be in the application of intelligent terminal
(APP) customized development is carried out on so that directly carry its network insertion used in the request access message that intelligent terminal is sent
It identifies, then network access Identifier can be then recorded in application layer by Web proxy server cluster when access message is asked in processing
In daily record.So as to which internet data acquisition system can apply daily record according to acquisition, determine the application layer daily record and network
Correspondence between access mark.
There is no said for NAT device between terminal and Web proxy server cluster for embodiment shown in FIG. 1
It is bright, when between terminal and Web proxy server cluster there are during NAT device, compared with embodiment shown in FIG. 1, difference lies in：It obtains
The mode for taking the correspondence between application layer daily record and network access Identifier is：
Bearing bed daily record, the bearing bed daily record are obtained from the aaa server of the operational support system, OSS of the internet
Include following field：Network access Identifier, this carrying layer conversation described in network access Identifier use IP address, this
The timestamp of the timestamp of bearing bed session start and this bearing bed conversation end；
NAT conversion logs are obtained, the NAT conversion logs include following field：Original access IP address, access IP
When the corresponding port numbers of access IP address, address mapping start after access IP address, the NAT after the corresponding port numbers in location, NAT
Between and the address mapping end time；
Application layer daily record is obtained from the Web proxy server cluster, the application layer daily record includes following field：This
Timestamp that secondary application-level request terminates, access end slogan, responds this application layer at the IP address for initiating this application-level request
Ask the URL of transport layer flow and this application-level request used；
When initiating in the IP address of this application-level request and described address conversion log in the application layer daily record
By original obtained the accesss IP address of access IP address conversion is identical, the initiation in the application layer daily record is this time answered
The corresponding access end slogan of the IP address end corresponding with the access IP address in described address conversion log asked with layer
Slogan it is identical and, address of content response in the application layer daily record success timestamp in described address conversion log
In the interval of conversion time, it is determined that the IP address and described address of initiating this application-level request in the application layer daily record
Original access IP address in conversion log corresponds to；
If the original access that network access Identifier uses described in this carrying layer conversation in the bearing bed daily record
IP address is identical with the original access IP address in described address conversion log, and during session start-stop in the bearing bed daily record
Between end include time of described address conversion log, then can obtain in the bearing bed daily record and carry institute in layer conversation with this
State the corresponding network access Identifier of original access IP address that network access Identifier uses；So as to
Establish the correspondence of the network access Identifier and the application layer daily record.
For example, it is assumed that certain collected a part of daily record of analysis.
Certain bearing bed log content is as follows：
Address conversion daily record on NAT device has following three：
| Field name | The address conversion time | Original ip address before conversion | IP address is accessed after conversion |
| Field value | 10 minutes and 00 second 11 points of on January 1st, 2014 | 10.100.100.100 | 20.100.100.100 |
The application layer daily record that Web proxy server cluster generates has following three：
| Field name | The access to content end time | The URL of this access request | Initiate the IP address of this access |
| Field value | 10 minutes and 00 second 10 points of on January 1st, 2014 | www.a.com/txt1.html | 20.100.100.100 |
| Field name | The access to content end time | The URL of this access request | Initiate the IP address of this access |
| Field value | 40 minutes and 00 second 10 points of on January 1st, 2014 | www.b.com/music1.mp3 | 20.100.100.200 |
| Field name | The access to content end time | The URL of this access request | Initiate the IP address of this access |
| Field value | 10 minutes and 00 second 11 points of on January 1st, 2014 | www.c.com/pic1.png | 20.100.100.100 |
More than daily record is based on to be analyzed：
A:20.100.100.100,20.100.100.200 in application layer access log the two initiate content requests
IP address can be converted within the period in 00 40: 00 second 10 minutes to 10: 10 points of on January 1st, 2014 by NAT
Original ip address is 10.100.100.100 before daily record retrospect revolution is changed；It can held using this address of 10.100.100.100
In carrier layer daily record, in 10 points of 00 second to 2014 00 minute January 1,10 points of primary accesses of 59 minutes and 30 seconds on January 1st, 2014,
Network access Identifier is traced back to as MDN numbers 18612345678.So as to establish 18612345678 and www.a.com/
Access relation between the two internet contents of txt1.html and www.b.com/music1.mp3.
B:This IP address for initiating content requests of 20.100.100.100 in application layer access log can be 2014
In 10 minutes and 00 second 11 points of on January 1, original ip address is 10.100.100.100 before being changed by the retrospect revolution of NAT conversion logs.But
It is that by the analysis to bearing bed daily record, 10.100.100.100 this IP address is only 00 minute 00 10 points of on January 1st, 2014
18612345678 this access mark are assigned in second to 10 points of 59 minutes 30 seconds this periods of on January 1st, 2014 to use.
Therefore, it is impossible to trace back to 18612345678 this network access Identifier by IP address 10.100.100.100, can not also establish
The access relation of internet content.
A kind of method for acquiring internet access content disclosed by the embodiments of the present invention, system is acquired applied to internet data
System, the internet data acquisition system are arranged on the node for meeting preset condition in the IP bearer networks of the telecom operators
On, the method includes：
The correspondence between application layer daily record and network access Identifier is obtained, the application layer daily record is visited by responding terminal
Ask the content requests message generation of URL, the content requests message by routing forwarding to the Web proxy server cluster,
The Web proxy server cluster, which is arranged in the IP bearer networks of the telecom operators, to be met on the node of preset condition,
The terminal uses the network access Identifier；
According to the correspondence, visit of the network access Identifier in preset time period, to the internet is acquired
Ask content.
A kind of method for acquiring internet access content disclosed by the embodiments of the present invention, applied to Web proxy server collection
Group, the Web proxy server cluster are arranged on the node for meeting preset condition in the IP bearer networks of the telecom operators
On, the method includes：
The content requests message of terminal access URL is received, the content requests message passes through routing forwarding to the Web generations
Manage server cluster；
Application layer daily record is generated by responding the content requests message, the application layer daily record is used to obtain the application
Correspondence between layer daily record and network access Identifier, the correspondence are used to acquire the network access Identifier when default
Between in section, to the access content of the internet, the terminal uses the network access Identifier.
With the above method correspondingly, the embodiment of the invention also discloses it is a kind of acquire internet access content device,
As shown in figure 3, including：
Redirectional system 301 is forwarded, is forwarded for the network access Identifier to be accessed to the content requests message routing of URL
To Web proxy server cluster, the Web proxy server cluster is arranged in the IP bearer networks of the telecom operators full
On the node of sufficient preset condition；
The Web proxy server cluster 302 is used for, and the content that URL is accessed by responding the network access Identifier please
Message is asked to generate application layer daily record；
Internet data acquisition system 303, for obtaining the corresponding pass between the application layer daily record and network access Identifier
System, and according to the correspondence, acquire the network access Identifier in preset time period, in the access of the internet
Hold, the internet data acquisition system, which is arranged in the IP bearer networks of the telecom operators, meets the preset condition
On node.
The embodiment of the invention also discloses a kind of internet data acquisition system, the internet data acquisition system setting
Meet on the node of preset condition in the IP bearer networks of the telecom operators, as shown in figure 4, including：
Acquisition module 401, for obtaining the correspondence between application layer daily record and network access Identifier, the application layer day
Will is generated by responding the content requests message of terminal access URL, and the content requests message passes through routing forwarding to the Web
Server proxy cluster, the Web proxy server cluster, which is arranged in the IP bearer networks of the telecom operators, to be met in advance
If on the node of condition, the terminal uses the network access Identifier；
Acquisition module 402, for according to the correspondence, acquire the network access Identifier in preset time period,
To the access content of the internet.
Optionally, the acquisition module can have following three kinds of specific implementations：
1st, the acquisition module includes：
First acquisition unit, for obtaining bearing bed daily record, the bearing bed from the aaa server in the internet
Daily record includes following field：Network access Identifier, this carrying layer conversation described in network access Identifier use IP address,
The timestamp of the timestamp of this bearing bed session start and this bearing bed conversation end；
Second acquisition unit, for obtaining application layer daily record, the application layer daily record from the Web proxy server cluster
Include following field：Timestamp that this application-level request terminates, the IP address for initiating this application-level request, response are this time
The transport layer flow and the URL of this application-level request that application-level request uses；
First determination unit, for working as the IP address for initiating this application-level request and this described carrying layer conversation
Described in the IP address matching that uses of network access Identifier and the timestamp that terminates of this described application-level request be located at described
When between the timestamp of secondary bearing bed session start and the timestamp of this bearing bed conversation end, the application layer daily record is determined
There are correspondences between the network access Identifier.
2nd, the acquisition module includes：
First acquisition unit, for obtaining bearing bed daily record, the bearing bed from the aaa server in the internet
Daily record includes following field：What network access Identifier used described in network access Identifier, this carrying layer conversation original connects
Enter the timestamp of IP address, the timestamp of this bearing bed session start and this bearing bed conversation end；
Third acquiring unit, for obtaining address conversion daily record, described address conversion log includes following field：It is original
Access IP address, the original access for accessing the corresponding port numbers of IP address, being obtained by the original access IP address conversion
IP address, the corresponding port numbers of the access IP address and address conversion time；
Second acquisition unit, for obtaining application layer daily record, the application layer daily record from the Web proxy server cluster
Include following field：Timestamp that this application-level request terminates, the IP address for initiating this application-level request, the initiation
The corresponding access end slogan of IP address of this application-level request, responds this application-level request at content response success timestamp
The transport layer flow and the URL of this application-level request used；
Correspondence relationship establishing unit, for work as in the application layer daily record initiation this application-level request IP address with
In described address conversion log by the original obtained access IP address of access IP address conversion is identical, application layer day
The initiation in the will this time corresponding access end slogan of IP address of application-level request and the institute in described address conversion log
State access the corresponding port numbers of IP address it is identical and, the content response success timestamp in the application layer daily record is described
In the interval of address conversion time in address conversion daily record, it is determined that this time application layer please for the initiation in the application layer daily record
The IP address asked is corresponding with the original access IP address in described address conversion log；If the carrying is into this in daily record
The original access IP address that network access Identifier uses described in carrying layer conversation is identical with the original access IP address, obtains
It is corresponding with the original access IP address that network access Identifier uses described in this carrying layer conversation in the bearing bed daily record
Network access Identifier, and establish the correspondence of the network access Identifier and the application layer daily record.
3rd, the acquisition module includes：
Second determination unit for applying daily record according to what is obtained, determines the application layer daily record and network access Identifier
Between correspondence, the application layer daily record includes the network access Identifier.
The embodiment of the invention also discloses a kind of Web proxy server cluster, the Web proxy server cluster is arranged on
Meet on the node of preset condition in the IP bearer networks of the telecom operators, as shown in figure 5, including：
Receiving module 501, for receiving the content requests message of terminal access URL, the content requests message passes through road
By being forwarded to the Web proxy server cluster；
Generation module 502 responds the content requests message generation application layer daily record, the application layer daily record for passing through
For obtaining the correspondence between the application layer daily record and network access Identifier, the correspondence is used to acquire the network
Access is identified in preset time period, to the access content of the internet, and the terminal uses the network access Identifier.
Optionally, the Web proxy server cluster can have the function of Internet content cache.
Optionally, the Web proxy server cluster can further include：
If for carrying the network access Identifier in the content requests message, the network is connect for logging modle
Inlet identity is recorded in the application layer daily record.
If the function described in the present embodiment method is realized in the form of SFU software functional unit and is independent product pin
It sells or in use, can be stored in a computing device read/write memory medium.Based on such understanding, the embodiment of the present invention
The part contribute to the prior art or the part of the technical solution can be embodied in the form of software product, this is soft
Part product is stored in a storage medium, used including some instructions so that computing device (can be personal computer,
Server, mobile computing device or network equipment etc.) perform all or part of step of each embodiment the method for the present invention
Suddenly.And aforementioned storage medium includes：USB flash disk, read-only memory (ROM, Read-Only Memory), is deposited mobile hard disk at random
The various media that can store program code such as access to memory (RAM, Random Access Memory), magnetic disc or CD.
Each embodiment is described by the way of progressive in this specification, the highlights of each of the examples are with it is other
The difference of embodiment, just to refer each other for same or similar part between each embodiment.
The foregoing description of the disclosed embodiments enables professional and technical personnel in the field to realize or use the present invention.
A variety of modifications of these embodiments will be apparent for those skilled in the art, it is as defined herein
General Principle can be realized in other embodiments without departing from the spirit or scope of the present invention.Therefore, it is of the invention
The embodiments shown herein is not intended to be limited to, and is to fit to and the principles and novel features disclosed herein phase one
The most wide range caused.
Claims (17)
- A kind of 1. method for acquiring internet access content, which is characterized in that applied to internet data acquisition system, the side Method includes：The correspondence between application layer daily record and network access Identifier is obtained, the application layer daily record is united by responding terminal access The content requests message generation of one Resource Locator URL, the content requests message pass through routing forwarding to Web proxy server Cluster, the Web proxy server cluster, which is arranged in the IP bearer networks of telecom operators, to be met on the node of preset condition, The preset condition includes：The node can connect internet data acquisition system and the Certificate Authority of telecom operators is kept accounts The time synchronization of Web proxy server cluster in communication and the node and the AAA system between AAA system, it is described AAA system includes multiple aaa servers；The terminal uses the network access Identifier；According to the correspondence, acquire the network access Identifier in preset time period, in the access of the internet Hold.
- 2. according to the method described in claim 1, it is characterized in that, between the acquisition application layer daily record and network access Identifier Correspondence includes：Bearing bed daily record is obtained from the aaa server in the internet, the bearing bed daily record includes following field：Net Network access Identifier uses described in network access mark, this carrying layer conversation IP address, this bearing bed session start The timestamp of timestamp and this bearing bed conversation end；Application layer daily record is obtained from the Web proxy server cluster, the application layer daily record includes following field：This should The timestamp that is terminated with layer request, responds the transmission that this application-level request uses at the IP address for initiating this application-level request The URL of laminar flow amount and this application-level request；When the IP address for initiating this application-level request and network access Identifier described in this described carrying layer conversation make IP address matches and the timestamp that terminates of this described application-level request be located at this bearing bed session start when Between stamp the timestamp of this bearing bed conversation end between when, determine between the application layer daily record and the network access Identifier There are correspondences.
- 3. according to the method described in claim 1, it is characterized in that, between the acquisition application layer daily record and network access Identifier Correspondence includes：Bearing bed daily record is obtained from the aaa server in the internet, the bearing bed daily record includes following field：Net Network access identifies, this carries the original access IP address, this bearing bed meeting that network access Identifier uses described in layer conversation Talk about the timestamp started and the timestamp of this bearing bed conversation end；Address conversion daily record is obtained, described address conversion log includes following field：Original access in address conversion daily record Corresponding port numbers of original access IP address in IP address, described address conversion log, by described address conversion log Access IP address, the corresponding port numbers of the access IP address and the address conversion time that original access IP address conversion obtains；Application layer daily record is obtained from the Web proxy server cluster, the application layer daily record includes following field：This should The timestamp terminated with layer request, the corresponding access end slogan of IP address, the IP address, interior for initiating this application-level request Hold and respond successful timestamp, respond transport layer flow that this application-level request uses and the URL of this application-level request；When in IP address and the described address conversion log of initiation this application-level request in the application layer daily record by institute State that the access IP address that the original access IP address conversion in address conversion daily record obtains is identical, the institute in the application layer daily record State the corresponding access end slogan of IP address it is identical with the corresponding port numbers of the access IP address in described address conversion log, And address conversion time of the content response success timestamp in described address conversion log in the application layer daily record In interval, it is determined that initiated in the IP address of this application-level request and described address conversion log in the application layer daily record Original access IP address correspond to；If the original access IP that network access Identifier uses described in this carrying layer conversation in the bearing bed daily record Location is identical with the original access IP address in described address conversion log, obtain in the bearing bed daily record with this bearing bed meeting The corresponding network access Identifier of original access IP address that network access Identifier described in words uses；Establish the correspondence of the network access Identifier and the application layer daily record.
- 4. according to the method described in claim 1, it is characterized in that, between the acquisition application layer daily record and network access Identifier Correspondence includes：Daily record is applied according to what is obtained, determines the correspondence between the application layer daily record and network access Identifier, the application Layer daily record includes the network access Identifier.
- A kind of 5. method for acquiring internet access content, which is characterized in that applied to Web proxy server cluster, the Web Server proxy cluster, which is arranged in the IP bearer networks of telecom operators, to be met on the node of preset condition, the method packet It includes：The content requests message of terminal access URL is received, the content requests message is taken by routing forwarding to the web proxy Business device cluster；Application layer daily record is generated by responding the content requests message, the application layer daily record is used to obtain the application layer day Correspondence between will and network access Identifier, the correspondence are used to acquire the network access Identifier in preset time period Access content interior, to the internet, the terminal use the network access Identifier.
- 6. according to the method described in claim 5, it is characterized in that, the Web proxy server cluster has internet content Caching function.
- 7. method according to claim 5 or 6, which is characterized in that further include：If carrying the network access Identifier in the content requests message, the network access Identifier is recorded in described answer With in layer daily record.
- A kind of 8. method for acquiring internet access content, which is characterized in that including：The content requests message routing that network access Identifier is accessed URL by the forwarding Redirectional system in the internet is forwarded to Web proxy server cluster, the Web proxy server cluster are arranged on to meet in the IP bearer networks of telecom operators and preset On the node of condition；The content requests message generation that the Web proxy server cluster accesses URL by responding the network access Identifier should With layer daily record；Internet data acquisition system obtains the correspondence between the application layer daily record and network access Identifier, and according to described Correspondence, acquire the network access Identifier in preset time period, to the access content of the internet.
- 9. according to the method described in claim 8, it is characterized in that, the preset condition includes：Connect the communication between the Certificate Authority accounting system of internet data acquisition system and telecom operators；When, there are during the conversion of IP address, being preserved on address-translating device between terminal and the Web proxy server cluster The daily record of address conversion；And it is connected to equipment on the node and synchronizes in time.
- 10. a kind of internet data acquisition system, which is characterized in that including：Acquisition module, for obtaining the correspondence between application layer daily record and network access Identifier, the application layer daily record passes through The content requests message generation of terminal access URL is responded, the content requests message passes through routing forwarding to Web proxy server Cluster, the Web proxy server cluster, which is arranged in the IP bearer networks of telecom operators, to be met on the node of preset condition, The terminal uses the network access Identifier；Acquisition module, for according to the correspondence, acquire the network access Identifier in preset time period, to it is described mutually The access content of networking.
- 11. internet data acquisition system according to claim 10, which is characterized in that the acquisition module includes：First acquisition unit, for obtaining bearing bed daily record, the bearing bed daily record from the aaa server in the internet Include following field：Network access Identifier, this carrying layer conversation described in network access Identifier use IP address, this The timestamp of the timestamp of bearing bed session start and this bearing bed conversation end；Second acquisition unit for obtaining application layer daily record from the Web proxy server cluster, wraps in the application layer daily record Include following field：Timestamp that this application-level request terminates, the IP address for initiating this application-level request, response are this time applied The transport layer flow and the URL of this application-level request that layer request uses；First determination unit, for working as the IP address for initiating this application-level request and institute in this described carrying layer conversation It states the IP address matching that network access Identifier uses and the timestamp that this described application-level request terminates is located at described this and holds When between the timestamp of the timestamp of carrier layer session start and this bearing bed conversation end, the application layer daily record and institute are determined State between network access Identifier that there are correspondences.
- 12. internet data acquisition system according to claim 10, which is characterized in that the acquisition module includes：First acquisition unit, for obtaining bearing bed daily record, the bearing bed daily record from the aaa server in the internet Include following field：The original access IP that network access Identifier uses described in network access Identifier, this carrying layer conversation The timestamp of address, the timestamp of this bearing bed session start and this bearing bed conversation end；Third acquiring unit, for obtaining address conversion daily record, described address conversion log includes following field：Address conversion Corresponding port numbers of original access IP address in original access IP address, described address conversion log in daily record, by described Access IP address that original access IP address conversion in address conversion daily record obtains, the corresponding port numbers of the access IP address With the address conversion time；Second acquisition unit for obtaining application layer daily record from the Web proxy server cluster, wraps in the application layer daily record Include following field：Timestamp that this application-level request terminates, IP address, the IP address pair for initiating this application-level request The access end slogan answered, content response success timestamp respond transport layer flow that this application-level request uses and this should The URL asked with layer；Correspondence relationship establishing unit, for work as in the application layer daily record initiation this application-level request IP address with it is described The access IP address obtained by the original access IP address conversion in described address conversion log in address conversion daily record is identical, The corresponding access end slogan of the IP address in the application layer daily record and the access IP in described address conversion log The corresponding port numbers in address it is identical and, content response in application layer daily record success timestamp is converted in described address In the interval of address conversion time in daily record, it is determined that the IP of initiation this application-level request in the application layer daily record Location is corresponding with the original access IP address in described address conversion log；If this bearing bed meeting in the bearing bed daily record The original access IP address that network access Identifier described in words uses and the original access IP address in described address conversion log It is identical, it obtains in the bearing bed daily record with the original access IP that uses of network access Identifier described in this carrying layer conversation The corresponding network access Identifier in location, and establish the correspondence of the network access Identifier and the application layer daily record.
- 13. internet data acquisition system according to claim 10, which is characterized in that the acquisition module includes：Second determination unit for applying daily record according to what is obtained, is determined between the application layer daily record and network access Identifier Correspondence, the application layer daily record include the network access Identifier.
- 14. a kind of Web proxy server cluster, which is characterized in that the Web proxy server cluster is arranged on telecom operators IP bearer networks in meet on the node of preset condition, including：Receiving module, for receiving the content requests message of terminal access URL, the content requests message by routing forwarding extremely The Web proxy server cluster；Generation module responds the content requests message generation application layer daily record for passing through, and the application layer daily record is used to obtain The correspondence between the application layer daily record and network access Identifier is taken, the correspondence is used to acquire the network insertion mark Know in preset time period, to the access content of internet, the terminal uses the network access Identifier.
- 15. Web proxy server cluster according to claim 14, which is characterized in that the Web proxy server cluster There is Internet content cache.
- 16. the Web proxy server cluster according to claims 14 or 15, which is characterized in that further include：Logging modle, if for carrying the network access Identifier in the content requests message, by the network insertion mark Knowledge is recorded in the application layer daily record.
- 17. a kind of device for acquiring internet access content, which is characterized in that including：Redirectional system is forwarded, the content requests message routing for network access Identifier to be accessed to URL is forwarded to web proxy clothes Business device cluster, the Web proxy server cluster are arranged on the section for meeting preset condition in the IP bearer networks of telecom operators Point on；The Web proxy server cluster is used for, and the content requests message that URL is accessed by responding the network access Identifier is given birth to Into application layer daily record；Internet data acquisition system, for obtaining the correspondence between the application layer daily record and network access Identifier, and according to According to the correspondence, acquire the network access Identifier in preset time period, to the access content of the internet.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410208321.0A CN105100015B (en) | 2014-05-16 | 2014-05-16 | A kind of method and device for acquiring internet access data |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410208321.0A CN105100015B (en) | 2014-05-16 | 2014-05-16 | A kind of method and device for acquiring internet access data |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN105100015A CN105100015A (en) | 2015-11-25 |
| CN105100015B true CN105100015B (en) | 2018-07-03 |
Family
ID=54579574
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201410208321.0A Active CN105100015B (en) | 2014-05-16 | 2014-05-16 | A kind of method and device for acquiring internet access data |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN105100015B (en) |
Families Citing this family (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10666961B2 (en) | 2016-01-08 | 2020-05-26 | Qualcomm Incorporated | Determining media delivery event locations for media transport |
| JP2017156872A (en) * | 2016-02-29 | 2017-09-07 | カシオ計算機株式会社 | Communication device, web access control method and program |
| CN106909325B (en) * | 2016-06-30 | 2020-08-18 | 阿里巴巴集团控股有限公司 | Log printing method and device and log printing system |
| CN110213315B (en) * | 2018-05-30 | 2022-01-07 | 腾讯科技（深圳）有限公司 | Path determining method and device |
| CN110956349B (en) * | 2018-09-27 | 2023-05-09 | 阿里巴巴集团控股有限公司 | Quality of service analysis method, system, device, server and electronic equipment |
| CN109451088A (en) * | 2018-10-30 | 2019-03-08 | 新华三大数据技术有限公司 | A kind of data access method and device |
| CN109743411B (en) * | 2018-12-10 | 2022-03-01 | 厦门市美亚柏科信息股份有限公司 | Method, device and storage medium for dynamically scheduling IP proxy pool in distributed environment |
| CN111600777A (en) * | 2020-05-20 | 2020-08-28 | 国网浙江省电力有限公司电力科学研究院 | Network flow collecting method and system based on VPN |
| CN113824685B (en) * | 2021-08-20 | 2023-07-14 | 联通沃音乐文化有限公司 | Mobile terminal directional flow agent system and method based on Android VpnService |
| CN114039875B (en) * | 2021-10-30 | 2023-09-01 | 北京网聚云联科技有限公司 | Data acquisition method, device and system based on eBPF technology |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7194454B2 (en) * | 2001-03-12 | 2007-03-20 | Lucent Technologies | Method for organizing records of database search activity by topical relevance |
| CN101083538A (en) * | 2006-05-30 | 2007-12-05 | 卓望数码技术（深圳）有限公司 | Real-time counting method for value added business of IP network environment |
| CN101651707A (en) * | 2009-09-22 | 2010-02-17 | 西安交通大学 | Method for automatically acquiring user behavior log of network |
| CN103259793A (en) * | 2013-05-02 | 2013-08-21 | 东北大学 | Method for inspecting deep packets based on suffix automaton regular engine structure |
| CN103347089A (en) * | 2013-07-16 | 2013-10-09 | 星云融创（北京）信息技术有限公司 | Method and device for separating and accelerating dynamic resources and static resources of website |
-
2014
- 2014-05-16 CN CN201410208321.0A patent/CN105100015B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7194454B2 (en) * | 2001-03-12 | 2007-03-20 | Lucent Technologies | Method for organizing records of database search activity by topical relevance |
| CN101083538A (en) * | 2006-05-30 | 2007-12-05 | 卓望数码技术（深圳）有限公司 | Real-time counting method for value added business of IP network environment |
| CN101651707A (en) * | 2009-09-22 | 2010-02-17 | 西安交通大学 | Method for automatically acquiring user behavior log of network |
| CN103259793A (en) * | 2013-05-02 | 2013-08-21 | 东北大学 | Method for inspecting deep packets based on suffix automaton regular engine structure |
| CN103347089A (en) * | 2013-07-16 | 2013-10-09 | 星云融创（北京）信息技术有限公司 | Method and device for separating and accelerating dynamic resources and static resources of website |
Also Published As
| Publication number | Publication date |
|---|---|
| CN105100015A (en) | 2015-11-25 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN105100015B (en) | A kind of method and device for acquiring internet access data | |
| JP4195450B2 (en) | Single sign-on method for packet radio network users roaming multi-country operator networks | |
| Wittie et al. | Exploiting locality of interest in online social networks | |
| CN103069776B (en) | Content distributing network (CDN) is expanded to mobile or cable network | |
| CN104994079B (en) | The treating method and apparatus of access request accelerates server | |
| CN103051725B (en) | Application and identification method, data digging method, Apparatus and system | |
| CN105025044B (en) | A kind of apparatus control method and system | |
| CN104160680B (en) | Cheating Technology for transparent proxy cache | |
| CN109491758A (en) | Docker mirror image distribution method, system, data gateway and computer readable storage medium | |
| CN104640114B (en) | A kind of verification method and device of access request | |
| CN106067890B (en) | A kind of domain name analytic method, apparatus and system | |
| CN109509041B (en) | Internet advertisement putting method and device | |
| CN101179389A (en) | Peer-to-peer file download system of IMS network | |
| CN107135236A (en) | A kind of detection method and system of target Domain Hijacking | |
| CN101375264A (en) | Techniques for accounting for multiple transactions in a transport control protocol (TCP) payload | |
| CN110796466B (en) | Internet advertisement putting method and device | |
| CN108390955A (en) | Domain Name acquisition method, Website access method and server | |
| CN103535011A (en) | Routing method, device, and system in content delivery network (CDN) | |
| CN103916491B (en) | Dynamic address mapping method and device based on NAT444 architecture | |
| CN103888539B (en) | Bootstrap technique, device and the P2P caching systems of P2P cachings | |
| CN102025595A (en) | Flow optimization method and system | |
| CN102695167A (en) | Mobile subscriber identity management method and apparatus thereof | |
| US7793352B2 (en) | Sharing network access capacities across internet service providers | |
| CN103997479B (en) | A kind of asymmetric services IP Proxy Methods and equipment | |
| CN104348841B (en) | Content distribution method, analysis and managing and control system and content distribution network system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |