CN102025559B - Method for detecting and processing dead links on basis of classification, and network equipment - Google Patents

Method for detecting and processing dead links on basis of classification, and network equipment Download PDF

Info

Publication number
CN102025559B
CN102025559B CN 201010536638 CN201010536638A CN102025559B CN 102025559 B CN102025559 B CN 102025559B CN 201010536638 CN201010536638 CN 201010536638 CN 201010536638 A CN201010536638 A CN 201010536638A CN 102025559 B CN102025559 B CN 102025559B
Authority
CN
China
Prior art keywords
link
linking status
record
chained record
dead chain
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN 201010536638
Other languages
Chinese (zh)
Other versions
CN102025559A (en
Inventor
张博
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Baidu Netcom Science and Technology Co Ltd
Original Assignee
Beijing Baidu Netcom Science and Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Baidu Netcom Science and Technology Co Ltd filed Critical Beijing Baidu Netcom Science and Technology Co Ltd
Priority to CN 201010536638 priority Critical patent/CN102025559B/en
Publication of CN102025559A publication Critical patent/CN102025559A/en
Application granted granted Critical
Publication of CN102025559B publication Critical patent/CN102025559B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Landscapes

  • Computer And Data Communications (AREA)
  • Data Exchanges In Wide-Area Networks (AREA)

Abstract

The invention relates to a method for detecting and processing dead links on the basis of classification, and network equipment. In the invention, the method comprises the steps of: updating or keeping a linked record of a link through a first predetermined rule and the current link state of the link in a dead link library; and carrying out corresponding operations on the link and/or the linked record according to the updated or kept linked record. Compared with the prior art, the invention has the following advantages of classifying the dead links into a plurality of states according to a plurality of detection results of the dead links and using different processing manners to the dead links in various states so as to realize the quick detection of the dead links while detecting an immense amount of dead links.

Description

Be used for carrying out method and the network equipment that dead chain detects and handles based on classification
Technical field
The present invention relates to field of computer technology, relate in particular to for carry out method and the network equipment that dead chain detects and handles based on classification.
Background technology
In the prior art, usually because of multiple reason, for example: the 1) database problem of dynamic website or webpage; 2) catalogue changes path (URL) change that causes; 3) move etc. the position of the file in the webpage or webpage, causes dead chain.On the one hand, for large-scale website, there is dead chain in the web page interlinkage that often is difficult in time find to provide; On the other hand, because dead chain once was the chain of living, therefore tends to searched engine and be recorded in the index database, when the relevant information of user's input information and dead chain is complementary, damned chain will be provided for the user, cause the user to obtain garbage.
In the prior art, only provide the detection mode of dead chain, but the internet link enormous amount no matter be to website or search engine, often is difficult to fast and effeciently upgrade the dead chain information of magnanimity, also lacks the early warning mechanism of dead chain.
Therefore, how to provide the dead chain of a kind of magnanimity that can come into force fast to detect processing method, become those skilled in the art's technical issues that need to address.
Summary of the invention
The purpose of this invention is to provide a kind of for carry out method and the network equipment that dead chain detects and handles based on classification.
According to an aspect of the present invention, provide a kind of being used in the network equipment to carry out the method that dead chain detects and handles based on classification, wherein, this method may further comprise the steps:
C is based on first pre-defined rule, in conjunction with the current link state that links in the dead chain storehouse, to upgrade or to keep the chained record of this link;
D, comes corresponding operation is carried out in this link and/or its chained record according to the described chained record that upgrades the back or keep based on second pre-defined rule.
According to another aspect of the present invention, also provide a kind of for carry out the network equipment that dead chain detects and handles based on classification, wherein, this network equipment comprises:
Record updating device, be used for based on first pre-defined rule, in conjunction with the current link state that links in the dead chain storehouse, to upgrade or to keep the chained record of this link;
Processing unit, be used for based on second pre-defined rule, according to the described chained record that upgrades the back or keep, come corresponding operation is carried out in this link and/or its chained record.
Compared with prior art, the present invention has the following advantages: the present invention is according to the repeated detection result of dead chain, the chain of checkmating is divided into various states, and the dead chain of various states adopted different processing modes, make and when detecting the dead chain of magnanimity, to realize the fast detecting of dead chain according to the solution of the present invention.
Description of drawings
By reading the detailed description of doing with reference to the following drawings that non-limiting example is done, it is more obvious that other features, objects and advantages of the present invention will become:
Fig. 1 is the method flow diagram that is used for carrying out based on classification dead chain detection and processing of one aspect of the invention;
Fig. 2 is the method flow diagram that is used for carrying out based on classification dead chain detection and processing of a preferred embodiment of the invention;
Fig. 3 is the method flow diagram that is used for carrying out based on classification dead chain detection and processing of another preferred embodiment of the present invention;
Fig. 4 is the method flow diagram that is used for carrying out based on classification dead chain detection and processing of another preferred embodiment of the present invention;
Fig. 5 is the method flow diagram that is used for carrying out based on classification dead chain detection and processing of another preferred embodiment of the present invention;
Fig. 6 is the network equipment structural representation that is used for carrying out based on classification dead chain detection and processing of one aspect of the invention;
Fig. 7 is the network equipment structural representation that is used for carrying out based on classification dead chain detection and processing of a preferred embodiment of the invention;
Fig. 8 is the network equipment structural representation that is used for carrying out based on classification dead chain detection and processing of another preferred embodiment of the present invention;
Fig. 9 is the dead chain detection of the linking status in short-term of another preferred embodiment of the present invention and the network equipment structural representation of handling;
Same or analogous Reference numeral represents same or analogous parts in the accompanying drawing.
Embodiment
Below in conjunction with accompanying drawing the present invention is described in further detail.
Fig. 1 shows the flow chart that is used for carrying out based on classification the method for dead chain detection and processing of one aspect of the invention.Wherein, the described network equipment includes but not limited to: 1) a plurality of webserver collection; 2) distributed network equipment; 3) based on set of computers of the cloud that is constituted by a large amount of computers or the webserver of cloud computing (Cloud Computing) etc.Wherein, cloud computing is a kind of of Distributed Calculation, a super virtual machine of being made up of the loosely-coupled computer collection of a group.The present invention also comprises dead chain storehouse, and described dead chain storehouse can be included in the network equipment, or with network equipment physical separation but link to each other by network service, and described dead chain storehouse can be as a whole, or is divided into a plurality of physical separation but the part that connects by network service.Described network includes but not limited to: the Internet, wide area network, metropolitan area network, local area network (LAN), VPN network, wireless self-organization network (Ad Hoc network) etc.Link in the dead chain storehouse can be upgraded by artificial or the network equipment.
In step S3, the network equipment is based on first pre-defined rule, in conjunction with the current link state that links in the dead chain storehouse, to upgrade or to keep the chained record of this link.Wherein, for a link, its linking status can by with this information that is associated that links, this link determining positions of storing maybe, for example, can link the identification information that is associated, can represent linking status with this and learn the linking status that this links by reading, perhaps, set up the chained library of different classifications in described dead chain storehouse, each chained library is used for the link of storage different linking state, then link residing chained library by this, can obtain the state of this link; Described chained record includes but not limited to following at least one:
1) this is linked at the testing result in each dead chain detection;
2) detection time of the each dead chain detection of this link;
3) the current link state of this link and url history state;
4) this link adds the time in the dead chain storehouse;
5) time of the link-state change of this link etc.
Particularly, the processing mode that has comprised the chained record of the link under one or more linking status in first pre-defined rule, when the network equipment according to first pre-defined rule, in the time of need carrying out respective handling to the chained record of the link of certain linking status, the link of searching this linking status from dead chain storehouse is carried out respective handling to its chained record; For the link of the linking status that does not relate in first pre-defined rule, it is constant that the network equipment is kept its chained record.
Wherein, the network equipment mode that obtains the link of required linking status from dead chain storehouse includes but not limited to:
1) by the identification information of query link, judges that whether this link is the link of required linking status, obtains corresponding link;
2) chained library at the link place of the required linking status of visit storage obtains corresponding link.
Wherein, the network equipment includes but not limited to the processing mode that chained record carries out respective handling according to first pre-defined rule:
The link of i is carried out dead chain and is detected, and with testing result and/or be updated to detection time in the chained record of this this link;
For example, based on first pre-defined rule, the network equipment need carry out dead chain to the link under the linking status and detect, and then obtains the link of this linking status Www.soopat.comAfter, the network equipment carries out dead chain to this link and detects, and testing result and/or detection time are updated to Www.soopat.comChained record in.
Wherein, described dead chain detects and includes but not limited to multiple detection mode:
1) for example, the network equipment is for link Www.soopat.comCarry out one-time detection, and according to this time testing result, judge whether this link is dead chain;
2) again for example, when user's visit capacity is lower than a predetermined low discharge threshold value, to link Www.soopat.comDetect, judge that it whether can successful access, if not, judge that then this is linked as dead chain, if, then when user's visit capacity is higher than a predetermined high flow capacity threshold value, to link Www.soopat.comCarry out one-time detection again, judge that it whether can successful access, if, judge that then this is linked as chain alive, if not, judge that then this is linked as dead chain;
3) more for example, according to statistics, obtain the section and slack hours section in rush hour of user's visit, then when the slack hours section, to link Www.soopat.comDetect, judge that it whether can successful access, if not, judge that then this is linked as dead chain, if, then in rush hour during section, to link Www.soopat.comCarry out one-time detection again, judge that it whether can successful access, if, judge that then this is linked as chain alive, if not, judge that then this is linked as dead chain.
Need to prove that the network equipment judges that the factor that link whether can successful access comprises following at least one:
1) whether the corresponding webpage of link can show;
2) whether the corresponding webpage of link can show full content;
3) whether the corresponding multimedia file of link can visit etc.
What need further specify is, above-mentioned for example only for dead chain detection mode is described better, but not the restriction that the present invention is done, in fact, it should be appreciated by those skilled in the art that anyly by link is detected, judge whether it is the mode of dead chain, all should be within the scope of the present invention, and comprise by reference.
Ii carries out union operation to a plurality of identical links under the same linking status, and records the number of links that merges in the chained record of the link after merging;
Particularly, may record a plurality of identical links in the dead chain storehouse, then based on first pre-defined rule, the network equipment taps into row to the same chain under the same linking status and merges, and records the number of links that merges in the chained record of the link after merging;
For example, in dead chain storehouse, for link Www.soopat.com, it can be added in the different moment till death in the chain storehouse, and relevant information difference when at every turn adding, or is stored in different positions, then can have a plurality of links simultaneously in the dead chain storehouse Www.soopat.com, the network equipment is based on first pre-defined rule, under the same linking status Www.soopat.comMerge, and Www.soopat.comChained record in record the number of links that merges.
Need to prove that though union operation relates to the operation of the redundant link of deletion, its essence is still the chained record of a link of change, therefore, still and with it be considered as the operation that the chained record to a link carries out.
Iii keep the link and chained record constant;
Concrete, comprised in first pre-defined rule when linking and be in predetermined following time of one or more linking status, keep the constant rule of this link and chained record thereof, then the network equipment is according to this rule, and it is constant to keep this chained record.Perhaps, for the linking status that does not relate in first pre-defined rule, link and chained record thereof that network equipment acquiescence is kept under this linking status are constant.
What need further specify is, above-mentioned for example only for the processing mode of carrying out based on first pre-defined rule among the present invention is described better, but not the restriction that the present invention is done, in fact, it should be appreciated by those skilled in the art that anyly based on first pre-defined rule, come link and chained record thereof are carried out the mode of respective handling, all should be within the scope of the present invention, and comprise by reference.
In step S4, the network equipment, comes corresponding operation is carried out in this link and/or its chained record according to the described chained record that upgrades the back or keep based on second pre-defined rule.
Concrete, second pre-defined rule comprises following at least one:
I meets the chained record requirement of the link of other linking status when the chained record of described link, and then the link-state change that will link is these other linking status, and correspondingly changes the chained record of this link;
Concrete, the network equipment is according to second pre-defined rule, detect the chained record of the link under each linking status, judge the chained record of the link under the linking status when the network equipment, when meeting the requiring of chained record under other linking status, the link-state change that the network equipment will link is these other linking status, and correspondingly records in the chained record of this link the change time of linking status.
For example, for link Www.soopat.com, the network equipment detects the nearest five times dead chain testing result that comprises in its chained record and is " chain of living, dead chain, the chain of living, dead chain, dead chain ", and then its number of times that is detected as dead chain surpasses twice, meets the chained record requirement of other linking status, then will Www.soopat.comLink-state change be satisfactory other states.
II then deletes this link when described link or its chained record meet the chained record requirement of Remove Links;
Particularly, the network equipment detects the chained record of the link under each linking status according to second pre-defined rule, judges the chained record of the link under the linking status when the network equipment, when meeting the requiring of chained record of Remove Links, the network equipment is deleted this link.
For example, for link Www.soopat.com, the network equipment detects and does not comprise the link that is complementary with it in the index database, and then the network equipment is deleted this link.
III then upgrades or keeps the chained record of this link when the chained record of described link meets the chained record requirement of the state at this link place.
Concrete, the network equipment is according to second pre-defined rule, detect the chained record of the link under each linking status, judge the chained record of the link under the linking status when the network equipment, meet the chained record requirement of the state at this link place, it is constant then to keep this link and chained record thereof, or records this judged result in the chained record of this link.
Need to prove that above-mentioned steps S3 and step S4 there is no sequencing.
In the present embodiment, linking status comprises multiple dividing mode, but to every kind of dividing mode, all can carry out correspondingly link detection and processing by step S3 and step S4.
For example, linking status is divided into first linking status, second linking status and the 3rd linking status.Correspondingly, comprise following rule in first pre-defined rule:
1) whether is dead chain with M1 as the link that detects first linking status blanking time, and upgrades the chained record of the link of first linking status according to testing result;
2) whether be dead chain with M2 as the link that detects second linking status blanking time, and upgrade the chained record of the link of second linking status according to testing result;
3) keep the chained record of the link of the 3rd linking status.
Comprised in second pre-defined rule according to the linking status of chained record change link and the rule of deletion or reservation link.Then the network equipment is based on first pre-defined rule, with the different time intervals dead chain being carried out in the link under first linking status and second linking status respectively detects, upgrade the chained record of the link of first linking status and second linking status, and it is constant to keep the chained record of the 3rd linking status.The network equipment according to each chained record, determines to carry out the linking status of change link based on second pre-defined rule, for example, is second linking status with first link-state change that links, or Remove Links, or keeps the constant operation of linking status.
Wherein, M1 is less than M2, then by detect the link under first linking status with the short time interval, can learn rapidly whether link is dead chain, when the link under first linking status is exceeded N1 detection for dead chain, then the linking status that will link is second linking status from first link-state change, make the network equipment to detect it with the long time interval, thereby reduce the operation burden of the network equipment, when the link under second linking status is exceeded N2 detection for dead chain, be the 3rd linking status with the linking status of this link from second link-state change, for the link under the 3rd linking status, think that it is the dead chain of chronicity, no longer detects.When link is in first linking status or second linking status following time, if repeated detection then should link from dead chain storehouse and delete for the chain of living.Wherein, M1, M2, N1, N2 are default value.
Again for example, linking status be divided into first linking status, second linking status ..., the n linking status.Correspondingly, comprise detection mode to the link under one or more linking status in first pre-defined rule.Comprised in second pre-defined rule according to the linking status of chained record change link and the rule of deletion or reservation link.Then the network equipment is by adopting different detection modes to the link under the different linking state, realize the effect that fast detecting and magnanimity detect, and according to chained record, link is placed suitable linking status, in order to pointed processing mode is adopted in different links.
Need to prove, above-mentioned for example only for the solution of the present invention is described better, but not limitation of the present invention, in fact, it should be appreciated by those skilled in the art that anyly to be divided into multiplely by the link in the chain storehouse of checkmating, and the link under this multiple linking status carried out pointed detection and processing, to realize the quick scheme that reaches the technique effect of magnanimity detection of dead chain, all should be within the scope of the present invention.
Fig. 2 is the flow chart that is used for carrying out based on classification the method for dead chain detection and processing of a preferred embodiment of the invention.The method that provides in the present embodiment also comprises step S1 and step S2.
In step S1, the network equipment obtains the corresponding link information undetermined of Internet resources of the detected visit that fails.Wherein, described Internet resources include but not limited to: 1) webpage; 2) audio frequency and video; 3) picture etc. all can have the Internet resources of chained address.Judge whether can successful access standard being described in detail with reference among the embodiment shown in Figure 1, comprise by reference at this, repeat no more.
The network equipment obtains described link information undetermined by following at least a mode:
1) obtains the corresponding link information of Internet resources of clicking the visit that fails of recording in the monitoring daily record;
When the user links by network equipment accesses network, the network equipment can be in several ways, for example, the javascript technology, whether monitor the corresponding Internet resources of this network linking can be by user's successful access, if the visit that fails, then the network equipment is recorded to this Internet resources corresponding link information to click and monitors in the daily record, with as link information undetermined;
2) network equipment is initiatively initiated visit to link, obtains described link information undetermined;
Particularly, the network equipment is to the web page interlinkage in the preset range, for example, web page interlinkage in the index database of search engine, or the web page interlinkage of the Internet resources that provide to the user of website etc., initiate initiatively visit, obtain testing result, and be to fail the web page interlinkage information of visit as link information undetermined with testing result.
For example, the network equipment initiatively to every day user's click volume be positioned at preceding 1000 link and initiate visit, whether can successful access to detect this preceding 1000 link, and link that can't successful access is as link information undetermined.
Need to prove, above-mentioned for example only for the present invention is described better, but not limitation of the present invention, those skilled in the art should understand that, any mode of obtaining described link information undetermined by record or active detecting, all should be within the scope of the present invention, and comprise by reference.
Then, in step S2, the network equipment upgrades link in the described dead chain storehouse according to described link information undetermined.
Particularly, the network equipment can add the whole links in the link information undetermined in the network equipment, perhaps, the network equipment is inquired about described dead chain storehouse, judging the link that whether has comprised in the described dead chain storehouse in this link information undetermined, and the link that will not be included in the link information undetermined in the described dead chain storehouse is added in the described dead chain storehouse.
Step S3 and step S4 being described in detail with reference among the embodiment shown in Figure 1, comprise at this by reference, repeat no more.
By the scheme that present embodiment provides, the network equipment can upgrade the link in the dead chain storehouse fast.
Fig. 3 is the method flow diagram that is used for carrying out based on classification dead chain detection and processing of another preferred embodiment of the present invention.Aforementioned with reference to the content among Fig. 1 and the described embodiment of Fig. 2, all comprise in the present embodiment by reference.Wherein, comprise step S11 among the step S1, comprise step S21 among the step S2, comprise step S31 and step S32 among the step S3, comprise step S41 among the step S4.In the present embodiment, linking status comprises interim linking status and long-term linking status.First pre-defined rule comprises also when described current link state is interim linking status whether poll detects described link is the rule of dead chain.
In step S11, the network equipment obtains the corresponding link information undetermined of Internet resources of the detected visit that fails in first predetermined interval.Wherein, the mode of obtaining described link information undetermined is described in detail in reference to the step S1 among the embodiment shown in Figure 2, comprises by reference at this, repeats no more.
In step S21, when should the corresponding link of link information undetermined being added in the described dead chain storehouse, the network equipment be set up the initial link record of this link, and the state that should link is set to interim linking status.
In step S31, the network equipment is based on first pre-defined rule, and poll detection linking status is whether the link of interim linking status is dead chain, and obtains the testing result of each poll.Wherein, detect link and whether be the mode of dead chain being described in detail with reference among the embodiment shown in Figure 1, comprise by reference at this, repeat no more.
Then, in step S32, the network equipment upgrades the chained record of this link according to the testing result that described each poll obtains.
Particularly, the network equipment whenever carries out once dead chain and detects, and is about to this time testing result and/or is updated to detection time in the chained record of respective links.
For example, for link Www.soopat.com, behind the every execution in step S31 of the network equipment, i.e. whether execution in step S32 is the testing result of dead chain with www.soopat.com and/or be updated to its detection time Www.soopat.comChained record in.
In step S41, the network equipment is based on described second pre-defined rule, and according to the chained record after the described renewal, carrying out the link-state change that will link is that long-term linking status is also correspondingly changed this chained record, or deletes the operation of this link.
Particularly, when chained record met following arbitrary condition, the interim link-state change that the network equipment will link was long-term linking status, and change link record correspondingly:
1) N continuous is detected as dead chain 3 times, and wherein N3 is preset threshold value;
For example, if N3=3 time, link Www.soopat.comChained record in be recorded to its continuous 3 times and be detected as dead chain, then the network equipment will Www.soopat.comLinking status be long-term linking status from interim link-state change, and in chained record the record linking status the change time;
2) time that is in interim linking status surpasses M3, and surpasses N4 time and be detected as dead chain, and wherein, M3, N4 are preset threshold value;
For example, if M3=8 days, N4=4 time, the current time that the network equipment obtains is 00:00 on November 1, link Www.soopat.comChained record in comprise following information: 1) linking status of this link from October 24 00:00 be interim linking status; 2) the dead chain testing result of this link is " chain of living, dead chain, the chain of living, dead chain, dead chain, dead chain, dead chain, dead chain ", and then the network equipment will Www.soopat.comLinking status be long-term linking status from interim link-state change, and in chained record the record linking status the change time be 00:00 on November 1.
When chained record met following arbitrary condition, the network equipment was deleted this link:
1) this link N continuous is detected as dead chain 5 times, and wherein N5 is preset threshold value;
For example, if N5=4 time, link Www.soopat.comChained record in be recorded to its continuous 4 times and be detected as chain alive, then the network equipment will Www.soopat.comFrom dead chain storehouse, delete;
2) this link time of being in interim linking status surpasses M4, and surpasses N6 time and be detected as chain alive, and wherein, M4, N6 are preset threshold value;
For example, if M4=8 days, N6=4 time, the November 1 current time that the network equipment obtains, link Www.soopat.comChained record in comprise following information: 1) linking status of this link from October 24 00:00 be interim linking status; 2) the dead chain testing result of this link is " chain of living, dead chain, the chain of living, dead chain, the chain of living, the chain of living, the chain of living, the chain of living ", and then the network equipment will Www.soopat.comFrom dead chain storehouse, delete.
3) this link is not included in index database or the database of reaching the standard grade that the Internet resources link is provided to the user for storage;
The network equipment mates linking in link and index database or the database of reaching the standard grade, if can't mate, then deletes this link.
The chained record that does not meet other linking status when chained record requires and does not meet the chained record requirement of Remove Links, the network equipment is judged the chained record of this link, meet the chained record requirement of the state at this link place, then the network equipment keep this link and chained record constant, or in the chained record of this link the record this judged result.
As a preferred version of the present invention, step S4 also comprises step S42 (not shown).Linking status also comprises the long history linking status.First pre-defined rule also comprises when described current link state is long-term linking status, be used for keeping this link chained record rule keep rule.
Keep rule based on this, it is constant that the network equipment is kept the chained record that is in the link under the long-term linking status.
In step S42, the network equipment is based on described second pre-defined rule, according to the described chained record of keeping, carrying out the link-state change that will link is that the long history linking status is also correspondingly changed the chained record of this link, or deletes the operation of this link.
Particularly, when chained record met following arbitrary condition, the long-term link-state change that the network equipment will link was the long history linking status, and change link record correspondingly:
1) link time of being in long-term linking status surpasses M5, and wherein M5 is predetermined threshold value;
For example, M5=30 days, the November 1 current time that the network equipment obtains, link Www.soopat.comChained record in comprise following information: the linking status of this link from September 30 00:00 be interim linking status, then the network equipment link-state change that will link is the long history linking status, and in chained record the change time of record linking status be 00:00 on November 1.
2) arrive the predetermined link-state change time;
For example, the predetermined link-state change time is 00:00 on November 1, then the network equipment changes to the long history linking status with the linking status of the link of all long-term linking status, and in chained record the record linking status the change time be 00:00 on November 1.
When chained record met following condition, the network equipment was deleted this link:
-this link is not included in index database or the database of reaching the standard grade that the Internet resources link is provided to the user for storage;
The network equipment mates linking in link and index database or the database of reaching the standard grade, if can't mate, then deletes this link.
Fig. 4 is the method flow diagram that is used for carrying out based on classification dead chain detection and processing of another preferred embodiment of the present invention.Aforementioned with reference to the content among the described embodiment of Fig. 3, all comprise in the present embodiment by reference.Wherein, step S3 also comprises step S33, and step S4 also comprises step S43.In the present embodiment, linking status comprises interim linking status, long-term linking status, long history linking status and permalink state.First pre-defined rule comprises that also when detecting a plurality of linking status be the link of long history linking status when identical, merges this a plurality of links, and correspondingly upgrades the linking status of the link after merging.
In step S33, based on first pre-defined rule, be that the link of long history linking status is identical when the network equipment detects a plurality of linking status, be about to this a plurality of identical links and merge, and the number of links that record merges in the chained record after merging.
For example, comprise that three linking status are the link of long history linking status in the dead chain storehouse Www.soopat.com, the then redundant link of network equipment deletion only keeps a link in dead chain storehouse Www.soopat.com, and in its chained record, be recorded under the long history linking status, this links merged number of links is four, namely this is linked at and occurred under the long history linking status four times.
In step S43, the network equipment is based on described second pre-defined rule, and according to the chained record after the described renewal, carrying out the link-state change that will link is that the permalink state is also correspondingly changed the chained record of this link, or deletes the operation of this link.
Particularly, when chained record met following condition, the network equipment was the permalink state with the long history link-state change of link, and change link record correspondingly:
-this links merged quantity greater than N7 time, and wherein, N7 is predetermined value;
For example, N7=3, link Www.soopat.comChained record in comprise following information: this links merged link number is 4, and then the network equipment link-state change that will link is the permalink state, and in chained record change time of record linking status.
Need to prove that this merging number of links is accumulated, for example, for link Www.soopat.comTwo links have been merged in the merging process for the first time, it is 2 that the network equipment upgrade to merge number at chained record, has merged two links again in the merging process for the second time, and then to upgrade the merging number in chained record be 3 to the network equipment, be the network equipment in merging first with the number of links that merges as merging number, and in follow-up merging, the link number that merges is subtracted one, and count addition with the merging of current record, upgrade the merging number, in fact this merging number has reflected link Www.soopat.comLinking status appear at number of times under the long history linking status.
When chained record met following condition, the network equipment was deleted this link:
-this link is not included in index database or the database of reaching the standard grade that the Internet resources link is provided to the user for storage;
The network equipment mates linking in link and index database or the database of reaching the standard grade, if can't mate, then deletes this link.
The link that is under the permalink state will be used as nonvolatil dead chain, be kept.
In the present embodiment, linking status is divided into interim linking status, long-term linking status, long history linking status and permalink state, for the link in the detected link information undetermined, in time add till death in the chain storehouse, and to set linking status be interim linking status, to improve the real-time that comes into force that dead chain detects; Be the link of interim linking status for linking status, carrying out poll detects, select the metastable link of dead chain situation, and be long-term linking status with the link-state change of the metastable link of damned chain situation, no longer further poll, to reduce the resource consumption of the network equipment, the dead chain that makes the network equipment can tackle magnanimity detects; And the link number of times of long history linking status also appears in the present invention by record, with will be repeatedly linking status to be modified serve as that the link of long-term linking status is added in the permanent dead chain storehouse, further reduce the resource consumption of the network equipment.
Fig. 5 is the method flow diagram that is used for carrying out based on classification dead chain detection and processing of another preferred embodiment of the present invention.
In step S 12, the network equipment obtains the corresponding link information undetermined of Internet resources of the detected visit that fails in second predetermined interval.Wherein, the mode of obtaining described link information undetermined is described in detail in reference to the step S1 among the embodiment shown in Figure 2, comprises by reference at this, repeats no more.
In step S22, the network equipment should add in the described dead chain storehouse in the corresponding link of link information undetermined, set up the initial link record of this link, and the state that should link is set to linking status in short-term.Wherein, the network equipment can directly add the corresponding link of this link information undetermined in the described dead chain storehouse, or after it is further detected processing, selects to be added in the described dead chain storehouse again.The described chained record that meets accident includes but not limited to that this link is added the information such as time in the chain storehouse till death.
Preferably, also comprise step S221 (not shown) and step S222 (not shown) among the step S22.
In step S221, the network equipment carries out secondary detection to detected link information undetermined in described second predetermined interval, to obtain testing result.Wherein, dead chain detection mode is described in detail in reference to the step S1 among the embodiment shown in Figure 1, comprises by reference at this, repeats no more.
In step S222, the network equipment is that the corresponding link of link information undetermined of dead chain adds in the described dead chain storehouse with testing result, set up the initial link record of this link, and the state that should link is set to linking status in short-term.
Preferably, comprise further also among the step S222 that it is that click volume is that the link of preceding N position is updated to the step in the dead chain storehouse of described level in short-term in the link information undetermined of dead chain that the network equipment is selected testing result, wherein, N is predetermined threshold, this click volume can be the click total amount that is recorded to, and also can be the click volume in a period of time.
For example, N=1000, then to select click volume be preceding 1000 link to the network equipment, adds till death in the chain storehouse.Need to prove, after the network equipment need not all link informations undetermined are detected, select again, as long as in step S221, the network equipment detects from high to low according to click volume, after then obtaining N dead chain result, the network equipment namely need not other link informations undetermined to be detected again.
The link that is under the linking status in short-term will regularly be deleted from dead chain storehouse.
Second predetermined interval in the present embodiment is shorter than first predetermined interval in the previous embodiment, thereby the scheme that present embodiment provides can further improve the detection speed of dead chain and the real-time that comes into force.
As an optimal way of the present invention, the present invention comprises that also the network equipment shields the step of all or part of link in the described dead chain storehouse.
For example, the network equipment whole links shieldings in the chain storehouse of can checkmating, perhaps, for previous embodiment, the network equipment only shields in short-term the link under dead chain state, interim dead chain state, long-term dead chain state and the permanent dead chain state, and does not shield link under the long-term interim dead chain state etc.
Need to prove, behind the dead chain under each linking status of method acquisition provided by the invention, also can be used for other side, for example, the dead chain rate of statistics different web sites etc., and the effect that is not limited only to shield.
Fig. 6 is the network equipment structural representation that is used for carrying out based on classification dead chain detection and processing of one aspect of the invention.Wherein, the described network equipment includes but not limited to: 1) a plurality of webserver collection; 2) distributed network equipment; 3) based on set of computers of the cloud that is constituted by a large amount of computers or the webserver of cloud computing (Cloud Computing) etc.Wherein, cloud computing is a kind of of Distributed Calculation, a super virtual machine of being made up of the loosely-coupled computer collection of a group.The present invention also comprises dead chain storehouse, and described dead chain storehouse can be included in the network equipment, or with network equipment physical separation but link to each other by network service, and described dead chain storehouse can be as a whole, or is divided into a plurality of physical separation but the part that connects by network service.Described network includes but not limited to: the Internet, wide area network, metropolitan area network, local area network (LAN), VPN network, wireless self-organization network (Ad Hoc network) etc.Link in the dead chain storehouse can be upgraded by artificial or the network equipment.
Record updating device 3 is based on first pre-defined rule, in conjunction with the current link state that links in the dead chain storehouse, to upgrade or to keep the chained record of this link.Wherein, for a link, its linking status can by with this information that is associated that links, this link determining positions of storing maybe, for example, can link the identification information that is associated, can represent linking status with this and learn the linking status that this links by reading, perhaps, set up the chained library of different classifications in described dead chain storehouse, each chained library is used for the link of storage different linking state, then link residing chained library by this, can obtain the state of this link; Described chained record includes but not limited to following at least one:
1) this is linked at the testing result in each dead chain detection;
2) detection time of the each dead chain detection of this link;
3) the current link state of this link and url history state;
4) this link adds the time in the dead chain storehouse;
5) time of the link-state change of this link etc.
Particularly, the mode that has comprised the chained record of handling the link under one or more linking status in first pre-defined rule, when recording updating device 3 according to first pre-defined rule, in the time of need carrying out respective handling to the chained record of the link of certain linking status, the link of searching this linking status from dead chain storehouse is carried out respective handling to its chained record; For the link of the linking status that does not relate in first pre-defined rule, it is constant that record updating device 3 is kept its chained record.
Wherein, record updating device 3 mode that obtains the link of required linking status from dead chain storehouse includes but not limited to:
1) by the identification information of query link, judges that whether this link is the link of required linking status, obtains corresponding link;
2) chained library at the link place of the required linking status of visit storage obtains corresponding link.
Record updating device 3 includes but not limited to following mode according to first pre-defined rule to the processing mode that chained record carries out respective handling:
The link of i is carried out dead chain and is detected, and with testing result and/or be updated to detection time in the chained record of this this link;
For example, based on first pre-defined rule, record updating device 3 need carry out dead chain to the link under the linking status and detect, and then obtains the link of this linking status Www.soopat.comAfter, 3 pairs of these links of record updating device are carried out dead chain and are detected, and testing result and/or detection time are updated to Www.soopat.comChained record in.
Wherein, described dead chain detects and includes but not limited to following multiple detection mode:
1) for example, record updating device 3 is for link Www.soopat.comCarry out one-time detection, and according to this time testing result, judge whether this link is dead chain;
2) again for example, when record updating device 3 is lower than a predetermined low discharge threshold value in user's visit capacity, to link Www.soopat.comDetect, judge that it whether can successful access, if not, judge that then this is linked as dead chain, if then record updating device 3 when user's visit capacity is higher than a predetermined high flow capacity threshold value, to link Www.soopat.comCarry out one-time detection again, judge that it whether can successful access, if, judge that then this is linked as chain alive, if not, judge that then this is linked as dead chain;
3) more for example, according to statistics, obtain the section and slack hours section in rush hour of user's visit, then when the slack hours section, 3 pairs of links of record updating device Www.soopat.comDetect, judge that it whether can successful access, if not, judge that then this is linked as dead chain, if then record updating device 3 in rush hour during section, to link Www.soopat.comCarry out one-time detection again, judge that it whether can successful access, if, judge that then this is linked as chain alive, if not, judge that then this is linked as dead chain.
Need to prove that record updating device 3 judges that the factor that link whether can successful access comprises following at least one:
1) whether the corresponding webpage of link can show;
2) whether the corresponding webpage of link can show full content;
3) whether the corresponding multimedia file of link can visit etc.
What need further specify is, above-mentioned for example only for dead chain detection mode is described better, but not the restriction that the present invention is done, in fact, it should be appreciated by those skilled in the art that anyly by link is detected, judge whether it is the mode of dead chain, all should be within the scope of the present invention, and comprise by reference.
Ii carries out union operation to a plurality of identical links under the same linking status, and records the number of links that merges in the chained record of the link after merging;
Particularly, may record a plurality of identical links in the dead chain storehouse, then based on first pre-defined rule, the same chain under 3 pairs of same linking status of record updating device taps into row and merges, and records the number of links that merges in the chained record of the link after merging;
For example, in dead chain storehouse, for link Www.soopat.com, it can be added in the different moment till death in the chain storehouse, and relevant information difference when at every turn adding, or is stored in different positions, then can have a plurality of links simultaneously in the dead chain storehouse Www.soopat.com, record updating device 3 is based on first pre-defined rule, under the same linking status Www.soopat.comMerge, and Www.soopat.comChained record in record the number of links that merges.
Need to prove that though union operation relates to the operation of the redundant link of deletion, its essence is still the chained record of a link of change, therefore, still and with it be considered as the operation that the chained record to a link carries out.
Iii keep the link and chained record constant;
Concrete, comprised in first pre-defined rule when linking and be in predetermined following time of one or more linking status, keep the constant rule of this link and chained record thereof, then record updating device 3 according to this rule, it is constant to keep this chained record.Perhaps, for the linking status that does not relate in first pre-defined rule, link and chained record thereof that record updating device 3 acquiescences are kept under this linking status are constant.
What need further specify is, above-mentionedly for example only come mode that chained record is handled for illustrating better among the present invention based on first pre-defined rule, but not the restriction that the present invention is done, in fact, it should be appreciated by those skilled in the art that anyly based on first pre-defined rule, come link and chained record thereof are carried out the mode of respective handling, all should be within the scope of the present invention, and comprise by reference.
Processing unit 4, comes corresponding operation is carried out in this link and/or its chained record according to the described chained record that upgrades the back or keep based on second pre-defined rule.
Particularly, second pre-defined rule comprises following at least one:
I meets the chained record requirement of the link of other linking status when the chained record of described link, and then the link-state change that will link is these other linking status, and correspondingly changes the chained record of this link;
Particularly, processing unit 4 is according to second pre-defined rule, detect the chained record of the link under each linking status, judge the chained record of the link under the linking status when processing unit 4, when meeting the requiring of chained record under other linking status, the link-state change that processing unit 4 will link is these other linking status, and correspondingly records in the chained record of this link the change time of linking status.
For example, for link Www.soopat.com, processing unit 4 detects the nearest five times dead chain testing result that comprises in its chained record and is " chain of living, dead chain, the chain of living, dead chain, dead chain ", and then its number of times that is detected as dead chain surpasses twice, meets the chained record requirement of other linking status, then will Www.soopat.comLink-state change be satisfactory other states.
II then deletes this link when described link or its chained record meet the chained record requirement of Remove Links;
Concrete, processing unit 4 detects the chained record of the link under each linking status according to second pre-defined rule, judges the chained record of the link under the linking status when processing unit 4, when meeting the requiring of chained record of Remove Links, deletes this link.
For example, for link Www.soopat.com, processing unit 4 detects and does not comprise the link that is complementary with it in the index database, then deletes this link.
III then upgrades or keeps the chained record of this link when the chained record of described link meets the chained record requirement of the state at this link place.
Concrete, processing unit 4 is according to second pre-defined rule, detect the chained record of the link under each linking status, judge the chained record of the link under the linking status when processing unit 4, meet the chained record requirement of the state at this link place, it is constant then to keep this link and chained record thereof, or records this judged result in the chained record of this link.
Need to prove that record updating device 3 there is no sequencing with processing unit 4 performed operating accordingly separately.
In the present embodiment, linking status comprises multiple dividing mode, but to every kind of dividing mode, all can carry out correspondingly link detection and processing with processing unit 4 by record updating device 3.
For example, linking status is divided into first linking status, second linking status and the 3rd linking status.Correspondingly, comprise following rule in first pre-defined rule:
1) whether is dead chain with M1 as the link that detects first linking status blanking time, and upgrades the chained record of the link of first linking status according to testing result;
2) whether be dead chain with M2 as the link that detects second linking status blanking time, and upgrade the chained record of the link of second linking status according to testing result;
3) keep the chained record of the link of the 3rd linking status.
Comprised in second pre-defined rule according to the linking status of chained record change link and the rule of deletion or reservation link.Then record updating device 3 based on first pre-defined rule, with the different time intervals dead chain being carried out in the link under first linking status and second linking status respectively detects, upgrade the chained record of the link of first linking status and second linking status, and it is constant to keep the chained record of the 3rd linking status.Processing unit 4 according to each chained record, determines to carry out the linking status of change link based on second pre-defined rule, for example, is second linking status with first link-state change that links, or Remove Links, or keeps the constant operation of linking status.
Wherein, M1 is less than M2, then record updating device 3 by detect the link under first linking status with the short time interval, can learn rapidly whether link is dead chain, when the link under first linking status is exceeded N1 detection for dead chain, the linking status that processing unit 4 will link is second linking status from first link-state change, make record updating device 3 to detect it with the long time interval, thereby reduce the operation burden of the network equipment, when the link under second linking status is exceeded N2 detection for dead chain, the linking status that processing unit 4 will link is the 3rd linking status from second link-state change, for the link under the 3rd linking status, think that it is the dead chain of chronicity, no longer detects.When link is in first linking status or second linking status following time, if repeated detection then should link from dead chain storehouse and delete for the chain of living.Wherein, M1, M2, N1, N2 are default value.
Again for example, linking status be divided into first linking status, second linking status ..., the n linking status.Correspondingly, comprise detection mode to the link under one or more linking status in first pre-defined rule.Comprised in second pre-defined rule according to the linking status of chained record change link and the rule of deletion or reservation link.Then record updating device 3 by different detection modes is adopted in the link under the different linking state, realize the effect that fast detecting and magnanimity detect, processing unit 4 is according to chained record, link is placed suitable linking status, in order to pointed processing mode is adopted in different links.
Need to prove, above-mentioned for example only for the solution of the present invention is described better, but not limitation of the present invention, in fact, it should be appreciated by those skilled in the art that anyly to be divided into multiplely by the link in the chain storehouse of checkmating, and the link under this multiple linking status carried out pointed detection and processing, to realize the quick scheme that reaches the technique effect of magnanimity detection of dead chain, all should be within the scope of the present invention.
Fig. 7 is the network equipment structural representation that is used for carrying out based on classification dead chain detection and processing of a preferred embodiment of the invention.In the present embodiment, the network equipment comprises deriving means 1, link updating device 2, record updating device 3 and processing unit 4.
Deriving means 1 obtains the corresponding link information undetermined of Internet resources of the detected visit that fails.Wherein, described Internet resources include but not limited to: 1) webpage; 2) audio frequency and video; 3) picture etc. all can have the Internet resources of chained address.Judge whether can successful access standard being described in detail with reference among the embodiment shown in Figure 1, comprise by reference at this, repeat no more.
Deriving means 1 obtains described link information undetermined by following at least a mode:
1) obtains the corresponding link information of Internet resources of clicking the visit that fails of recording in the monitoring daily record;
When the user links by network equipment accesses network, the network equipment can be in several ways, for example, the javascript technology, whether can by user successful access, if fail visit, then the network equipment is recorded to this Internet resources corresponding link information and clicks in the monitoring daily record if monitoring the corresponding Internet resources of this network linking, deriving means 1 obtains the link of clicking the visit that fails of recording in the monitoring daily record, as link information undetermined;
2) net deriving means 1 is initiatively initiated visit to link, obtains described link information undetermined;
Particularly, web page interlinkage in 1 pair of preset range of deriving means, for example, web page interlinkage in the index database of search engine, or the web page interlinkage of the Internet resources that provide to the user of website etc., initiate initiatively visit, obtain testing result, and be to fail the web page interlinkage information of visit as link information undetermined with testing result.
For example, deriving means 1 initiatively is positioned at preceding 1000 link to user's click volume and initiates visit, whether can successful access to detect this preceding 1000 link, and link that can't successful access is as link information undetermined.
Need to prove, above-mentioned for example only for the present invention is described better, but not limitation of the present invention, those skilled in the art should understand that, any mode of obtaining described link information undetermined by record or active detecting, all should be within the scope of the present invention, and comprise by reference.
Link updating device 2 upgrades link in the described dead chain storehouse according to described link information undetermined.
Particularly, link updating device 2 can add the whole links in the link information undetermined in the network equipment, perhaps, the described dead chain of link updating device 2 inquiries storehouse, judging the link that whether has comprised in the described dead chain storehouse in this link information undetermined, and the link that will not be included in the link information undetermined in the described dead chain storehouse is added in the described dead chain storehouse.
Record updating device 3 and processing unit 4 being described in detail with reference among the embodiment shown in Figure 6, comprise at this by reference, repeat no more.
By the scheme that present embodiment provides, the network equipment can upgrade the link in the dead chain storehouse fast.
Fig. 8 is the network equipment structural representation that is used for carrying out based on classification dead chain detection and processing of another preferred embodiment of the present invention.Aforementioned with reference to the content among Fig. 6 and the described embodiment of Fig. 7, all comprise in the present embodiment by reference.In the present embodiment, deriving means 1 comprises the first sub-deriving means 11, link updating device 2 comprises the first sublink updating device 21, and record updating device 3 comprises first checkout gear 31 and the first subrecord updating device 32, and processing unit 4 comprises sub-processing unit 41.In the present embodiment, linking status comprises interim linking status and long-term linking status.First pre-defined rule comprises also when described current link state is interim linking status whether poll detects described link is the rule of dead chain.
The first sub-deriving means 11 obtains the corresponding link information undetermined of Internet resources of the detected visit that fails in first predetermined interval.Wherein, obtain the mode of described link information undetermined being described in detail with reference among the embodiment shown in Figure 7, comprise by reference at this, repeat no more.
The first sublink updating device 21 should be added in the described dead chain storehouse in the corresponding link of link information undetermined, set up the initial link record of this link, and the state that should link is set to interim linking status.
First checkout gear 31 is based on first pre-defined rule, and poll detection linking status is whether the link of interim linking status is dead chain, and obtains the testing result of each poll.Wherein, detect link and whether be the mode of dead chain being described in detail with reference among the embodiment shown in Figure 6, comprise by reference at this, repeat no more.
The first subrecord updating device 32 upgrades the chained record of this link according to the testing result that described each poll obtains.
Particularly, first checkout gear 31 whenever carries out once dead chain and detects, and the first subrecord updating device 32 is about to this time testing result and/or is updated to detection time in the chained record of respective links.
For example, for link Www.soopat.com, after first checkout gear 31 detected this link at every turn, whether the first subrecord updating device 32 was about to this time www.soopat.com and is the testing result of dead chain and/or is updated to its detection time Www.soopat.comChained record in.
The first sub-processing unit 41 is based on described second pre-defined rule, and according to the chained record after the described renewal, carrying out the link-state change that will link is that long-term linking status is also correspondingly changed this chained record, or deletes the operation of this link.
Particularly, when chained record met following arbitrary condition, the interim link-state change that the first sub-processing unit 41 will link was long-term linking status, and change link record correspondingly:
1) N continuous is detected as dead chain 3 times, and wherein N3 is preset threshold value;
For example, if N3=3 time, link Www.soopat.comChained record in be recorded to its continuous 3 times and be detected as dead chain, then the first sub-processing unit 41 will Www.soopat.comLinking status be long-term linking status from interim link-state change, and in chained record the record linking status the change time;
2) time that is in interim linking status surpasses M3, and surpasses N4 time and be detected as dead chain, and wherein, M3, N4 are preset threshold value;
For example, if M3=8 days, N4=4 time, the current time that the network equipment obtains is 00:00 on November 1, link Www.soopat.comChained record in comprise following information: 1) linking status of this link from October 24 00:00 be interim linking status; 2) the dead chain testing result of this link is " chain of living, dead chain, the chain of living, dead chain, dead chain, dead chain, dead chain, dead chain ", and then the first sub-processing unit 41 will Www.soopat.comLinking status be long-term linking status from interim link-state change, and in chained record the record linking status the change time be 00:00 on November 1.
When chained record meets following arbitrary condition, first sub-processing unit 41 these links of deletion:
1) this link N continuous is detected as dead chain 5 times, and wherein N5 is preset threshold value;
For example, if N5=4 time, link Www.soopat.comChained record in be recorded to its continuous 4 times and be detected as chain alive, then the first sub-processing unit 41 will Www.soopat.comFrom dead chain storehouse, delete;
2) this link time of being in interim linking status surpasses M4, and surpasses N6 time and be detected as chain alive, and wherein, M4, N6 are preset threshold value;
For example, if M4=8 days, N6=4 time, the November 1 current time that the network equipment obtains, link Www.soopat.comChained record in comprise following information: 1) linking status of this link from October 24 00:00 be interim linking status; 2) the dead chain testing result of this link is " chain of living, dead chain, the chain of living, dead chain, the chain of living, the chain of living, the chain of living, the chain of living ", and then the first sub-processing unit 41 will Www.soopat.comFrom dead chain storehouse, delete.
3) this link is not included in index database or the database of reaching the standard grade that the Internet resources link is provided to the user for storage;
The first sub-processing unit 41 mates linking in link and index database or the database of reaching the standard grade, if can't mate, then deletes this link.
The chained record that does not meet other linking status when chained record requires and does not meet the chained record requirement of Remove Links, the first sub-processing unit 41 is judged the chained record of this link, meet the chained record requirement of the state at this link place, then the first sub-processing unit 41 keep this link and chained record constant, or in the chained record of this link the record this judged result.
As a preferred version of the present invention, processing unit 4 also comprises the second sub-processing unit (not shown).Linking status also comprises the long history linking status.First pre-defined rule also comprises when described current link state is long-term linking status, be used for keeping this link chained record rule keep rule.
Keep rule based on this, it is constant that record updating device 3 is kept the chained record that is in the link under the long-term linking status.
The second sub-processing unit is based on described second pre-defined rule, and according to the described chained record of keeping, carrying out the link-state change that will link is that the long history linking status is also correspondingly changed the chained record of this link, or deletes the operation of this link.
Particularly, when chained record met following arbitrary condition, the long-term link-state change that the second sub-processing unit will link was the long history linking status, and change link record correspondingly:
1) link time of being in long-term linking status surpasses M5, and wherein M5 is predetermined threshold value;
For example, M5=30 days, the November 1 current time that the network equipment obtains, link Www.soopat.comChained record in comprise following information: the linking status of this link from September 30 00:00 be interim linking status, then the link-state change that will link of the second sub-processing unit is the long history linking status, and in chained record the change time of record linking status be 00:00 on November 1.
2) arrive the predetermined link-state change time;
For example, the predetermined link-state change time is 00:00 on November 1, then the linking status of the link of second sub-all long-term linking status of processing unit changes to the long history linking status, and in chained record the record linking status the change time be 00:00 on November 1.
When chained record met following condition, the second sub-processing unit was deleted this link:
-this link is not included in index database or the database of reaching the standard grade that the Internet resources link is provided to the user for storage;
The second sub-processing unit mates linking in link and index database or the database of reaching the standard grade, if can't mate, then deletes this link.
As a preferred version of the present invention, processing unit 4 also comprises the 3rd sub-processing unit (not shown).Linking status also comprises the permalink state.First pre-defined rule comprises that also when detecting a plurality of linking status be the link of long history linking status when identical, merges this a plurality of links, and correspondingly upgrades the linking status of the link after merging.
Based on first pre-defined rule, be that the link of long history linking status is identical when record updating device 3 detects a plurality of linking status, be about to this a plurality of identical links and merge, and the number of links that record merges in the chained record after merging.
For example, comprise that three linking status are the link of long history linking status in the dead chain storehouse Www.soopat.com, then record the redundant link of updating device 3 deletions, only in dead chain storehouse, keep a link Www.soopat.com, and in its chained record, be recorded under the long history linking status, this links merged number of links is four, namely this is linked at and occurred under the long history linking status four times.
The 3rd sub-processing unit is based on described second pre-defined rule, and according to the chained record after the described renewal, carrying out the link-state change that will link is that the permalink state is also correspondingly changed the chained record of this link, or deletes the operation of this link.
Particularly, when chained record met following condition, the 3rd sub-processing unit was the permalink state with the long history link-state change of link, and change link record correspondingly:
-this links merged quantity greater than N7 time, and wherein, N7 is predetermined value;
For example, N7=3, link Www.soopat.comChained record in comprise following information: this links merged link number is 4, and then the link-state change that will link of the 3rd sub-processing unit is the permalink state, and in chained record change time of record linking status.
Need to prove that this merging number of links is accumulated, for example, for link Www.soopat.comTwo links have been merged in the merging process for the first time, it is 2 that record updating device 3 upgrade to merge number at chained record, has merged two links again in the merging process for the second time, and then recording updating device 3, to upgrade the merging number in chained record be 3, namely record updating device 3 in merging first with the number of links that merges as merging number, and in follow-up merging, the link number that merges is subtracted one, and count addition with the merging of current record, upgrade the merging number, in fact this merging number has reflected link Www.soopat.comLinking status appear at number of times under the long history linking status.
When chained record met following condition, the 3rd sub-processing unit was deleted this link:
-this link is not included in index database or the database of reaching the standard grade that the Internet resources link is provided to the user for storage;
The 3rd sub-processing unit mates linking in link and index database or the database of reaching the standard grade, if can't mate, then deletes this link.
The link that is under the permalink state will be used as nonvolatil dead chain, be kept.
In the present embodiment, linking status is divided into interim linking status, long-term linking status, long history linking status and permalink state, for the link in the detected link information undetermined, in time add till death in the chain storehouse, and to set linking status be interim linking status, to improve the real-time that comes into force that dead chain detects; Be the link of interim linking status for linking status, carrying out poll detects, select the metastable link of dead chain situation, and be long-term linking status with the link-state change of the metastable link of damned chain situation, no longer further poll, to reduce the resource consumption of the network equipment, the dead chain that makes the network equipment can tackle magnanimity detects; And the link number of times of long history linking status also appears in the present invention by record, with will be repeatedly linking status to be modified serve as that the link of long-term linking status is added in the permanent dead chain storehouse, further reduce the resource consumption of the network equipment.
Fig. 9 is the dead chain detection of the linking status in short-term of another preferred embodiment of the present invention and the network equipment structural representation of handling.In the present embodiment, deriving means 1 comprises that also the second sub-deriving means 12, link updating device also comprise the second sublink updating device 22.
The second sub-deriving means 12 obtains the corresponding link information undetermined of Internet resources of the detected visit that fails in second predetermined interval.Wherein, obtain the mode of described link information undetermined being described in detail with reference among the embodiment shown in Figure 7, comprise by reference at this, repeat no more.
The second sublink updating device 22 should add in the described dead chain storehouse in the corresponding link of link information undetermined, set up the initial link record of this link, and the state that should link is set to linking status in short-term.Wherein, the second sublink updating device 22 can directly add the corresponding link of this link information undetermined in the described dead chain storehouse, or after it is further detected processing, selects to be added in the described dead chain storehouse again.The described chained record that meets accident includes but not limited to that this link is added the information such as time in the chain storehouse till death.
As a preferred version of the present invention, also comprise the second checkout gear (not shown) and the 3rd sublink updating device (not shown) in the second sublink updating device 22.
Second checkout gear carries out secondary detection to detected link information undetermined in described second predetermined interval, to obtain testing result.Wherein, dead chain detection mode comprises at this by reference being described in detail with reference among the embodiment shown in Figure 6, repeats no more.
The 3rd sublink updating device is that the corresponding link of link information undetermined of dead chain adds in the described dead chain storehouse with testing result, set up the initial link record of this link, and the state that should link is set to linking status in short-term.
As a preferred version of the present invention, the 3rd sublink updating device also further comprises choice device, and it is that click volume is that the link of preceding N position is updated in the dead chain storehouse of described level in short-term in the link information undetermined of dead chain that choice device is selected testing result.Wherein, N is predetermined threshold, and this click volume can be the click total amount that is recorded to, and also can be the click volume in a period of time.
For example, N=1000, then to select click volume be preceding 1000 link to choice device, adds till death in the chain storehouse.Need to prove, after choice device need not to wait for that second checkout gear detects all link informations undetermined, select again, as long as second checkout gear detects from high to low according to click volume, after then choice device obtained N dead chain result, second checkout gear namely need not other link informations undetermined to be detected again.
The link that is under the linking status in short-term will regularly be deleted from dead chain storehouse.
Second predetermined interval in the present embodiment is shorter than first predetermined interval in the previous embodiment, thereby the scheme that present embodiment provides can further improve the detection speed of dead chain and the real-time that comes into force.
As an optimal way of the present invention, the network equipment also comprises screening arrangement, and screening arrangement shields all or part of link in the described dead chain storehouse.
For example, the screening arrangement whole links shieldings in the chain storehouse of can checkmating, perhaps, for previous embodiment, screening arrangement only shields in short-term the link under dead chain state, interim dead chain state, long-term dead chain state and the permanent dead chain state, and does not shield link under the long-term interim dead chain state etc.
Need to prove, behind the dead chain under each linking status of method acquisition provided by the invention, also can be used for other side, for example, the dead chain rate of statistics different web sites etc., and the effect that is not limited only to shield.
To those skilled in the art, obviously the invention is not restricted to the details of above-mentioned one exemplary embodiment, and under the situation that does not deviate from spirit of the present invention or essential characteristic, can realize the present invention with other concrete form.Therefore, no matter from which point, all should regard embodiment as exemplary, and be nonrestrictive, scope of the present invention is limited by claims rather than above-mentioned explanation, therefore is intended to be included in the present invention dropping on the implication that is equal to important document of claim and all changes in the scope.Any Reference numeral in the claim should be considered as limit related claim.In addition, obviously other unit or step do not got rid of in " comprising " word, and odd number is not got rid of plural number.A plurality of unit of stating in system's claim or device also can be realized by software or hardware by a unit or device.The first, the second word such as grade is used for representing title, and does not represent any specific order.

Claims (24)

1. one kind is used for carrying out the method that dead chain detects and handles based on classification in the network equipment, and wherein, this method may further comprise the steps:
C is based on first pre-defined rule, and the current link state in conjunction with linking in the dead chain storehouse to upgrade or to keep the chained record of this link, wherein, has comprised the processing mode of the chained record of the link under one or more linking status in described first pre-defined rule;
D, comes corresponding operation is carried out in this link and/or its chained record according to the described chained record that upgrades the back or keep based on second pre-defined rule, and wherein, described second pre-defined rule comprises following at least one:
-when the chained record of described link meets the chained record requirement of the link of other linking status, then the link-state change that will link is these other linking status, and correspondingly changes the chained record of this link;
-when described link or its chained record meet the chained record requirement of Remove Links, then delete this link;
-when the chained record of described link meets the chained record requirement of the state at this link place, then upgrade or keep the chained record of this link.
2. method according to claim 1, wherein, this method is further comprising the steps of:
A obtains the corresponding link information undetermined of Internet resources of the detected visit that fails;
B upgrades link in the described dead chain storehouse according to described link information undetermined.
3. method according to claim 2, wherein, described linking status comprises interim linking status, described step a is further comprising the steps of:
-obtain the corresponding link information undetermined of Internet resources of the detected visit that fails in first predetermined interval;
Described step b is further comprising the steps of:
-should the corresponding link of link information undetermined add in the described dead chain storehouse, set up the initial link record of this link, and the state that should link is set to interim linking status.
4. method according to claim 3, wherein, described first pre-defined rule also comprises following rule:
-when described current link state was interim linking status, whether poll detects described link was dead chain;
Wherein, described step c is further comprising the steps of:
Whether-poll detects this link is dead chain, and obtains the testing result of each poll;
-according to the testing result that described each poll obtains, upgrade the chained record of this link.
5. method according to claim 4, wherein, described linking status also comprises long-term linking status, described steps d is further comprising the steps of:
-based on described second pre-defined rule, according to the chained record after the described renewal, carrying out the link-state change that will link is that long-term linking status is also correspondingly changed this chained record, or deletes the operation of this link.
6. according to claim 3 or 4 described methods, wherein, described linking status also comprises the long history linking status, and wherein, described first pre-defined rule also comprises following rule:
-when described current link state is long-term linking status, keep the rule of the chained record of this link;
Wherein, described steps d is further comprising the steps of:
-based on described second pre-defined rule, according to the described chained record of keeping, carrying out the link-state change that will link is that the long history linking status is also correspondingly changed the chained record of this link, or deletes the operation of this link.
7. method according to claim 6, wherein, described linking status also comprises the permalink state, described first pre-defined rule also comprises following rule:
-be the link of long history linking status when identical when detecting a plurality of linking status, merge this a plurality of links, and correspondingly upgrade the chained record of the link after merging;
Wherein, described steps d is further comprising the steps of:
-based on described second pre-defined rule, according to the chained record after the described renewal, carrying out the link-state change that will link is that the permalink state is also correspondingly changed the chained record of this link, or deletes the operation of this link.
8. according to each described method in the claim 2 to 5, wherein, described linking status also comprises linking status in short-term, and wherein, described step a is further comprising the steps of:
-obtain the corresponding link information undetermined of Internet resources of the detected visit that fails in second predetermined interval;
Described step b is further comprising the steps of:
-should the corresponding link of link information undetermined add in the described dead chain storehouse, set up the initial link record of this link, and the state that should link is set to linking status in short-term.
9. method according to claim 8, wherein, described step b is further comprising the steps of:
B1 carries out secondary detection to detected link information undetermined in described second predetermined interval, to obtain testing result;
B2 is that the corresponding link of link information undetermined of dead chain adds in the described dead chain storehouse with testing result, set up the initial link record of this link, and the state that should link is set to linking status in short-term.
10. method according to claim 9, wherein, described step b2 is further comprising the steps of:
-to select testing result be that click volume is that the link of preceding N position is updated in the dead chain storehouse of described level in short-term in the link information undetermined of dead chain, wherein, N is predetermined threshold.
11. according to each described method in the claim 1 to 5, wherein, this method is further comprising the steps of:
Part or all of link in the described dead chain of-shielding storehouse.
12. according to each described method in the claim 1 to 5, wherein, the described network equipment comprises: network host, single network server, a plurality of webserver collection or based on the set of computers of cloud computing.
13. one kind is used for carrying out the network equipment that dead chain detects and handles based on classification, wherein, this network equipment comprises:
The record updating device, be used for based on first pre-defined rule, in conjunction with the current link state that links in the dead chain storehouse, to upgrade or to keep the chained record of this link, wherein, the processing mode that has comprised the chained record of the link under one or more linking status in described first pre-defined rule;
Processing unit is used for based on second pre-defined rule, according to the described chained record that upgrades the back or keep, comes corresponding operation is carried out in this link and/or its chained record, and wherein, described second pre-defined rule comprises following at least one:
-when the chained record of described link meets the chained record requirement of the link of other linking status, then the link-state change that will link is these other linking status, and correspondingly changes the chained record of this link;
-when described link or its chained record meet the chained record requirement of Remove Links, then delete this link;
-when the chained record of described link meets the chained record requirement of the state at this link place, then upgrade or keep the chained record of this link.
14. the network equipment according to claim 13, wherein, this network equipment also comprises:
Deriving means is used for obtaining the corresponding link information undetermined of Internet resources of the detected visit that fails;
The link updating device is used for upgrading according to described link information undetermined the link in described dead chain storehouse.
15. the network equipment according to claim 14, wherein, described linking status comprises interim linking status, and described deriving means comprises:
The first sub-deriving means is used for obtaining the corresponding link information undetermined of Internet resources of the detected visit that fails in first predetermined interval;
Described link updating device comprises:
The first sub-updating device is used for and should the corresponding link of link information undetermined adds described dead chain storehouse, set up the initial link record of this link, and the state that should link is set to interim linking status.
16. the network equipment according to claim 15, wherein, described first pre-defined rule comprises following rule:
-when described current link state was interim linking status, whether poll detects described link was dead chain;
Wherein, described record updating device comprises:
First checkout gear, whether be dead chain, and obtain the testing result of each poll if detecting this link for poll;
The first subrecord updating device for the testing result that obtains according to described each poll, upgrades the chained record of this link.
17. the network equipment according to claim 16, wherein, described linking status also comprises long-term linking status, and described processing unit also comprises:
The first sub-processing unit is used for based on described second pre-defined rule, and according to the chained record after the described renewal, carrying out the link-state change that will link is that long-term linking status is also correspondingly changed this chained record, or deletes the operation of this link.
18. according to claim 16 or the 17 described network equipments, wherein, described linking status also comprises the long history linking status, wherein, described first pre-defined rule also comprises following rule:
-when described current link state is long-term linking status, keep the rule of the chained record of this link;
Wherein, described processing unit also comprises:
The second sub-processing unit is used for based on described second pre-defined rule, and according to the described chained record of keeping, carrying out the link-state change that will link is long history linking status or the operation of deleting this link.
19. the network equipment according to claim 18, wherein, described linking status also comprises the permalink state, and described first pre-defined rule also comprises following rule:
-be the link of long history linking status when identical when detecting a plurality of linking status, merge this a plurality of links, and correspondingly upgrade the chained record of the link after merging;
Wherein, described processing unit also comprises:
The 3rd sub-processing unit is used for based on described second pre-defined rule, and according to the chained record after the described renewal, carrying out the link-state change that will link is permalink state or the operation of deleting this link.
20. according to each described network equipment in the claim 14 to 17, wherein, described linking status also comprises linking status in short-term, described deriving means also comprises:
The second sub-deriving means is used for obtaining the corresponding link information undetermined of Internet resources of the detected visit that fails in second predetermined interval;
Described link updating device also comprises:
The second sublink updating device is used for and should the corresponding link of link information undetermined adds described dead chain storehouse, set up the initial link record of this link, and the state that should link is set to linking status in short-term.
21. the network equipment according to claim 20, wherein, described link updating device also comprises:
Second checkout gear is used for detected link information undetermined in described second predetermined interval is carried out secondary detection, to obtain testing result;
The 3rd sublink updating device adds described dead chain storehouse for the corresponding link of link information undetermined that with testing result is dead chain, set up the initial link record of this link, and the state that should link is set to linking status in short-term.
22. the network equipment according to claim 21, wherein, described the 3rd sublink updating device comprises:
Choice device, be used for selecting testing result is that the link that the link information click volume undetermined of dead chain is preceding N position is updated in the dead chain storehouse of described level in short-term, wherein, N is predetermined threshold.
23. according to each described network equipment in the claim 13 to 17, wherein, this network equipment also comprises:
Screening arrangement is used for the part or all of link in the described dead chain of shielding storehouse.
24. according to each described method in the claim 13 to 17, wherein, the described network equipment comprises: network host, single network server, a plurality of webserver collection or based on the set of computers of cloud computing.
CN 201010536638 2010-11-09 2010-11-09 Method for detecting and processing dead links on basis of classification, and network equipment Active CN102025559B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN 201010536638 CN102025559B (en) 2010-11-09 2010-11-09 Method for detecting and processing dead links on basis of classification, and network equipment

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN 201010536638 CN102025559B (en) 2010-11-09 2010-11-09 Method for detecting and processing dead links on basis of classification, and network equipment

Publications (2)

Publication Number Publication Date
CN102025559A CN102025559A (en) 2011-04-20
CN102025559B true CN102025559B (en) 2013-07-03

Family

ID=43866452

Family Applications (1)

Application Number Title Priority Date Filing Date
CN 201010536638 Active CN102025559B (en) 2010-11-09 2010-11-09 Method for detecting and processing dead links on basis of classification, and network equipment

Country Status (1)

Country Link
CN (1) CN102025559B (en)

Families Citing this family (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103198062B (en) * 2012-01-04 2017-07-25 百度在线网络技术(北京)有限公司 A kind of method and system for monitoring the dead chain of the page and js mistakes
CN102663062B (en) * 2012-03-30 2015-01-14 北京奇虎科技有限公司 Method and device for processing invalid links in search result
CN103678072A (en) * 2012-09-05 2014-03-26 百度在线网络技术(北京)有限公司 Method and device for testing system
CN104158697B (en) * 2013-10-18 2017-07-21 深圳信息职业技术学院 A kind of dead chain detection method and device
CN104598458B (en) * 2013-10-30 2019-07-16 腾讯科技(深圳)有限公司 Page detection method and device
CN104750741A (en) * 2013-12-30 2015-07-01 中国移动通信集团湖南有限公司 Invalid link processing method and invalid link processing device
CN106682041A (en) * 2015-11-11 2017-05-17 北京国双科技有限公司 Method and device for detecting webpage broken link
US9473440B1 (en) 2016-01-19 2016-10-18 International Business Machines Corporation Hyperlink validation
CN108255868B (en) * 2016-12-29 2020-11-24 北京国双科技有限公司 Method and device for checking links in website
CN112269666B (en) * 2020-11-10 2023-07-25 北京百度网讯科技有限公司 Applet dead-link detection method and device, computing device and medium

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101211340A (en) * 2006-12-29 2008-07-02 上海芯盛电子科技有限公司 Dynamic network crawler based on client end /service end
CN101582913A (en) * 2008-05-14 2009-11-18 北京帮助在线信息技术有限公司 Equipment and method for graded dispatching of platforms
EP2129042A1 (en) * 2007-03-08 2009-12-02 Huawei Technologies Co., Ltd. A multicast network system, node and a method for detecting a fault of a multicast network link

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101136789A (en) * 2006-08-30 2008-03-05 华为技术有限公司 Method and device for implementing terminal-to-terminal link detection, routing strategy rearrangement
CN101083625B (en) * 2007-07-13 2011-04-06 华为技术有限公司 Method and apparatus for expediting link convergence
CN101873235A (en) * 2010-05-25 2010-10-27 中兴通讯股份有限公司 Detection method of equipment network link, network management system and network system

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101211340A (en) * 2006-12-29 2008-07-02 上海芯盛电子科技有限公司 Dynamic network crawler based on client end /service end
EP2129042A1 (en) * 2007-03-08 2009-12-02 Huawei Technologies Co., Ltd. A multicast network system, node and a method for detecting a fault of a multicast network link
CN101582913A (en) * 2008-05-14 2009-11-18 北京帮助在线信息技术有限公司 Equipment and method for graded dispatching of platforms

Also Published As

Publication number Publication date
CN102025559A (en) 2011-04-20

Similar Documents

Publication Publication Date Title
CN102025559B (en) Method for detecting and processing dead links on basis of classification, and network equipment
US11755387B1 (en) Updating code of an app feature based on a value of a query feature
US11989707B1 (en) Assigning raw data size of source data to storage consumption of an account
US11641372B1 (en) Generating investigation timeline displays including user-selected screenshots
US11132111B2 (en) Assigning workflow network security investigation actions to investigation timelines
US20220171736A1 (en) Managing datasets generated by search queries
US20190166145A1 (en) Selecting Network Security Event Investigation Timelines in a Workflow Environment
EP3299972B1 (en) Efficient query processing using histograms in a columnar database
KR101557294B1 (en) Search results ranking using editing distance and document information
US10860655B2 (en) Creating and testing a correlation search
US9251157B2 (en) Enterprise node rank engine
US11755635B2 (en) Presentation and sorting of summaries of alert instances triggered by search queries
US20130318514A1 (en) Map generator for representing interrelationships between app features forged by dynamic pointers
US20130318496A1 (en) Detection of central-registry events influencing dynamic pointers and app feature dependencies
US11816172B2 (en) Data processing method, server, and computer storage medium
CN105183873A (en) Malicious clicking behavior detection method and device
US8954413B2 (en) Methods and apparatus for adaptively harvesting pertinent data
US10540360B2 (en) Identifying relationship instances between entities
CN106339372B (en) Method and device for optimizing search engine
CN106327039A (en) Weekly report information processing method and apparatus
CN110442616A (en) A kind of page access path analysis method and system for big data quantity
JP5663742B2 (en) Image search server and image information management method for image search server
KR20140056536A (en) News recommendation system and method for recommending news
Tiwari Extraction of user specified web knowledge using Spatial Data Mining

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
C14 Grant of patent or utility model
GR01 Patent grant
EE01 Entry into force of recordation of patent licensing contract

Application publication date: 20110420

Assignee: Beijing small mutual Entertainment Technology Co., Ltd.

Assignor: BEIJING BAIDU NETCOM SCIENCE AND TECHNOLOGY Co.,Ltd.

Contract record no.: 2017990000087

Denomination of invention: Method for detecting and processing dead links on basis of classification, and network equipment

Granted publication date: 20130703

License type: Exclusive License

Record date: 20170315

EE01 Entry into force of recordation of patent licensing contract