CN108090080A - For replacing the method and system and crawling method of parsing template - Google Patents
For replacing the method and system and crawling method of parsing template Download PDFInfo
- Publication number
- CN108090080A CN108090080A CN201611036900.7A CN201611036900A CN108090080A CN 108090080 A CN108090080 A CN 108090080A CN 201611036900 A CN201611036900 A CN 201611036900A CN 108090080 A CN108090080 A CN 108090080A
- Authority
- CN
- China
- Prior art keywords
- parsing template
- parsing
- template
- task
- content
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 48
- 230000009193 crawling Effects 0.000 title claims description 10
- 238000012544 monitoring process Methods 0.000 claims abstract description 24
- 241000270322 Lepidosauria Species 0.000 claims abstract description 14
- 230000015654 memory Effects 0.000 claims description 14
- 238000000605 extraction Methods 0.000 claims description 5
- 230000008859 change Effects 0.000 description 18
- 238000010586 diagram Methods 0.000 description 7
- 230000006870 function Effects 0.000 description 6
- 230000008569 process Effects 0.000 description 5
- 238000004458 analytical method Methods 0.000 description 4
- 230000003993 interaction Effects 0.000 description 4
- 230000000694 effects Effects 0.000 description 3
- 230000007246 mechanism Effects 0.000 description 3
- 230000006978 adaptation Effects 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 2
- 238000001514 detection method Methods 0.000 description 2
- 230000000737 periodic effect Effects 0.000 description 2
- 238000004321 preservation Methods 0.000 description 2
- 230000003044 adaptive effect Effects 0.000 description 1
- 230000009194 climbing Effects 0.000 description 1
- 239000000470 constituent Substances 0.000 description 1
- 238000007796 conventional method Methods 0.000 description 1
- 238000012217 deletion Methods 0.000 description 1
- 230000037430 deletion Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 239000002360 explosive Substances 0.000 description 1
- 238000009434 installation Methods 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012856 packing Methods 0.000 description 1
- 238000005728 strengthening Methods 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/95—Retrieval from the web
- G06F16/951—Indexing; Web crawling techniques
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The disclosure provides a kind of method and system for replacement parsing template, and the parsing template crawls task parsing for reptile.Method includes：It is registered to task monitor, to monitor the variation of the parsing template, the task monitor storage variation for parsing template and monitoring the parsing template；It obtains the parsing template from the task monitor and changes message；Obtain the parsing template content after changing.What the disclosure provided can realize that parsing template content is changed in completion in time in the case where not restarting to crawler capturing end for replacing the method and system of parsing template.
Description
Technical field
This disclosure relates to Internet technical field, in particular to a kind of method for replacement parsing template with being
System.
Background technology
With the explosive growth of internet information, it is soft to there are more and more applications based on obtaining bulk information
Part, such as search engine, e-commerce price comparing system, public sentiment system etc..Wherein, web crawlers is these using indispensable
Important bottom component.And for big data quantity internet information, single node reptile can not meet the needs of real, in
It is that distributed network reptile is come into being.
For a set of distributed network crawler system, reptile dispatching terminal and crawler capturing end two parts are generally divided into.Wherein,
Reptile dispatching terminal is mainly responsible for the reasonable distribution for the task that captures to multiple crawler capturing ends, while is received preservation grasping end and returned
The crawl result data returned；Crawler capturing end is mainly responsible for analog access request, actually goes the Html original data of crawl webpage,
And pass through the parsing template configured, Html originals data are subjected to parsing extraction, get crawl as a result, returning to reptile scheduling
End.The content for wherein parsing template is generally regular expression, for matching web page contents, so as to get crawl data.
Because the webpage Html structures of each website are different, task is captured for different websites, configuration
It is also different to parse template.When website revision upgrades, website Html structures can change, and just need to climb to all at this time
The parsing template of worm grasping end carries out adaptation update, is otherwise just present with the problem of parsing data exception.
Therefore, it is necessary to a kind of improved methods for replacing parsing template.
It should be noted that information is only used for strengthening the reason to the background of the disclosure disclosed in above-mentioned background section
Solution, therefore can include not forming the information to the prior art known to persons of ordinary skill in the art.
The content of the invention
Being designed to provide for the disclosure is a kind of for replacing the method and system of parsing template, is used to implement not to climbing
Worm grasping end is completed to change parsing template content in time in the case of being restarted, and reaches parsing template content automation
The effect that heat is replaced.
According to one aspect of the disclosure, a kind of method for replacement parsing template is provided, the parsing template is used for
Reptile crawls task parsing, including：
It is registered to task monitor, to monitor the variation of the parsing template, the task monitor stores the parsing
Template and the variation for monitoring the parsing template；
It obtains the parsing template from the task monitor and changes message；
Obtain the parsing template content after changing.
In a kind of exemplary embodiment of the disclosure, the task monitor is realized by Zookeeper servers.
In a kind of exemplary embodiment of the disclosure, the content of the parsing template includes the canonical of extraction web page contents
Expression formula.
In a kind of exemplary embodiment of the disclosure, the task monitor stores one or more institutes in the form of Task Tree
State parsing template.
In a kind of exemplary embodiment of the disclosure, monitoring the variation of the parsing template includes monitoring one or more solutions
Analyse the variation of template.
In a kind of exemplary embodiment of the disclosure, further include：By the parsing template content after the variation with Key-
Value forms are stored into memory.
In a kind of exemplary embodiment of the disclosure, further include：To parsing template initialization.
According to one aspect of the disclosure, a kind of system for replacement parsing template is provided, the parsing template is used for
Reptile crawls task parsing, including：
Registration module is monitored, for being registered to task monitor, to monitor the variation of the parsing template, the task prison
The control device storage parsing template and the variation for monitoring the parsing template；
It notifies acquisition module, changes message for obtaining the parsing template from the task monitor；
Template renewal module is parsed, for obtaining the content of the parsing template after changing, by the content of the parsing template
Storage is in memory.
In a kind of exemplary embodiment of the disclosure, the task monitor is realized by Zookeeper servers.
According to one aspect of the disclosure, a kind of crawling method is provided, including being replaced as described in above-mentioned any one
The method for changing parsing template.
The method for replacement parsing template that the disclosure provides is being connected to by being monitored to parsing template change
Updated parsing template is obtained during Notification of Changes, reduces the process of manual intervention update parsing template, realizes to parsing
Template renewal timely responds to, and processor efficiency caused by avoiding periodic monitor parsing template renewal wastes, and improves parsing
The efficiency of template renewal.Meanwhile it by the way that the content of updated parsing template is write direct in memory, realizes and is not restarting
Update parsing template under conditions of crawler capturing end has achieved the effect that parsing template content automation heat replaces.
It should be appreciated that above general description and following detailed description are only exemplary and explanatory, not
The disclosure can be limited.
Description of the drawings
Attached drawing herein is merged in specification and forms the part of this specification, shows the implementation for meeting the disclosure
Example, and for explaining the principle of the disclosure together with specification.It should be evident that the accompanying drawings in the following description is only the disclosure
Some embodiments, for those of ordinary skill in the art, without creative efforts, can also basis
These attached drawings obtain other attached drawings.
Fig. 1 schematically shows a kind of flow chart of method for replacement parsing template in disclosure exemplary embodiment.
Fig. 2 schematically shows a kind of schematic diagram of Task Tree in disclosure exemplary embodiment.
Fig. 3 meanings property shows the interaction schematic diagram of middle parsing template renewal in disclosure exemplary embodiment.
Fig. 4 schematically shows a kind of block diagram of system for replacement parsing template in disclosure exemplary embodiment.
Specific embodiment
Example embodiment is described more fully with reference to the drawings.However, example embodiment can be with a variety of shapes
Formula is implemented, and is not understood as limited to example set forth herein；On the contrary, these embodiments are provided so that the disclosure will more
Fully and completely, and by the design of example embodiment comprehensively it is communicated to those skilled in the art.Described feature, knot
Structure or characteristic can be in any suitable manner incorporated in one or more embodiments.In the following description, provide perhaps
More details fully understand embodiment of the present disclosure so as to provide.It it will be appreciated, however, by one skilled in the art that can
One or more in the specific detail are omitted with technical solution of the disclosure or others side may be employed
Method, constituent element, system, step etc..In other cases, be not shown in detail or describe known solution to avoid a presumptuous guest usurps the role of the host and
So that all aspects of this disclosure thicken.
In addition, attached drawing is only the schematic illustrations of the disclosure, identical reference numeral represents same or similar portion in figure
Point, thus repetition thereof will be omitted.Attached some block diagrams shown in figure are functional entitys, not necessarily necessary and object
The entity managed or be logically independent is corresponding.Software form may be employed to realize these functional entitys or in one or more
These functional entitys are realized in hardware module or integrated circuit or in heterogeneous networks and/or processor system and/or microcontroller
These functional entitys are realized in system.
Disclosure example embodiment is described in detail below in conjunction with the accompanying drawings.
Fig. 1 is a kind of flow chart for the method 100 that template is parsed for replacement, the solution in disclosure example embodiment
The task that crawls that analysis template can be used for reptile parses.
It with reference to figure 1, in S102, is registered to task monitor, to monitor the variation of the parsing template, the Mission Monitor
The device storage parsing template and the variation for monitoring the parsing template.
In this example embodiment, task monitor can be realized by Zookeeper servers, for example, can pass through
The Watcher monitoring mechanisms that Zookeeper is provided realize the variation of monitoring parsing template.Zookeeper is a kind of open source code
Distributed application program coordination service, can be realized by the way that its source code is integrated in application software to the profit of Zookeeper
With.In Zookeeper, when certain object sets some node of monitoring Zookeeper, if the node content is become
More, Zookeeper servers can notify all objects for setting and monitoring the node automatically, inform its node content monitored
It changes, here it is the Watcher monitoring mechanisms of Zookeeper.In the other embodiment of the disclosure, task monitor
It can be realized by other instruments, the disclosure is not particularly limited this.
In some embodiments, multiple crawler capturing ends can are registered to task monitor, to monitor corresponding parsing mould
The variation of plate.Monitor the parsing template changes the variation that can include monitoring one or more parsing templates.That is, crawler capturing
Correspondence between end and parsing template can be the relation of one-to-many relation, many-to-one relation either multi-to-multi,
One crawler capturing end can monitor multiple parsing templates, and a parsing template can also be monitored by multiple crawler capturing ends.
It is noted that in some embodiments of the present disclosure, the parsing template to crawler capturing end can also be included
It is initialized.For example, crawler capturing end can obtain all corresponding parsing templates first, the process of parsing template is being obtained
In, the variation for monitoring the parsing template is registered to task monitor, i.e. crawler capturing end can not have what is changed in parsing template
When perform the corresponding task of initial parsing template.Certainly, parsing template is initialized and registered to task monitor can
To be carried out at the same time, can not also be carried out at the same time, the disclosure is not limited.
Step S104 obtains the parsing template from the task monitor and changes message.
The mode that acquisition parsing template changes message can include passive acquisition or active obtaining.Passive obtain can be by appointing
Monitor transmission parsing template of being engaged in changes message informing, and active obtaining can parse template by the active search of crawler capturing end and change
Message.In this example embodiment, when realizing task monitor using Zookeeper servers, obtain parsing template and become
The mode of dynamic message is passive mode, when parsing template content changes, is owned by Zookeeper server proactive notifications
Monitor the crawler capturing end of the parsing template.
The method that existing timing detection version number replaces parsing template is usually to record update parsing template every time
Version number, automatic regular polling compare local record parsing template version number whether with server set newest parsing template version
It number has differences, when version number is different, then the newest parsing template of download server, local parsing template is replaced
It changes, while local version number is updated to as server.Promptness cannot be met compared to the above method, increase CPU is born
The shortcomings that load, the disclosure are used to replace the method for parsing template by using the monitoring mechanism of Zookeeper to parsing template
It is monitored, when parsing template content changes, the parsing templates is monitored by Zookeeper server proactive notifications are all
Crawler capturing end, can to avoid increase cpu load, while can parse template have variation when notify crawler capturing in time
Upgrading in time for parsing template is realized at end.
When crawler capturing end monitors the variation of one or more parsing templates, the parsing template content obtained after changing can be with
Including obtaining one or more parsing template contents.When one, which parses template, changes, crawler capturing end can obtain the solution
The parsing template for analysing template changes message；When multiple parsing templates change simultaneously, crawler capturing end can obtain more
The parsing template of a parsing template changes message.Content update or parsing template deletion can be included by parsing the variation of template,
It can be realized by manually changing the parsing template content stored in task monitor.
Step S106 obtains the parsing template content after changing.
When crawler capturing end obtain change after one or more parsing template contents after, can by the parsing template content with
The form of Key-value is stored in memory.In the process, key can represent the corresponding task names of a parsing template,
Value can represent the content of the parsing template.When the content for parsing template changes, crawler capturing end can be according to this
The corresponding task names of template, i.e. key values are parsed, this corresponding parsing template content is stored in the form of value in memory.
A kind of existing whole parsing template replacement method for replacing crawler capturing end code bag is generally included to parsing mould
After plate is updated, manually crawler capturing end code is packaged, then replaces original all code bags with fresh code bag,
And crawler capturing end is restarted.For this method since it is desired that hand press-packing, upgrading replacement efficiency is too low, and replaces crawl
It needs to restart crawler capturing end during bag, the work efficiency drop at crawler capturing end can be caused.This example embodiment
For replacing the method for parsing template by the way that directly parsing template content is stored in content, aforesaid way loading solution is avoided
The shortcomings that analysis template content needs to restart crawler capturing end, cause reptile operating interruptions, the heat for realizing parsing template are replaced
It changes.
The method for replacement parsing template that the disclosure provides is being connected to by being monitored to parsing template change
Updated parsing template is obtained during Notification of Changes, reduces the process of manual intervention update parsing template, realizes to parsing
Template renewal timely responds to, and processor efficiency caused by avoiding periodic monitor parsing template renewal wastes, and improves parsing
The efficiency of template renewal.Meanwhile it by the way that the content of updated parsing template is write direct in memory, realizes and is not restarting
Update parsing template under conditions of crawler capturing end has achieved the effect that parsing template content automation heat replaces.
The disclosure is described in detail for replacing the method for parsing template with reference to specific embodiment.
In some embodiments, the task monitor storage parsing template can include storing in the form of Task Tree
One or more described parsing templates.The content of the parsing template includes the regular expression of extraction web page contents.
Fig. 2 shows a kind of schematic diagram of Task Tree of the disclosure.With reference to figure 2, Task Tree 200 can include general assignment 201
And subtask 2011,2022 and 2013.It should be noted that, although three subtasks are illustrated only in attached drawing, but
In actual implementation, subtask can have multiple.General assignment 201 searches parsing template as reptile dispatching terminal and crawler capturing end
Interface, to manage the management path that the corresponding task of all parsing templates is provided convenience.
Fig. 3 shows that this example embodiment replaces the interaction schematic diagram of the complete procedure of parsing template.With reference to figure 3, on
The friendship of registration center's system end 301, Zookeeper servers 302 and crawler capturing end 303 can be included by stating complete interaction
Mutual process.
Registration center's system end 301 can individually be disposed as autonomous system, and major function can include parsing template note
Volume and content synchronization to Zookeeper servers.Wherein, parsing template registering functional can include corresponding to when some crawls task
Parsing template content when needing change, enter registration center's system by operating personnel, input change mission number and correspondence
New task parsing template content, with complete to parsing template content change configuration.After the completion of above-mentioned configuration, registration center
System background can be connected automatically to Zookeeper servers, in a manner of tree hierarchy structure as shown in Figure 2, appoint always
The child node of the corresponding task is found or created under business root node, is the parsing template just configured by the curriculum offering of child node
Content.Above-mentioned storage and management parsing template mode or other modes, such as by queue, stack, figure or other
Data structure known to field technology personnel realizes that the disclosure is not restricted this.
The major function at crawler capturing end 302 can include change monitoring and parsing template renewal.Wherein change monitoring work(
The change of parsing template content can be monitored, parsing template renewal function can be on the premise of it need not restart to parsing mould
Plate content is updated.
When crawler capturing end 302 starts, under general assignment root node, can be obtained by connecting Zookeeper servers
The parsing template content of the corresponding subtask node storage of the task of crawling is taken, while each subtask node is supervised in registration
Control.It should be noted that in some implementations, further include crawler capturing end and travel through all subtask nodes, the disclosure to this not
Make particular determination.During group task node content change, crawler capturing end 302 can receive the transmission of Zookeeper servers
Parsing template change notice, and again to parsing template content be read out, to get newest parsing template content.
It, can be by parsing template content in the form of Key-Value when crawler capturing end 302 gets parsing template content
Preservation is stored in local memory.Wherein Key can include the corresponding title for crawling task of parsing template, and Value can be wrapped
Include the regular expression content that parsing template includes.When crawler capturing end 302 reacquires the newest parsing to subtask node
During template content, the corresponding Value values of the Key can be directly changed to the content of newest parsing template, that is, be not required
Parsing template content is completed while restarting crawler capturing end 302 to replace.
In registration center's system end 301, parsing the specific interaction of template configuration can include：
A1. user enters registration center's system.
A2. user's input crawls the number of task and the what be new for crawling the corresponding parsing template of task.Its
Middle mission number can be the globally unique number that can be identified for that a task, and parsing template content can include extraction webpage
The regular expression of Html contents.
A3. user, which clicks on, determines, preserves mission number and its corresponding parsing template content to registration center's system
In database.
A4. registration center's system is synchronized to template content is parsed on Zookeeper servers.
Registration center's system end 301 starts connection Zookeeper server threads from the background, then is serviced into Zookeeper
The device end course of work.The Zookeeper server ends course of work can include：
B1. general assignment node is created or finds, wherein general assignment node can be as all root nodes for crawling task.
B2. according to above-mentioned mission number, under general assignment node, the subtask section of the corresponding task is checked whether there is
Point, if so, the node content then is updated to newest parsing template content.
B3. if not checking the subtask node of the corresponding task, corresponding be somebody's turn to do can be created under general assignment node
The subtask node of task, and nodename is arranged to mission number, node content is configured to the corresponding parsing mould of the task
The content of plate.
For crawler capturing end 302, change monitor function can be realized by two kinds of workflows.
A kind of workflow is when above-mentioned general assignment node crawls task parsing template and when being created for convenience of management, is climbed
Worm grasping end 302 can be firstly connected to Zookeeper servers, read the subtask that crawl task is corresponded under general assignment node
Node, using the subtask namespace node of reading as Key values, subtask node content is as Value values, with Key-Value's
Form is stored in the memory at crawler capturing end 302, completes to crawl the Initialize installation of task parsing template.
Another workflow is when above-mentioned general assignment node has been created, and solution has been preserved at crawler capturing end 302
When analysing template, crawler capturing end 302 can be in the initialization procedure of above-mentioned parsing template, first to task monitor
The parsing of subtask node under (Zookeeper in this example embodiment) registration monitoring general assignment root node and/or root node
The variation of template, the API which may be employed Zookeeper offers carry out registration monitoring.When node content changes,
Zookeeper can broadcast the crawler capturing end 302 that notice has monitored the change subtask node automatically.
When crawler capturing end 302, which gets parsing template, changes notice, following workflow can be carried out to realize solution
Analyse template renewal function.
After crawler capturing end 302 receives parsing template variation notice, thread can be started and be connected to Zookeeper services
Device reads the corresponding subtask node of parsing template of the variation, to obtain the content of updated newest parsing template.It can be with
Using the corresponding task name (or number) that crawls of the parsing template as Key values, the content of the parsing template as Value values, with
Task is crawled in above-mentioned key-value forms update crawler capturing end 302 and parses the Map memories pair of template content mapping
As.Since crawler capturing end 302 directly uses the Map memory object values when being parsed to crawl content, so after update
The replacement of parsing template content is completed, avoids and restarts crawler capturing end during parsing template is replaced.
Corresponding to above method embodiment, the disclosure also provides a kind of system for replacement parsing template, the parsing
Template crawls task parsing for reptile.Fig. 4 is a kind of block diagram of the system for replacement parsing template.With reference to figure 4, it is
System 400 can include monitoring registration parsing template 402, notice acquisition module 404 and parsing template renewal module 406.
Monitoring registration module 402 can be used for registering to task monitor, described to monitor the variation of the parsing template
The task monitor storage parsing template and the variation for monitoring the parsing template.
Notice acquisition module 404 can be used for obtaining the parsing template variation message from the task monitor.
Parsing template renewal module 406 can be used for the content for obtaining the parsing template after changing, by the parsing template
Content storage in memory.
In a kind of exemplary embodiment of the disclosure, the task monitor is realized by Zookeeper servers.
Above system 400 can be used to implement the above method 100.Since each functions of modules of system 400 is corresponding
It is described in detail in embodiment of the method, the disclosure is repeated no more in this.
What the disclosure provided is changed for replacing the method and system of parsing template by monitoring parsing template content, is realized
The content of automatic replacement parsing template, avoids the poor efficiency for manually replacing parsing Form board tape；By being sent out in parsing template
Parsing template Notification of Changes is sent when changing dynamic, the wasting of resources that timing detection parsing template variation is brought is avoided, ensure that
Crawler capturing end upgrades to parsing template in time；Meanwhile it is stored directly in crawler capturing end by the way that template content will be parsed
In memory, avoid and restart crawler capturing end when update parses template, ensure that the uninterrupted operation of crawler capturing end program.
Further, the disclosure also provides a kind of crawling method, can include as described above for replacement parsing mould
The method 100 of plate.What the crawling method that the disclosure provides can be used for crawler capturing end crawls work.
Those skilled in the art will readily occur to the disclosure its after considering specification and putting into practice invention disclosed herein
Its embodiment.This application is intended to cover any variations, uses, or adaptations of the disclosure, these modifications, purposes or
Person's adaptive change follows the general principle of the disclosure and including the undocumented common knowledge in the art of the disclosure
Or conventional techniques.Description and embodiments are considered only as illustratively, and the true scope and spirit of the disclosure are by appended
Claim is pointed out.
Claims (10)
1. a kind of for replacing the method for parsing template, the parsing template crawls task parsing for reptile, and feature exists
In, including：
It is registered to task monitor, to monitor the variation of the parsing template, the task monitor stores the parsing template
And monitor the variation of the parsing template；
It obtains the parsing template from the task monitor and changes message；
Obtain the parsing template content after changing.
2. according to the method described in claim 1, it is characterized in that, the task monitor is real by Zookeeper servers
It is existing.
3. according to the method described in claim 1, it is characterized in that, the content of the parsing template includes extraction web page contents
Regular expression.
4. according to the method described in claim 1, it is characterized in that, the task monitor stored in the form of Task Tree it is one or more
A parsing template.
5. according to the method described in claim 4, it is characterized in that, the variation of the monitoring parsing template is one or more including monitoring
The variation of a parsing template.
6. it according to the method described in claim 1, it is characterized in that, further includes：By the parsing template content after the variation with
Key-value forms are stored into memory.
7. it according to the method described in claim 1, it is characterized in that, further includes：To parsing template initialization.
8. a kind of for replacing the system of parsing template, the parsing template crawls task parsing for reptile, and feature exists
In, including：
Registration module is monitored, for being registered to task monitor, to monitor the variation of the parsing template, the task monitor
It stores the parsing template and monitors the variation of the parsing template；
It notifies acquisition module, changes message for obtaining the parsing template from the task monitor；
Template renewal module is parsed, for obtaining the content of the parsing template after changing, the content of the parsing template is stored
In memory.
9. system according to claim 8, which is characterized in that the task monitor is real by Zookeeper servers
It is existing.
10. a kind of crawling method, which is characterized in that including replacing the method for parsing template as described in claim 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201611036900.7A CN108090080A (en) | 2016-11-22 | 2016-11-22 | For replacing the method and system and crawling method of parsing template |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201611036900.7A CN108090080A (en) | 2016-11-22 | 2016-11-22 | For replacing the method and system and crawling method of parsing template |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN108090080A true CN108090080A (en) | 2018-05-29 |
Family
ID=62168646
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201611036900.7A Pending CN108090080A (en) | 2016-11-22 | 2016-11-22 | For replacing the method and system and crawling method of parsing template |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108090080A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110717131A (en) * | 2018-06-27 | 2020-01-21 | 北京国双科技有限公司 | Page revising monitoring method and related system |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101957816A (en) * | 2009-07-13 | 2011-01-26 | 上海谐宇网络科技有限公司 | Webpage metadata automatic extraction method and system based on multi-page comparison |
| CN103942335A (en) * | 2014-05-07 | 2014-07-23 | 武汉大学 | Construction method of uninterrupted crawler system oriented to web page structure change |
-
2016
- 2016-11-22 CN CN201611036900.7A patent/CN108090080A/en active Pending
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101957816A (en) * | 2009-07-13 | 2011-01-26 | 上海谐宇网络科技有限公司 | Webpage metadata automatic extraction method and system based on multi-page comparison |
| CN103942335A (en) * | 2014-05-07 | 2014-07-23 | 武汉大学 | Construction method of uninterrupted crawler system oriented to web page structure change |
Non-Patent Citations (3)
| Title |
|---|
| QING____: "Zookeeper-Watcher机制与异步调用原理", 《HTTPS://WWW.ITEYE.COM/BLOG/SHIFT-ALT-CTRL-1847320》 * |
| ZMYCOCO2: "Apache ZooKeeper Watcher 机制源码解释", 《HTTPS://BLOG.CSDN.NET/MICHAELZHOU224/ARTICLE/DETAILS/51923478》 * |
| 明静: "zookeeper 官方文档——综述", 《HTTPS://WWW.CNBLOGS.COM/JSSTUDYJJ/P/5365059.HTML》 * |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110717131A (en) * | 2018-06-27 | 2020-01-21 | 北京国双科技有限公司 | Page revising monitoring method and related system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110781007B (en) | Task processing method, device, server, client, system and storage medium | |
| US10985997B2 (en) | Systems and methods for domain-driven design and execution of metamodels | |
| US9578082B2 (en) | Methods for dynamically generating an application interface for a modeled entity and devices thereof | |
| KR101055030B1 (en) | Devices, Systems, and Methods for Data Server Managed Web Services Runtime | |
| US9158532B2 (en) | Methods for managing applications using semantic modeling and tagging and devices thereof | |
| US7395320B2 (en) | Providing automatic policy enforcement in a multi-computer service application | |
| US8631071B2 (en) | Recognition of and support for multiple versions of an enterprise canonical message model | |
| JP6164440B2 (en) | Application upgrade method and apparatus | |
| US20190052531A1 (en) | Systems and methods for service mapping | |
| US20110196957A1 (en) | Real-Time Policy Visualization by Configuration Item to Demonstrate Real-Time and Historical Interaction of Policies | |
| US7937716B2 (en) | Managing collections of appliances | |
| US20090063650A1 (en) | Managing Collections of Appliances | |
| US20100235493A1 (en) | Extendable distributed network management system and method | |
| US20140207932A1 (en) | Information processing system and information processing method | |
| US20190286500A1 (en) | Systems and method for event parsing | |
| US20150081625A1 (en) | Service polling method and system, and computer storage medium | |
| US20220182851A1 (en) | Communication Method and Apparatus for Plurality of Administrative Domains | |
| CN110457555A (en) | Collecting method, device and computer equipment, storage medium based on Docker | |
| CN108090080A (en) | For replacing the method and system and crawling method of parsing template | |
| CN110188258B (en) | Method and device for acquiring external data by using crawler | |
| Baird et al. | Self-adapting workflow reconfiguration | |
| CN104298750B (en) | Renewal processing method and processing device for real-time system communication | |
| CN116016702B (en) | Application observable data acquisition processing method, device and medium | |
| Gao | On Provisioning and configuring ensembles of IoT, network functions and cloud resources | |
| Wennerström | Active Assurance in Kubernetes |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20180529 |