CN104281680B - Data processing system, method and device for obtaining site resource - Google Patents
Data processing system, method and device for obtaining site resource Download PDFInfo
- Publication number
- CN104281680B CN104281680B CN201410521135.2A CN201410521135A CN104281680B CN 104281680 B CN104281680 B CN 104281680B CN 201410521135 A CN201410521135 A CN 201410521135A CN 104281680 B CN104281680 B CN 104281680B
- Authority
- CN
- China
- Prior art keywords
- data
- picture
- web
- appointed website
- website
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/95—Retrieval from the web
- G06F16/951—Indexing; Web crawling techniques
Abstract
The invention discloses a kind of data processing system, method and devices for obtaining site resource, wherein the system comprises：Data screening device for receiving the web data captured by web crawlers, and carries out Screening Treatment to the web data received in receive process, will filter out and be sent to web analysis server with the relevant web data of appointed website；Web analysis server obtains first structure data for carrying out dissection process with the relevant web data of appointed website according to preset parsing strategy pair, and first structure data is preserved to database；Database obtains the second structural data of the resource for describing appointed website for carrying out Data Fusion according to the first structure data received within a predetermined period of time.Using the present invention, the update cycle of site resource can be shortened, the timeliness of site resource is improved, for video website, moreover it is possible to which that improves video resource goes out figure rate, improves user experience.
Description
Technical field
The present invention relates to data processing fields, more particularly, it is related to a kind of for obtaining at the data of site resource
Reason system, method and device.
Background technology
Search engine is to use based on the site resource (site resource is usually described with structural data) included in database
Family provides search service.The search result of search engine is directly related with the site resource included in database, therefore, in order to carry
High user experience, needs the site resource that timely updates.
In the prior art, generally use such as under type updates site resource：First, web crawlers (spider) is waited for grab
The webpage of crawl is stored in first database and establishes index by the webpage for taking magnanimity；Then, to the full dose in first database
Webpage carries out screening and structural data parses (this operation is usually by manually triggering), and analysis result is stored in the second data
Library；Finally, the processing such as data fusion, foundation index are carried out to a variety of data by the second database to be shown on line.
Due to waiting for the time of web crawlers crawl webpage very long and being related to the data access processing to first database,
This causes the calculating cost (including database cost and time cost) for single webpage larger；Due to entire data screening,
Resolving is batch processing offline, this causes the integral cycle of data update longer.
Disadvantages described above causes the prior art that can not include newest site resource in time, this largely effects on the search body of user
It tests.And more complicated site resource is calculated for the stronger site resource of timeliness demand and structural data, is used
The prior art is even more to include in time.By taking the more demanding video resource of timeliness as an example, data processing is complex, and one
Aspect needs to can be only achieved preferable data cover from list of videos page and the common analytic structure data of video playing page；It is another
Aspect needs fusion and the relevant picture of webpage that could improve user experience in follow-up displaying, however the sea that massive video is brought
Amount picture resource is difficult to be completed in a short time the processing such as crawl, storage conversion at all.It is provided when using the prior art more new video
When source, not only the update cycle is long, cannot be satisfied the timeliness requirement of video resource, and due to picture processing and structural data
The progress of processing is difficult to control, it is easy to occur can not exhibiting pictures the case where, seriously affect user experience.
Invention content
In order to solve the defect present in the prior art, embodiment of the present invention provides a kind of for obtaining site resource
Data processing system, method and device, the defects of prior art data update period long, poor in timeliness of resource can be overcome.
In a first aspect, an embodiment of the present invention provides a kind of data processing systems for obtaining site resource, including：
Data screening device, for receiving the web data captured by web crawlers, and to receiving in receive process
Web data carry out Screening Treatment, will filter out and be sent to web analysis service with the relevant web data of appointed website
Device；
Web analysis server, for according to preset parsing strategy pair and the relevant web data of the appointed website into
Row dissection process, obtain with the relevant first structure data of the appointed website, and by the first structure data preserve
To database；
The database melts for carrying out data according to the first structure data received within a predetermined period of time
Conjunction is handled, and obtains the second structural data of the resource for describing the appointed website.
Optionally, in a kind of realization method of the present embodiment, the data screening device is specifically used for, and is receiving webpage
During data, according to URL (the Uniform Resoure Locator of the appointed website：Uniform resource locator) just
Then expression formula carries out Screening Treatment to the web data received.
Optionally, in another realization method of the present embodiment, when the appointed website is video website, the net
Page resolution server is specifically used for：It is to be regarded with the appointed website in web data that the web analysis server receives
When the relevant web data of frequency broadcast page, dissection process is carried out according to the first parsing strategy；It is connect in the web analysis server
When the web data received is web data relevant with the list of videos page of the appointed website, parsed according to described first
The second different parsing strategy of strategy carries out dissection process.
Optionally, in another realization method of the present embodiment, the data processing system further includes picture processing
System；The web analysis server is additionally operable to, and the image link parsed in the dissection process is sent to the figure
Piece processing subsystem；The picture processing subsystem, for according to the image link capture original image and according to picture at
Reason strategy handles the original image to obtain new picture, preserves the new picture and generates picture chain for the new picture
It connects, and the pictorial information of the image link comprising the original image and the image link of the new picture is sent to described
Database.
Still optionally further, the picture processing subsystem includes picture crawl server, picture processing server and figure
Sheet data library, wherein the picture captures server, for capturing the original image according to the image link, and by institute
It states original image and its image link is sent to the picture processing server；The picture processing server, for according to figure
Piece processing strategy is handled to obtain the new picture to the original image, and by the image link of the original image and institute
New picture is stated to preserve to the picture database；The picture database, for generating image link for the new picture, and will
The pictorial information is sent to the database.
Or still optionally further, shown database is additionally operable to according to second structural data and the pictorial information
Carry out Data Fusion.
Second aspect, an embodiment of the present invention provides a kind of data processing methods for obtaining site resource, including：
The web data captured by web crawlers is received, and the web data received is screened in receive process
Processing, obtains and the relevant web data of appointed website；
Dissection process is carried out with the relevant web data of the appointed website according to preset parsing strategy pair, is obtained and institute
State the relevant first structure data of appointed website；
Data Fusion is carried out according to the first structure data parsed within a predetermined period of time, is used
In the second structural data of the resource for describing the appointed website.
Optionally, in a kind of realization method of the present embodiment, the web data in receive process to receiving
Carrying out Screening Treatment includes：During receiving web data, according to the URL regular expressions of the appointed website to receiving
The web data arrived carries out Screening Treatment.
Optionally, in another realization method of the present embodiment, when the appointed website is video website, described
Carrying out dissection process with the relevant web data of the appointed website according to preset parsing strategy pair includes：When with the specified net
When relevant web data of standing is web data relevant with the video playing page of the appointed website, according to the first parsing strategy
Carry out the dissection process；When being list of videos page phase with the appointed website with the relevant web data of the appointed website
When the web data of pass, the dissection process is carried out according to the second parsing strategy different from the first parsing strategy.
Optionally, in another realization method of the present embodiment, the method further includes：It will be in the dissection process mistake
The image link parsed in journey is sent to picture processing subsystem, and the picture processing subsystem is for executing following processing：
It captures original image according to the image link and handles strategy according to picture and the original image is handled and newly schemed
Piece preserves the new picture and generates image link for the new picture；Receive the figure sent by the picture processing subsystem
Piece information, the pictorial information include the image link of the image link and the new picture of the original image；According to described
Second structural data and the pictorial information carry out Data Fusion.
Optionally, in another realization method of the present embodiment, the method further includes：According in the dissection process
In parse image link crawl original image；It handles strategy according to picture and the original image is handled and newly schemed
Piece preserves the new picture and preserves and be that the new picture generates image link；According to second structural data and picture
Information carries out Data Fusion, and the pictorial information includes the picture of the image link and the new picture of the original image
Link.
The third aspect, an embodiment of the present invention provides a kind of data processing equipments for obtaining site resource, including：
Data screening module, for receiving the web data captured by web crawlers, and to receiving in receive process
Web data carry out Screening Treatment, obtain and the relevant web data of appointed website；
Data resolution module, for being carried out according to preset parsing strategy pair and the relevant web data of the appointed website
Dissection process obtains and the relevant first structure data of the appointed website；
Data fusion module, for according to the first structure data that parse within a predetermined period of time into line number
According to fusion treatment, the second structural data of the resource for describing the appointed website is obtained.
Optionally, in first realization method of the present embodiment, the data screening module is specifically used for, and is receiving webpage
During data, Screening Treatment is carried out to the web data received according to the URL regular expressions of the appointed website.
Optionally, in second realization method of the present embodiment, the data resolution module includes：
First analyzing sub-module, for the appointed website be video website and with the relevant net of the appointed website
When page data is web data relevant with the video playing page of the appointed website, carried out at parsing according to the first parsing strategy
Reason；Second analyzing sub-module, for the appointed website be video website and with the relevant webpage number of the appointed website
According to for web data relevant with the list of videos page of the appointed website when, according to different from the first parsing strategy the
Two parsing strategies carry out dissection process.
Optionally, in the third realization method of the present embodiment, the data processing equipment further includes：Image link is sent
Module, the image link for parsing the data resolution module in the dissection process are sent to picture processing subsystem
System；Wherein, the picture processing subsystem is for executing following processing：Original image and basis are captured according to the image link
Picture processing strategy handles the original image to obtain new picture, preserves the new picture and is generated for the new picture
Image link, and the pictorial information of the image link comprising the original image and the image link of the new picture is sent
To the data processing equipment；Pictorial information receiving module, for receiving the pictorial information；The data fusion module is also used
In carrying out Data Fusion according to second structural data and the pictorial information, obtain including the pictorial information
Structural data.
Optionally, in the 4th realization method of the present embodiment, the data processing equipment further includes：Picture captures mould
Block, the image link for being parsed in the dissection process according to the data resolution module capture original image, and figure
Piece processing module obtains new picture for handling the strategy processing original image according to picture, and preserving the new picture is simultaneously
The new picture generates image link；The data fusion module is additionally operable to, and is believed according to second structural data and picture
Breath carries out Data Fusion, obtains the structural data for including the pictorial information, the pictorial information includes described original
The image link of the image link of picture and the new picture.
Fourth aspect, an embodiment of the present invention provides a kind of data processing systems for obtaining site resource, including：Root
According to the data processing equipment of the first or second realization method of the third aspect or third aspect of the embodiment of the present invention；Be used for
Preserve the database of second structural data.
5th aspect, an embodiment of the present invention provides a kind of data processing systems for obtaining site resource, including：Root
According to the data processing equipment of the third realization method of the third aspect of the embodiment of the present invention；With, for preserve include the picture
The database of the structural data of information.
6th aspect, an embodiment of the present invention provides a kind of data processing systems for obtaining site resource, including：Root
According to the data processing equipment of the 4th realization method of the third aspect of the embodiment of the present invention, picture processing subsystem and for preserving
Include the database of the structural data of the pictorial information.Wherein, picture processing subsystem, for according to the image link
Crawl original image simultaneously handles the original image to obtain new picture according to picture processing strategy, preserves the new picture
And generate image link for the new picture, and by the picture of the image link comprising the original image and the new picture
The pictorial information of link is sent to the data processing equipment.
Various embodiments using the present invention have the advantages that：
On the one hand, by carrying out Screening Treatment and dissection process to web data during receiving web data, from
And (such as per hour) Data Fusion can be carried out to achieve the purpose that update site resource at regular intervals, this effective gram
The defects of offline batch processing of the prior art leads to data update period length, the poor in timeliness of resource is taken.On the other hand, it is counting
During calculating structural data, captured according to the image link address progress picture parsed in dissection process and subsequent
Picture processing, can improve video resource goes out figure rate, provides better search experience to the user.
Description of the drawings
Fig. 1 is a kind of block diagram for obtaining the data processing system of site resource according to the ... of the embodiment of the present invention；
Fig. 2A is a kind of square for obtaining the data processing system of video website resource according to the ... of the embodiment of the present invention
Figure；
Fig. 2 B are a kind of block diagrams of the picture processing subsystem in Fig. 2A illustrated embodiments；
Fig. 3 is a kind of flow signal for obtaining the data processing method of site resource according to the ... of the embodiment of the present invention
Figure；
Fig. 4 is that a kind of flow of data processing method for obtaining video website resource according to the ... of the embodiment of the present invention is shown
It is intended to；
Fig. 5 is that a kind of flow of data processing method for obtaining video website resource according to the ... of the embodiment of the present invention is shown
It is intended to；
Fig. 6 is a kind of block diagram for obtaining the data processing equipment of site resource according to the ... of the embodiment of the present invention；
Fig. 7 is a kind of block diagram for obtaining the data processing equipment of site resource according to the ... of the embodiment of the present invention；
Fig. 8 is a kind of block diagram for obtaining the data processing equipment of site resource according to the ... of the embodiment of the present invention；
Fig. 9 A-9C are a kind of squares for obtaining the data processing system of site resource according to the ... of the embodiment of the present invention
Figure.
Specific implementation mode
It is described in detail to various aspects of the present invention below in conjunction with the drawings and specific embodiments.Wherein, many institute's weeks
Module, unit and its mutual connection, link, communication or the operation known are not shown or do not elaborate.Also, institute
Feature, framework or the function of description can in any way combine in one or more embodiments.People in the art
Member is it should be appreciated that following various embodiments are served only for the protection domain for example, and is not intended to limit the present invention.May be used also
To be readily appreciated that, module or unit or step in each embodiment described herein and shown in the drawings can be matched by various differences
It sets and is combined and designs.
【First embodiment】
Fig. 1 is a kind of block diagram for obtaining the data processing system of site resource according to the ... of the embodiment of the present invention, ginseng
According to Fig. 1, data processing system 1 includes data screening device 10, web analysis server 20 and database 30, is carried out separately below
Explanation.
Data screening device 10, for receiving the web data captured by web crawlers, and to receiving in receive process
The web data arrived carries out Screening Treatment, will filter out and is sent to web analysis service with the relevant web data of appointed website
Device 20.
Optionally, in a kind of realization method of the present embodiment, data screening device 10 can be directly logical with web crawlers
Believe and continue receive web data, can also with for preserve web crawlers crawl web data database communication and continue
Receive web data, can also with for forwarding the data transfer equipment of web data that web crawlers captured to communicate and continuing
Receive web data.
Optionally, in a kind of realization method of the present embodiment, data screening device 10 can be according to the URL of appointed website
Regular expression carries out Screening Treatment to the web data received, obtains and the relevant web data of appointed website.
Web analysis server 20, for being carried out according to preset parsing strategy pair and the relevant web data of appointed website
Dissection process, obtain with the relevant first structure data of appointed website, and first structure data are preserved to database 30.
Optionally, in a kind of realization method of the present embodiment, web analysis server 20 constantly receives data screening
The web data that device 10 is sent, and dissection process is carried out after receiving web data every time, alternatively, periodically (example
Such as, every one minute) dissection process is carried out to the web data received.
Optionally, in a kind of realization method of the present embodiment, by taking appointed website is video website as an example, web analysis clothes
Being engaged in device 20 can be when receiving web data relevant with the video playing page of appointed website, according to the first parsing strategy progress
Parsing；When receiving web data relevant with the list of videos page of appointed website, according to different from the first parsing strategy
Second parsing strategy is parsed.That is, parsing strategy in the present embodiment may include it is a variety of respectively with parsed
The corresponding parsing of data is tactful and is not limited to a kind of parsing strategy.
Database 30, for being carried out at data fusion according to the first structure data received within a predetermined period of time
Reason, obtains the second structural data for describing appointed website resource.It should be noted that " first structure data " and
It is referred to as other that " first " and " second " referred in " the second structural data " is used only as name, in addition to this, not to structural data
Constitute any restrictions.
Optionally, in a kind of application scenarios of the present embodiment, when the sieve for carrying out web data for multiple appointed websites
When choosing processing, dissection process and Data Fusion, database 30 receive respectively with the relevant first structure of different web sites
Data, and following manner may be used and carry out Data Fusion：
Mode one, database 30 periodically carry out Data Fusion, including：The tool that will be received in current period
There are the first structure data of identical URL to carry out data fusion, is corresponded to the second structural data of different web sites respectively.
Mode two, database 30 periodically carry out Data Fusion, including：It, will be in the period in each period
The first structure data with identical URL inside received are merged to obtain fusion results, then will be at nearest two
Or the fusion results with identical URL that more than two periods obtain blend, and are corresponded to the second knot of different web sites respectively
Structure data.
Mode three, database 30 periodically carry out Data Fusion, including：Have what is received in current period
The first structure data of identical URL carry out data fusion, and the result after fusion is had with what is be calculated in previous cycle
The second structural data of identical URL blends, and obtains the second structural data for corresponding to different web sites respectively in current period.
Optionally, in a kind of realization method of the present embodiment, database 30 be calculated the second structural data it
Afterwards, it is that the foundation of the second structural data is indexed for retrieving on line.
The data processing system 1 provided using the embodiment of the present invention can in real time or in time capture web crawlers
Web data carry out screening and dissection process, so as to carry out at regular intervals Data Fusion reach update website money
The purpose in source, this, which effectively overcomes the offline batch processing of the prior art, leads to data update period length, the poor in timeliness etc. of resource
Defect.In addition, the data processing system 1 that the embodiment of the present invention is provided has built a complete flow chart of data processing, it can
Continuous operation and avoid manpower intervention.
【Second embodiment】
Data processing system 1 shown in FIG. 1 be suitable for obtaining various types of websites (such as：News website, video network
Stand, education and scientific research website, military website etc.) resource.For obtaining video website resource, it is contemplated that showed with graphic form
Video resource can improve user experience, and the present invention still further provides a kind of preferred number for obtaining video website resource
According to processing system, as shown in Figure 2 A, data processing system 2 is in addition to including data screening device 10,20 and of web analysis server
Further include picture processing subsystem 40 outside database 30.It illustrates separately below, wherein although not filled to data screening
It sets 10, web analysis server 20 and database 30 is described in detail, but three can have in the embodiment shown in fig. 1
All features, do not repeat herein.
In the present embodiment, data screening device 10 is for receiving the web data captured by web crawlers, and is receiving
Screening Treatment is carried out to the web data received in the process, will filter out and sent out with the relevant web data in designated website
It send to web analysis server 20.
Web analysis server 20, for according to preset parsing strategy pair and the relevant web data in designated website
Carry out dissection process, obtain with the relevant first structure data in designated website, and by first structure data preserve to
Database 30, and, for the image link parsed in dissection process to be sent to picture processing subsystem 40.
Optionally, in a kind of realization method of the present embodiment, web analysis server 10 is according to page where web data
Whether face includes player, judges that web data is and the relevant web data of video playing page or related with list of videos page
Web data, if it is the former, then according to first parsing strategy parsed (analysis result belongs to first structure data)；
If it is the latter, then according to being parsed from tactful the second different parsing strategy of the first parsing, (analysis result belongs to the first knot
Structure data).Wherein, pair include image link with the analysis result of the relevant web data of list of videos page, for example, comprising from
The image link parsed in web page source code.
In the present embodiment, picture processing subsystem 40 is for executing following processing：Original graph is captured according to image link
Piece simultaneously handles original image to obtain new picture according to picture processing strategy；It preserves new picture and generates picture for new picture
Link；And the pictorial information of the image link comprising original image and the image link of new picture is sent to database 30.
Using data processing system 2 provided in this embodiment, picture processing is carried out by picture processing subsystem 40, it can
Obtain with the relevant image data in designated website, convenient for subsequently need show video resource picture when be called or
Carry out Data Fusion.
Optionally, in a kind of realization method of the present embodiment, as shown in Figure 2 B, picture processing subsystem 40 may include figure
Piece captures server 41, picture processing server 42 and picture database 43.
Picture crawl server 41 is used to receive image link (the i.e. figure of original image of the transmission of web analysis server 20
Piece links), original image is captured according to image link, and original image and its image link are sent to picture processing server
42。
Picture processing server 42 handles original image to obtain new picture for handling strategy according to picture, and
The image link of original image and new picture are preserved to picture database 43.
Illustratively, picture processing server 42 can be used following manner and handle original image：First to original
Picture is analyzed, and identifies the Two-Dimensional Moment system of battle formations of its pixel to obtain the length and width information of picture；Then, according to presetting
Good rule the operations such as compresses original image, is cut and obtaining new picture, and the new picture that makes that treated, which meets, shows requirement.
Picture database 43, for generating image link for new picture, and by the image link comprising original image and newly
The pictorial information of the image link of picture is sent to database 30.
Optionally, in a kind of realization method of the present embodiment, database 30 is in addition to being used for according within a predetermined period of time
The first structure data received are calculated except the second structural data, according to the second structural data and can also connect
The pictorial information received carries out Data Fusion.For example, being inscribed for the second structural data and in the predetermined amount of time
Data wherein with identical URL are carried out Data Fusion by the pictorial information received.Using this realization method, with difference
Process executes the calculating of the calculating and pictorial information of structural data, can improve picture treatment effeciency, to improve video money
Source goes out figure rate, provides better search experience to the user.
【3rd embodiment】
Data processing system according to the ... of the embodiment of the present invention is illustrated above in association with attached drawing, it is right below in conjunction with the accompanying drawings
Data processing method according to the ... of the embodiment of the present invention illustrates.
Fig. 3 is a kind of flow signal for obtaining the data processing method of site resource according to the ... of the embodiment of the present invention
Figure, reference Fig. 3, the method includes：
300：The web data captured by web crawlers is received, and the web data received is carried out in receive process
Screening Treatment obtains and the relevant web data of appointed website.
Optionally, in a kind of realization method of the present embodiment, during receiving web data, according to appointed website
URL regular expressions Screening Treatment is carried out to the web data that receives.
302：Dissection process is carried out according to preset parsing strategy pair and the relevant web data of appointed website, obtains and refers to
Determine the relevant first structure data in website.
Wherein, a kind of parsing strategy may be used for same or same class website web data, it is also possible to use
A variety of parsing strategies.For example, for the web data of news website, a kind of parsing strategy may be used and parsed；For regarding
The web data of frequency website still can use not with list of videos page correlation according to web data is related to video playing page
Same parsing strategy is parsed.
304：Data Fusion is carried out according to the first structure data parsed within a predetermined period of time, is used
In the second structural data of the resource of description appointed website.
Optionally, it in a kind of application scenarios of the present embodiment, can periodically carry out at data fusion in 304
Reason, concrete mode please refer to the previously described three kinds of modes of (but not limited to), do not repeat herein.
Optionally, in a kind of realization method of the present embodiment, after 304, index is established for the second structural data
For being retrieved on line.
In a kind of specific implementation of the present embodiment, 300 are executed by data screening device 10 and is sent out implementing result
Web analysis server 20 is given, 302 are executed by web analysis server 20 and implementing result is sent to database 30, then
304 are executed by database 30.Wherein, various pieces execute the detailed process of each step and refer to description above, do not go to live in the household of one's in-laws on getting married herein
It states.
The data processing method provided using the embodiment of the present invention, by real time or in time to web crawlers crawl
Web data carries out screening and dissection process, can carry out the mesh that Data Fusion reaches update site resource at regular intervals
, this effectively overcomes the defects of offline batch processing of the prior art leads to data update period length, the poor in timeliness of resource.
【Fourth embodiment】
Fig. 4 is that a kind of flow of data processing method for obtaining video website resource according to the ... of the embodiment of the present invention is shown
It is intended to, reference Fig. 4, the method includes：
400：The web data captured by web crawlers is received, and the web data received is carried out in receive process
Screening Treatment obtains and the relevant web data in designated website.
402：Dissection process is carried out according to preset parsing strategy pair and the relevant web data in designated website, is obtained
With the relevant first structure data in designated website.
Optionally, in a kind of realization method of the present embodiment, when with the relevant web data in designated website be with
When the relevant web data of the video playing page of designated website, dissection process is carried out according to the first parsing strategy；When with finger
When to determine the relevant web data of video website be web data relevant with the list of videos page of designated website, according to the
The second different parsing strategy of one parsing strategy carries out dissection process.
404：Data Fusion is carried out according to the first structure data parsed within a predetermined period of time, is used
In the second structural data of the resource of description appointed website.
406:The image link parsed during dissection process is sent to picture processing subsystem.At the picture
Reason subsystem is for executing following processing：Original image is captured according to image link and strategy is handled to original image according to picture
It is handled to obtain new picture, preserve new picture and generates image link for new picture.
Wherein, the explanation in Fig. 2A and Fig. 2 B illustrated embodiments may refer to for the explanation of picture processing subsystem, this
Place does not repeat.
408：The pictorial information that picture processing subsystem is sent is received, the pictorial information includes the picture chain of original image
Connect the image link with new picture.
410：Data Fusion is carried out according to the second structural data and pictorial information.Optionally, with the second structuring
The pictorial information that data are merged is the pictorial information received in the predetermined amount of time.
In the present embodiment, be not intended to limit 404 and 406-408 executes sequence, or even a kind of deformation in the present embodiment
In example, 404 can be accomplished by the following way simultaneously with 410：According to the first structure parsed within a predetermined period of time
Data and the pictorial information received carry out Data Fusion, obtain the structural data for including pictorial information.
The same or similar step of embodiment as shown in figure 3 in the present embodiment can realization method having the same, this
Place does not repeat.
In a kind of specific implementation of the present embodiment, 400 are executed by data screening device 10 and is sent out implementing result
Web analysis server 20 is given, 402 are executed by web analysis server 20 and implementing result is sent to database 30, by net
The page execution 404 of resolution server 20, will by picture processing subsystem 40 image link is sent to picture processing subsystem 40
Pictorial information is sent to database 30 (that is, executing 408 by database 30), and 406 and 410 are executed by database 30.Wherein, each
The detailed process that part executes each step refers to description above, does not repeat herein.
The data processing method provided using the embodiment of the present invention, other than having the advantages that embodiment illustrated in fig. 3,
Video resource can also be improved goes out figure rate, improves user experience.
【5th embodiment】
Fig. 5 is that a kind of flow of data processing method for obtaining video website resource according to the ... of the embodiment of the present invention is shown
It is intended to, reference Fig. 5, the method includes：
500：The web data captured by web crawlers is received, and the web data received is carried out in receive process
Screening Treatment obtains and the relevant web data in designated website.
502：Dissection process is carried out according to preset parsing strategy pair and the relevant web data in designated website, is obtained
With the relevant first structure data in designated website.
504：Data Fusion is carried out according to the first structure data parsed within a predetermined period of time, is used
In the second structural data of the resource of description appointed website.
506：Original image is captured according to the image link parsed in dissection process.
508：Strategy is handled according to picture original image is handled to obtain new picture, preserve new picture and be new picture
Generate image link.
510：Data Fusion is carried out according to the second structural data and pictorial information, obtains the knot for including pictorial information
Structure data.Pictorial information includes the image link of the image link and new picture of original image.
In the present embodiment, be not intended to limit 504 and 506-508 executes sequence, or even a kind of deformation in the present embodiment
In example, 504 can be accomplished by the following way simultaneously with 510：According to the first structure parsed within a predetermined period of time
Data and the pictorial information received carry out Data Fusion, obtain the structural data for including pictorial information.
In the present embodiment and the same or similar step of Fig. 3 and embodiment illustrated in fig. 4 can realization side having the same
Formula is not repeated herein.
In a kind of specific implementation of the present embodiment, 500 are executed by data screening device 10 and is sent out implementing result
Web analysis server 20 is given, 502 are executed by web analysis server 20 and implementing result is sent to database 30, by net
Image link is sent to picture crawl server 41 and executes 506 to capture server 41 by picture by page resolution server 20, by
Picture processing server 42 and picture database 43 execute 508 and pictorial information are sent to database 30, are held by database 30
Row 504 and 510.Wherein, various pieces execute the detailed process of each step and refer to description above, do not repeat herein.
The data processing method provided using the embodiment of the present invention, other than having the advantages that embodiment illustrated in fig. 3,
Video resource can also be improved goes out figure rate, improves user experience.
【Sixth embodiment】
Below with acquisition " http:The present invention is said for this video website resource of //www.bugaboo.tv "
Bright, the features such as Rule of judgment, specific processing mode for being referred in following citing may be incorporated for Fig. 1 to embodiment illustrated in fig. 5
In.
First, data screening device 10 receive web crawlers return web data, by bugaboo.tv/ (watch |
Video)/.* screens URL, gets the webpage of the video playing page A and list of videos page B of bugaboo.tv websites
Data, and the web data of acquisition is sent to web analysis server 20.
Then, web analysis server 20 is loaded into preset parsing strategy, by judging that the page is known with the presence or absence of player
Do not go out that A is video playing page, B is list of videos page.Corresponding parsing strategy is applied mechanically respectively to the web data of the A pages and B pages
The web data in face carries out the extraction of structured message, and the A pages can extract data C (such as table one), including title, abstract,
Time etc.；The B pages can extract data D, including 21 list factors (such as table two), each list factor includes corresponding resource
URL, title, image link, broadcasting time.
Table one：Data C exemplary plots
Table two：First factor schematic diagram of data D
Web analysis server 20 obtains the image link (21 total) of 21 factors in data D, sends it to figure
Piece captures server 41.Meanwhile web analysis server 20 stamps the type mark for distinguishing the two for data C and data D,
And data C and data D are sent to database 30.
Data C is entered library storage by database 30 after receiving data C and data D, according to type mark, to data D into
Row is decomposed and is formatted, and is 21 datas (such as table three) by 21 Factorizations, 21 datas are entered library storage later.
Table three：Example after first Factorization of data D
Database 30 at regular intervals merges data of the full library with identical URL, such as table of the result after fusion
Shown in four：
Table four：Example after first factor of data A and data D merges (after decomposition)
Picture captures server 41 after receiving image link, according to image link (for example, http://i.bug-a-
Boo.tv/images/xxx.jpg picture crawl) is carried out, jpg picture files (i.e. original image) are got, by the jpg of acquisition
Picture file and image link are sent to picture processing server 42.
Picture processing server 42 processing such as parses jpg picture files, is cut and obtaining new picture file, and will be new
The image link of picture file, original image file and original image file sends and preserves to picture database 43.
Picture database 43 is that new picture file generates new image link NEW_URL, and by the picture chain comprising new picture
It connects and the pictorial information of the image link of original image (such as table five) is sent to database 30.
| Original image URL | New picture URL |
| http://i.bug-a-boo.tv/images/xxx.jpg | NEW_URL |
Table five：Pictorial information
After database 30 receives pictorial information, data fusion is carried out according to URL, obtains data A, data D and pictorial information
Fusion results (such as table six).
Table six：The few examples of data after fusion
【7th embodiment】
Fig. 6 is a kind of block diagram for obtaining the data processing equipment of site resource according to the ... of the embodiment of the present invention, ginseng
According to Fig. 6, data processing equipment 6 includes data screening module 61, data resolution module 62 and data fusion module 63.Separately below
It illustrates.
Data screening module 61, for receiving the web data captured by web crawlers, and to receiving in receive process
The web data arrived carries out Screening Treatment, obtains and the relevant web data of appointed website.For example, according to the URL of appointed website
Regular expression carries out Screening Treatment to obtain and the relevant web data of appointed website to the web data received.
Data resolution module 62, for being solved with the relevant web data of appointed website according to preset parsing strategy pair
Analysis is handled, and is obtained and the relevant first structure data of appointed website.
Optionally, in a kind of realization method of the present embodiment, as illustrated with the dotted box, data resolution module includes：
First analyzing sub-module 621, for appointed website be video website and with the relevant webpage number of appointed website
According to for web data relevant with the video playing page of appointed website when, according to first parsing strategy carry out dissection process；With
Two analyzing sub-modules 622, for being video website in appointed website and with the relevant web data of appointed website being and specified
When the relevant web data of the list of videos page of website, parsed according to the second parsing strategy different from the first parsing strategy
Processing.
Data fusion module 63, for carrying out data according to the first structure data parsed within a predetermined period of time
Fusion treatment obtains the second structural data of the resource for describing appointed website.
In the present embodiment, modules can be used for executing the corresponding steps or corresponding in Fig. 3 to embodiment illustrated in fig. 5
The optional realization method of step, explanation and limitation for corresponding steps in Fig. 3 to Fig. 5 are equally applicable to each in the present embodiment
The explanation of processing performed by a module and limitation.
The data processing equipment 6 provided using the present embodiment can shorten the update cycle of site resource, improve website
The timeliness of resource.
【8th embodiment】
Fig. 7 is a kind of block diagram for obtaining the data processing equipment of site resource according to the ... of the embodiment of the present invention, ginseng
According to Fig. 7, data processing equipment 7 other than including data screening module 61, data resolution module 62 and data fusion module 63,
Further include image link sending module 71 and pictorial information receiving module 72.
The image link that image link sending module 71 is used to parse data resolution module 62 in dissection process is sent out
It send to picture processing subsystem (for example, picture processing subsystem 40).Wherein, picture processing subsystem is for executing following place
Reason：It captures original image according to image link and strategy is handled according to picture and original image is handled to obtain new picture, protect
It deposits new picture and generates image link for new picture, and by the image link of the image link comprising original image and new picture
Pictorial information be sent to data processing equipment 7.
Pictorial information receiving module 72, the pictorial information sent for receiving picture processing subsystem.
Data fusion module 63 is additionally operable to other than executing processing performed in the embodiment shown in fig. 6 according to the
Two structural datas and pictorial information carry out Data Fusion, obtain the structural data for including pictorial information.
In the present embodiment, modules can be used for executing the corresponding steps or corresponding steps in embodiment illustrated in fig. 4
Optional realization method, explanation and limitation for corresponding steps in Fig. 4 are equally applicable to modules institute in the present embodiment
The explanation of the processing of execution and limitation.
The data processing equipment 7 provided using the present embodiment, other than it can improve the timeliness of site resource, needle
To video website, moreover it is possible to which that improves its video resource goes out figure rate, improves user experience.
【9th embodiment】
Fig. 8 is a kind of block diagram for obtaining the data processing equipment of site resource according to the ... of the embodiment of the present invention, ginseng
According to Fig. 8, data processing equipment 8 other than including data screening module 61, data resolution module 62 and data fusion module 63,
Further include picture handling module 81 and picture processing module 82.
Picture handling module 81 is used to be captured according to the image link that data resolution module 62 parses in dissection process
Original image.
Picture processing module 82 obtains new picture for handling strategy processing original image according to picture, preserves new picture
And generate image link for new picture.For example, new picture is preserved into database and generates image link for new picture.
Data fusion module 63 is additionally operable to other than executing processing performed in the embodiment shown in fig. 6 according to the
Two structural datas and pictorial information carry out Data Fusion, obtain the structural data for including pictorial information.The picture
Information includes the image link of the image link and new picture of original image.
In the present embodiment, modules can be used for executing the corresponding steps or corresponding steps in embodiment illustrated in fig. 5
Optional realization method, explanation and limitation for corresponding steps in Fig. 5 are equally applicable to modules institute in the present embodiment
The explanation of the processing of execution and limitation.
The data processing equipment 8 provided using the present embodiment, other than it can improve the timeliness of site resource, needle
To video website, moreover it is possible to which that improves its video resource goes out figure rate, improves user experience.
【Tenth embodiment】
Fig. 9 A-9C are a kind of squares for obtaining the data processing system of site resource according to the ... of the embodiment of the present invention
Figure.
In an embodiment of the present invention, as shown in Figure 9 A, data processing system includes previously described data processing
Device 6 and database for preserving the second structural data.Wherein, the second structure that data processing equipment 6 will be calculated
Change data send and preserve to database, to update the data in library with the relevant data of site resource.
In another embodiment of the invention, as shown in Figure 9 B, data processing system includes at previously described data
Reason device 7, the database for preserving the structural data for including pictorial information and previously described picture processing subsystem
40.Wherein, the structural data comprising pictorial information being calculated is sent and is preserved to database by data processing equipment 7,
To update the data in library with the relevant data of site resource.
In the another embodiment of the present invention, as shown in Figure 9 C, data processing system includes at previously described data
Manage device 8 and the database for preserving the structural data for including pictorial information.Wherein, data processing equipment 8 will calculate
To the structural data comprising pictorial information send and preserve to database, it is related to site resource in library to update the data
Data.
Through the above description of the embodiments, those skilled in the art can be understood that the present invention can be by
The mode of software combination hardware platform is realized.Based on this understanding, technical scheme of the present invention makes tribute to background technology
That offers can be expressed in the form of software products in whole or in part, which can be stored in storage and be situated between
In matter, such as ROM/RAM, magnetic disc, CD, including some instructions use is so that a computer equipment (can be individual calculus
Machine, server, smart mobile phone either network equipment etc.) it executes described in certain parts of each embodiment of the present invention or embodiment
Method.
The term and wording used in description of the invention is just to for example, be not intended to constitute restriction.Ability
Field technique personnel should be appreciated that under the premise of not departing from the basic principle of disclosed embodiment, to the above embodiment
In each details can carry out various change.Therefore, the scope of the present invention is only determined by claim, in the claims, unless
It is otherwise noted, all terms should be understood by the broadest rational meaning.
Claims (14)
1. a kind of data processing system for obtaining site resource, which is characterized in that including：
Data screening device, for receiving the web data captured by web crawlers, and the net in receive process to receiving
Page data carries out Screening Treatment, will filter out and is sent to web analysis server with the relevant web data of appointed website；
Web analysis server, for being solved with the relevant web data of the appointed website according to preset parsing strategy pair
Analysis is handled, obtain with the relevant first structure data of the appointed website, and the first structure data are preserved to number
According to library；
The database, for being carried out at data fusion according to the first structure data received within a predetermined period of time
Reason, obtains the second structural data of the resource for describing the appointed website；
The web analysis server is additionally operable to, and the image link parsed in the dissection process is sent to picture processing
Subsystem；
The picture processing subsystem, for capturing original image according to the image link and handling strategy to institute according to picture
It states original image to be handled to obtain new picture, preserve the new picture and generates image link for the new picture, and will
Including the pictorial information of the image link of the original image and the image link of the new picture is sent to the database.
2. data processing system as described in claim 1, which is characterized in that
The data screening device is specifically used for, and during receiving web data, unified according to the appointed website provides
Source finger URL URL regular expressions carry out Screening Treatment to the web data received.
3. data processing system as described in claim 1, which is characterized in that when the appointed website is video website, institute
Web analysis server is stated to be specifically used for：
It is the relevant net of video playing page with the appointed website in the web data that the web analysis server receives
When page data, dissection process is carried out according to the first parsing strategy；
It is the relevant net of list of videos page with the appointed website in the web data that the web analysis server receives
When page data, dissection process is carried out according to the second parsing strategy different from the first parsing strategy.
4. data processing system as described in claim 1, which is characterized in that the picture processing subsystem includes picture crawl
Server, picture processing server and picture database, wherein
The picture captures server, for capturing the original image according to the image link, and by the original image
And its image link is sent to the picture processing server；
The picture processing server handles the original image to obtain the new figure for handling strategy according to picture
Piece, and the image link of the original image and the new picture are preserved to the picture database；
The pictorial information for generating image link for the new picture, and is sent to the number by the picture database
According to library.
5. data processing system as described in claim 1, which is characterized in that
Shown database is additionally operable to, and Data Fusion is carried out according to second structural data and the pictorial information.
6. a kind of data processing method for obtaining site resource, which is characterized in that including：
The web data captured by web crawlers is received, and the web data received is carried out at screening in receive process
Reason, obtains and the relevant web data of appointed website；
Dissection process is carried out according to preset parsing strategy pair and the relevant web data of the appointed website, is obtained and the finger
Determine the relevant first structure data in website；
Data Fusion is carried out according to the first structure data parsed within a predetermined period of time, is obtained for retouching
State the second structural data of the resource of the appointed website；
The method further includes：
The image link parsed during the dissection process is sent to picture processing subsystem, picture processing
System is for executing following processing：Original image is captured according to the image link and strategy is handled to described original according to picture
Picture is handled to obtain new picture, is preserved the new picture and is generated image link for the new picture；
The pictorial information sent by the picture processing subsystem is received, the pictorial information includes the picture of the original image
The image link of link and the new picture；
Data Fusion is carried out according to second structural data and the pictorial information.
7. data processing method as claimed in claim 6, which is characterized in that the webpage in receive process to receiving
Data carry out Screening Treatment：
During receiving web data, according to the URL regular expressions of the appointed website to the web data that receives
Carry out Screening Treatment.
8. data processing method as claimed in claim 6, which is characterized in that when the appointed website is video website, institute
It states and includes according to preset parsing strategy pair and the relevant web data progress dissection process of the appointed website：
When being the relevant web data of video playing page with the appointed website with the relevant web data of the appointed website
When, the dissection process is carried out according to the first parsing strategy；
When being the relevant web data of list of videos page with the appointed website with the relevant web data of the appointed website
When, the dissection process is carried out according to the second parsing strategy different from the first parsing strategy.
9. the data processing method as described in any one of claim 6-8, which is characterized in that the method further includes：
Original image is captured according to the image link parsed in the dissection process；
Strategy is handled according to picture the original image is handled to obtain new picture, preserve the new picture and preserve and be institute
It states new picture and generates image link；
Data Fusion is carried out according to second structural data and pictorial information, the pictorial information includes described original
The image link of the image link of picture and the new picture.
10. a kind of data processing equipment for obtaining site resource, which is characterized in that including：
Data screening module, for receiving the web data captured by web crawlers, and the net in receive process to receiving
Page data carries out Screening Treatment, obtains and the relevant web data of appointed website；
Data resolution module, for being parsed with the relevant web data of the appointed website according to preset parsing strategy pair
Processing, obtains and the relevant first structure data of the appointed website；
Data fusion module melts for carrying out data according to the first structure data parsed within a predetermined period of time
Conjunction is handled, and obtains the second structural data of the resource for describing the appointed website；
The data processing equipment further includes：
Image link sending module, the image link hair for parsing the data resolution module in the dissection process
It send to picture processing subsystem；Wherein, the picture processing subsystem is for executing following processing：It is grabbed according to the image link
It takes original image and strategy is handled according to picture and the original image is handled to obtain new picture, preserve the new picture simultaneously
Image link is generated for the new picture, and by the picture chain of the image link comprising the original image and the new picture
The pictorial information connect is sent to the data processing equipment,
Pictorial information receiving module, for receiving the pictorial information；
The data fusion module is additionally operable to, and is carried out at data fusion according to second structural data and the pictorial information
Reason, obtains the structural data for including the pictorial information.
11. data processing equipment as claimed in claim 10, which is characterized in that
The data screening module is specifically used for, during receiving web data, according to the URL canonicals of the appointed website
Expression formula carries out Screening Treatment to the web data received.
12. data processing equipment as claimed in claim 10, which is characterized in that the data resolution module includes：
First analyzing sub-module, for the appointed website be video website and with the relevant webpage number of the appointed website
According to for web data relevant with the video playing page of the appointed website when, according to first parsing strategy carry out dissection process；
Second analyzing sub-module, for the appointed website be video website and with the relevant webpage number of the appointed website
According to for web data relevant with the list of videos page of the appointed website when, according to different from the first parsing strategy the
Two parsing strategies carry out dissection process.
13. the data processing equipment as described in any one of claim 10-12, which is characterized in that
The data processing equipment further includes：
Picture handling module, the image link crawl for being parsed in the dissection process according to the data resolution module
Original image, and
Picture processing module obtains new picture for handling the strategy processing original image according to picture, preserves the new figure
Piece simultaneously generates image link for the new picture；
The data fusion module is additionally operable to, and Data Fusion is carried out according to second structural data and pictorial information,
Obtain including the structural data of the pictorial information, the pictorial information includes the image link of the original image and described
The image link of new picture.
14. a kind of data processing system for obtaining site resource, which is characterized in that
The data processing system includes：
Data processing equipment as described in any one of claim 10-13, and,
Database for preserving second structural data；
Or, the data processing system includes：
Data processing equipment as claimed in claim 10,
Database for preserving the structural data for including the pictorial information, and
Picture processing subsystem, for capturing original image according to the image link and handling strategy to the original according to picture
Beginning picture is handled to obtain new picture, is preserved the new picture and is generated image link for the new picture, and will include
The pictorial information of the image link of the original image and the image link of the new picture is sent to the data processing equipment；
Or, the data processing system includes：
Data processing equipment as claimed in claim 13, and
Database for preserving the structural data for including the pictorial information.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410521135.2A CN104281680B (en) | 2014-09-30 | 2014-09-30 | Data processing system, method and device for obtaining site resource |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410521135.2A CN104281680B (en) | 2014-09-30 | 2014-09-30 | Data processing system, method and device for obtaining site resource |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN104281680A CN104281680A (en) | 2015-01-14 |
| CN104281680B true CN104281680B (en) | 2018-08-21 |
Family
ID=52256553
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201410521135.2A Active CN104281680B (en) | 2014-09-30 | 2014-09-30 | Data processing system, method and device for obtaining site resource |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN104281680B (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106327039A (en) * | 2015-06-25 | 2017-01-11 | 中兴通讯股份有限公司 | Weekly report information processing method and apparatus |
| CN107180054B (en) * | 2016-03-11 | 2020-05-12 | 阿里巴巴集团控股有限公司 | Data processing method and device |
| CN108228667A (en) * | 2016-12-22 | 2018-06-29 | 钢钢网电子商务（上海）股份有限公司 | A kind of integration method and system of iron and steel resource data information |
| CN115221453B (en) * | 2022-09-20 | 2023-03-10 | 太平金融科技服务(上海)有限公司深圳分公司 | Media resource management method, device, server and medium |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102298622A (en) * | 2011-08-11 | 2011-12-28 | 中国科学院自动化研究所 | Search method for focused web crawler based on anchor text and system thereof |
| CN102622443A (en) * | 2012-03-13 | 2012-08-01 | 北京邮电大学 | Customized screening system and method for microblog |
| CN102930059A (en) * | 2012-11-26 | 2013-02-13 | 电子科技大学 | Method for designing focused crawler |
-
2014
- 2014-09-30 CN CN201410521135.2A patent/CN104281680B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102298622A (en) * | 2011-08-11 | 2011-12-28 | 中国科学院自动化研究所 | Search method for focused web crawler based on anchor text and system thereof |
| CN102622443A (en) * | 2012-03-13 | 2012-08-01 | 北京邮电大学 | Customized screening system and method for microblog |
| CN102930059A (en) * | 2012-11-26 | 2013-02-13 | 电子科技大学 | Method for designing focused crawler |
Also Published As
| Publication number | Publication date |
|---|---|
| CN104281680A (en) | 2015-01-14 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN104281680B (en) | Data processing system, method and device for obtaining site resource | |
| CN104301393B (en) | Laboratory data gathers and management system | |
| CN104426985B (en) | Show the method, apparatus and system of webpage | |
| CN104615627B (en) | A kind of event public feelings information extracting method and system based on microblog | |
| CN104135507B (en) | A kind of method and apparatus of door chain | |
| CN104408102B (en) | For network hot word and the data processing method and device of the degree of association of object | |
| CN109033115A (en) | A kind of dynamic web page crawler system | |
| CN106534145B (en) | A kind of application and identification method and equipment | |
| CN103455600B (en) | A kind of video URL grasping means, device and server apparatus | |
| CN105302815B (en) | The filter method and device of the uniform resource position mark URL of webpage | |
| CN108021604A (en) | A kind of web crawlers method for crawling barrage in Dou Yu webcast websites main broadcaster room | |
| CN103618766B (en) | A kind of method and web game interactive server for carrying out web game interaction | |
| CN108959539B (en) | Rule-configurable webpage data analysis method | |
| CN107766234A (en) | A kind of assessment method, the apparatus and system of the webpage health degree based on mobile device | |
| CN107766509A (en) | A kind of method and apparatus of webpage static backup | |
| CN109189214A (en) | Mobile device-based augmented reality interactive system, device and method | |
| CN110188717A (en) | Image acquiring method and device | |
| CN104967698B (en) | A kind of method and apparatus crawling network data | |
| CN106454249A (en) | Device for simulating multipath high-definition real-time audio and video transmission and method thereof | |
| CN103530337B (en) | Identify the device and method of Invalid parameter in uniform resource position mark URL | |
| CN109408669A (en) | A kind of content auditing method and device for different application scene | |
| CN104580127B (en) | Method for processing business, server and client | |
| CN106529456A (en) | Information matching and information transmitting/receiving method, device and target object finding system | |
| CN108280228A (en) | A kind of processing method and relevant device of webpage | |
| CN111061807A (en) | Distributed data acquisition and analysis system and method, server and medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| EE01 | Entry into force of recordation of patent licensing contract |
Application publication date: 20150114 Assignee: Beijing Intellectual Property Management Co.,Ltd. Assignor: BEIJING BAIDU NETCOM SCIENCE AND TECHNOLOGY Co.,Ltd. Contract record no.: X2023110000095 Denomination of invention: Data processing system, method, and device for obtaining website resources Granted publication date: 20180821 License type: Common License Record date: 20230821 |
|
| EE01 | Entry into force of recordation of patent licensing contract |