CN107103079A - The live broadcasting method and system of a kind of dynamic website - Google Patents
The live broadcasting method and system of a kind of dynamic website Download PDFInfo
- Publication number
- CN107103079A CN107103079A CN201710278347.6A CN201710278347A CN107103079A CN 107103079 A CN107103079 A CN 107103079A CN 201710278347 A CN201710278347 A CN 201710278347A CN 107103079 A CN107103079 A CN 107103079A
- Authority
- CN
- China
- Prior art keywords
- dynamic website
- user
- latest data
- authorization message
- website
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/23—Updating
- G06F16/2358—Change logging, detection, and notification
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/25—Integrating or interfacing systems involving database management systems
- G06F16/252—Integrating or interfacing systems involving database management systems between a Database Management System and a front-end application
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/95—Retrieval from the web
- G06F16/951—Indexing; Web crawling techniques
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
- H04N21/85—Assembly of content; Generation of multimedia applications
- H04N21/854—Content authoring
- H04N21/8543—Content authoring using a description language, e.g. Multimedia and Hypermedia information coding Expert Group [MHEG], eXtensible Markup Language [XML]
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computer Security & Cryptography (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Information Transfer Between Computers (AREA)
Abstract
The invention discloses a kind of live broadcasting method of dynamic website and system, this method includes：According to the authorization message that user is selected in dynamic website, the corresponding permanent token for logging in dynamic website of authorization message is obtained；A reptile robot is distributed for each token, the current latest data of the pre-set user of dynamic website is obtained using reptile robot, and current latest data is stored to database；The current latest data stored in database is monitored, the latest data that pre-set user updates is obtained, and latest data is sent to default live platform；The present invention is by obtaining the permanent token for logging in dynamic website, access of the dynamic website to reptile robot is solved to limit, by distributing a reptile robot for each token, multiple reptile robots can be used to capture the current latest data of the pre-set user of dynamic website, the effective dynamic website avoided such as Sina weibo improves live real-time to crawling the limitation of frequency.
Description
Technical field
The present invention relates to Computer Applied Technology field, the live broadcasting method and system of more particularly to a kind of dynamic website.
Background technology
With the development of modern society's science and technology, people are to the demand more and more higher of life, and Crawl Robot also should by demand
For the various aspects of life, such as live broadcast system.Crawl Robot are also referred to as reptile robot, Web Spider or network
Robot, it is that search engine downloads webpage from internet, and gathers resource from related web page interlinkage, is One function
The automatic capture program of very strong webpage.At the beginning of this technology is born, it is mainly used among search engine, is to determine one
The important composition part of individual search engine search performance and autgmentability.
In the prior art, most of live broadcast systems often towards static website, and towards be exempt from log in website.
And with the security more and more higher of website at this stage, such as this kind of dynamic website of Sina weibo is past when Website login
Just have no idea to climb using traditional toward offer username and password and identifying code, so, traditional live broadcast system is needed
Worm robot is towards the live relevant information of this kind of dynamic website.Also, Sina weibo is also monitored to association requests, at certain
Request of one period from a certain IP excessively frequently can carry out denied access for a period of time to its IP.Therefore, it is traditional live
System can not carry out live to this kind of dynamic website of Sina weibo.
This allows for needing related personnel's acquisition up-to-date information to need constantly to go refresh page to obtain target data, so
And such working method not only greatly consumes the energy of people, moreover, being difficult to avoid that intolerable delay.For example,
There are many big V can real-time live broadcast quotations on the stock market and related stock information, conventional people's acquisition among Sina weibo to blog
Relevant information can only be obtained by refresh page, and such shortcoming often not only consumes manpower, and often miss pass
The important information of key.Therefore, how using reptile robot the data in dynamic website are carried out with live, the live delay of reduction
Time, Consumer's Experience is improved, be urgent problem now.
The content of the invention
It is an object of the invention to provide a kind of live broadcasting method of dynamic website and system, to utilize reptile robot to dynamic
Data in website carry out live, reduction live time delay, raising Consumer's Experience.
In order to solve the above technical problems, the present invention provides a kind of live broadcasting method of dynamic website, including：
According to the authorization message that user is selected in dynamic website, the acquisition authorization message is corresponding forever to log in described move
The token of state website；Wherein, the authorization message includes the log-on message that corresponding selected user logs in dynamic website；
A reptile robot is distributed for each token, the dynamic website is obtained using the reptile robot
The current latest data of pre-set user, and the current latest data is stored to database；
The current latest data stored in the database is monitored, the pre-set user is obtained and updates most
New data, and the latest data is sent to default live platform.
Optionally, it is described according to the authorization message that user is selected in dynamic website, obtain the authorization message corresponding forever
The token of the dynamic website is logged in long, including：
According to the authorization message, forever stepped on by the way that the Oauth2.0 protocol emulations user acquisition authorization message is corresponding
The token of Lu Suoshu dynamic websites.
Optionally, it is described according to the authorization message, the authorization message is obtained by Oauth2.0 protocol emulations user
Before the corresponding token for forever logging in the dynamic website, in addition to：
The authorization requests for obtaining the authorization message are sent to the selected user；
Obtain the authorization message that the selected user returns.
Optionally, the current newest number of the pre-set user that the dynamic website is obtained using the reptile robot
According to, including：
The reptile robot of predetermined number is distributed for each pre-set user；
The corresponding reptile robot of each pre-set user obtained successively in respective preset time it is described it is current most
New data.
Optionally, this method also includes：
By fixed platform website according to pre-set level to the corresponding whole latest data of each pre-set user
Counted and show statistical result.
In addition, present invention also offers a kind of live broadcast system of dynamic website, including：
Access mandate module, for according to the authorization message that user is selected in dynamic website, obtaining the authorization message pair
That answers forever logs in the token of the dynamic website；Wherein, the authorization message logs in Dynamic Networks including corresponding selected user
The log-on message stood；
Data acquisition module, for distributing a reptile robot for each token, utilizes the reptile robot
The current latest data of the pre-set user of the dynamic website is obtained, and the current latest data is stored to database；
The database, for storing the corresponding current latest data of each pre-set user；
Monitor module, for being monitored to the current latest data stored in the database, obtains described
The latest data that pre-set user updates, and the latest data is sent to default live platform.
Optionally, the access mandate module, including：
Token acquiring unit, for according to the authorization message, by being awarded described in Oauth2.0 protocol emulations user acquisition
Weigh the corresponding token for forever logging in the dynamic website of information.
Optionally, the access mandate module, including：
Transmitting element, the authorization requests of the authorization message are obtained for being sent to the selected user；
Receiving unit, for obtaining the authorization message that the selected user returns.
Optionally, the data acquisition module, including：
Allocation unit, the reptile robot for distributing predetermined number for each pre-set user；
Acquiring unit, is obtained in respective preset time successively for the corresponding reptile robot of each pre-set user
Take the current latest data.
Optionally, the system also includes：
Statistical module, for by fixed platform website according to pre-set level to the corresponding whole of each pre-set user
The latest data is counted and shows statistical result.
A kind of live broadcasting method of dynamic website provided by the present invention, including：Awarded according to user is selected in dynamic website
Information is weighed, the corresponding token for forever logging in the dynamic website of the authorization message is obtained；Wherein, the authorization message includes
Corresponding selected user logs in the log-on message of dynamic website；A reptile robot is distributed for each token, institute is utilized
The current latest data that reptile robot obtains the pre-set user of the dynamic website is stated, and the current latest data is stored
To database；The current latest data stored in the database is monitored, obtains what the pre-set user updated
Latest data, and the latest data is sent to default live platform.
It can be seen that, the present invention is by according to the authorization message that user is selected in dynamic website, obtaining authorization message corresponding forever
The token of dynamic website is logged in long, and reptile robot can be made forever to log in dynamic website by token, dynamic website is solved
Access limitation to reptile robot, by distributing a reptile robot for each token, obtains dynamic using reptile robot
The current latest data of the pre-set user of state website, can use multiple reptile robots working as to the pre-set user of dynamic website
Preceding latest data is captured, and effectively avoids the dynamic website of Sina weibo such as and the limit of frequency is crawled to reptile robot
System, improves live real-time, improves Consumer's Experience.In addition, present invention also offers a kind of live system of dynamic website
System, equally with above-mentioned beneficial effect.
Brief description of the drawings
In order to illustrate more clearly about the embodiment of the present invention or technical scheme of the prior art, below will be to embodiment or existing
There is the accompanying drawing used required in technology description to be briefly described, it should be apparent that, drawings in the following description are only this
The embodiment of invention, for those of ordinary skill in the art, on the premise of not paying creative work, can also basis
The accompanying drawing of offer obtains other accompanying drawings.
A kind of flow chart of the live broadcasting method for dynamic website that Fig. 1 is provided by the embodiment of the present invention；
A kind of system structure diagram of the live broadcasting method for dynamic website that Fig. 2 is provided by the embodiment of the present invention.
The flow chart of the live broadcasting method for another dynamic website that Fig. 3 is provided by the embodiment of the present invention；
The token of the live broadcasting method for another dynamic website that Fig. 4 is provided by the embodiment of the present invention obtains schematic diagram；
The system flow schematic diagram of the live broadcasting method for another dynamic website that Fig. 5 is provided by the embodiment of the present invention；
A kind of structure chart of the live broadcast system for dynamic website that Fig. 6 is provided by the embodiment of the present invention.
Embodiment
To make the purpose, technical scheme and advantage of the embodiment of the present invention clearer, below in conjunction with the embodiment of the present invention
In accompanying drawing, the technical scheme in the embodiment of the present invention is clearly and completely described, it is clear that described embodiment is
A part of embodiment of the present invention, rather than whole embodiments.Based on the embodiment in the present invention, those of ordinary skill in the art
The every other embodiment obtained under the premise of creative work is not made, belongs to the scope of protection of the invention.
It refer to Fig. 1 and Fig. 2, a kind of flow of the live broadcasting method for dynamic website that Fig. 1 is provided by the embodiment of the present invention
Figure；A kind of system structure diagram of the live broadcasting method for dynamic website that Fig. 2 is provided by the embodiment of the present invention.This method can be with
Including：
Step 101：According to the authorization message that user is selected in dynamic website, the acquisition authorization message is corresponding forever to step on
The token of Lu Suoshu dynamic websites；Wherein, the authorization message includes the login letter that corresponding selected user logs in dynamic website
Breath.
It is understood that selected user can be the existing subscriber in dynamic website, that is, the note in dynamic website
Volume user.Can be to send to obtain authorization message to selected user for the specific acquisition modes for the authorization message for selecting user
Authorization requests, and receive the authorization message that selected user returns, that is, any registered user into dynamic website sends
The authorization requests of authorization message are obtained, the authorization message that registered user returns is received, wherein, by the registration of the authorization message of return
User is used as selected user；The authorization message of user can also be selected for directly acquisition, that is, can be specially in dynamic website
Registered user, directly obtains the authorization message of the registered user, wherein, the registered user can be selected user.As long as can be with
Corresponding token is obtained according to the authorization message of selected user, specific acquisition modes and choosing for the authorization message of selected user
Determine the specific personnel selection of user, can voluntarily be set according to practical scene and user's request by designer, the present embodiment pair
This does not do any limitation.
It should be noted that for being obtained in this step by authorization message, authorization message is corresponding forever to log in Dynamic Networks
The concrete mode for the token stood, can use Oauth2.0 technologies, that is, according to authorization message, pass through Oauth2.0 agreement moulds
Intend user and obtain the corresponding token of authorization message；Other technologies can also be used.As long as can obtain permanent by authorization message
The token of dynamic website is logged in, the concrete mode for obtaining token, the present embodiment does not do any limitation.
Specifically, this step can be with as shown in Fig. 2 the access mandate module of dynamic website live broadcast system be sent out to dynamic website
Authorization message is sent, the corresponding token of the authorization message (AccessRobot) that dynamic website is returned is received.The present embodiment to this not
It is restricted.
Step 102：A reptile robot is distributed for each token, obtains described dynamic using the reptile robot
The current latest data of the pre-set user of state website, and the current latest data is stored to database.
Wherein, reptile robot is utilized to obtain the specific of the current latest data of the pre-set user of dynamic website in this step
Mode, can be that each reptile robot logs in dynamic website by token and from IP proxy servers acquisition IP address, then
The current latest data that pre-set user is found in the time and the pre-set user is obtained is crawled in default safety；Can also be other
Mode, as long as the current latest data of the pre-set user of dynamic website can be obtained using reptile robot, for specifically obtaining
Mode is taken, the present embodiment does not do any limitation.
It is understood that because the dynamic website of such as Sina weibo can be in certain time period from the requested of a certain IP
For a period of time, that is, to the frequency that crawls of each reptile robot limited in frequently denied access can be carried out to its IP
System, thus can use multiple reptile robots obtained jointly by this step dynamic website pre-set user current newest number
According to mode, the reptile robot of predetermined number, the corresponding reptile machine of each pre-set user are such as distributed for each pre-set user
People obtains current latest data in respective preset time successively.The present embodiment does not do any limitation to this.
It should be noted that in this step pre-set user can be dynamic website in the need for carry out live user, such as
Pre-set user can be Sina weibo in the need for carry out the big V in live stock market.The current latest data of pre-set user can be
The announced newest data of pre-set user, such as current latest data can be Sina weibo in the need for carry out live stock market
The announced newest microbloggings of big V.For pre-set user and current latest data particular content can by designer according to
Practical scene and user's request are voluntarily set, and the present embodiment does not do any limitation to this.
Specifically, this step can be with as shown in Fig. 2 the access mandate module of dynamic website live broadcast system be by token
(Accesstoken) send to data acquisition module, data acquisition module is that each token is distributed in a reptile robot group
Reptile robot (Crawl Robot), each reptile robot obtains each difference IP by IP proxy servers, is arranged on
Safety is crawled to be carried out crawling the current latest data of pre-set user in dynamic website in the time.
Step 103：The current latest data stored in the database is monitored, the pre-set user is obtained
The latest data of renewal, and the latest data is sent to default live platform.
Wherein, the latest data that pre-set user updates can be obtained and stored in database for first reptile robot
The current latest data that updates of pre-set user, such as the V issues big to a stock market in Sina weibo of multiple reptile robots
When newest microblogging is obtained, after the big V in the stock market updates a microblogging, that is, after one microblogging of issue, pre-set user is more
New latest data can obtain for first reptile robot and store the microblogging of database.
As long as it is understood that can ensure to obtain the latest data that pre-set user updates, for logarithm in this step
The concrete mode being monitored according to the current latest data stored in storehouse, can be mode as shown in Figure 2, and dynamic website is live
The data-interface (Restful) that the monitor module of system is provided by WebService, continuous polling data storehouse, until finding
The latest data that pre-set user updates, can also use other modes, the present embodiment is unrestricted to this.
It should be noted that default live platform can be QQ groups and/or wechat group and/or platform webcast website.Tool
Body, this step can be with as shown in Fig. 2 the data that the monitor module of dynamic website live broadcast system is provided by WebService
Interface (Restful), continuous polling data storehouse until finding the latest data that pre-set user updates, and latest data is pushed
Carried out to QQ groups, wechat group and platform webcast website live.
It is preferred that, the method that the present embodiment is provided can also include：By fixed platform website according to pre-set level pair
Each corresponding whole latest datas of pre-set user are counted and show statistical result.That is, for the present embodiment institute
The method of offer, the latest data that can be not only updated by QQ groups, wechat group and platform webcast website to pre-set user is carried out
It is live, by platform website in the past live data can also be carried out with related statistics, Consumer's Experience is improved.
In the present embodiment, the embodiment of the present invention is by according to the authorization message that user is selected in dynamic website, obtaining and authorizing
The corresponding permanent token for logging in dynamic website of information, can make reptile robot forever log in dynamic website by token, solve
Access of the dynamic website to reptile robot of having determined is limited, and by distributing a reptile robot for each token, utilizes reptile
Robot obtains the current latest data of the pre-set user of dynamic website, can use multiple reptile robots to dynamic website
The current latest data of pre-set user is captured, and effectively avoids the dynamic website such as Sina weibo and reptile robot is climbed
The limitation of frequency is taken, live real-time is improved, improves Consumer's Experience.
It refer to Fig. 3, Fig. 4 and Fig. 5, the live broadcasting method for another dynamic website that Fig. 3 is provided by the embodiment of the present invention
Flow chart；The token of the live broadcasting method for another dynamic website that Fig. 4 is provided by the embodiment of the present invention obtains schematic diagram；Figure
The system flow schematic diagram of the live broadcasting method of the 5 another dynamic websites provided by the embodiment of the present invention.This method can be wrapped
Include：
Step 201：The authorization requests for obtaining authorization message are sent to selected user.
Wherein, this step can be that all or part of registered user sends authorization requests into dynamic website, that is, such as
Shown in Fig. 4, the interface that dynamic website live broadcast system is provided as resources requesting party by dynamic website (is registered to resource owner
User) request mandate.
It is understood that the selection mode of the part registered user sent for authorization requests, can be by designer
Voluntarily set according to practical scene, the present embodiment does not do any limitation to this.
Step 202：Obtain the authorization message that selected user returns.
Wherein, selected user can be the registered user for returning to authorization message.This step can be with as shown in figure 4, resource be gathered around
The person of having gives to resources requesting party and authorized.
As long as it should be noted that the permanent token for logging in dynamic website can be obtained by authorization message, for obtaining
The particular content for the authorization message that selected user returns, that is, the particular content included in authorization message, the present embodiment are not done
Any limitation.
Step 203：According in dynamic website select user authorization message, obtain authorization message it is corresponding forever log in it is dynamic
The token of state website.
Wherein, this step can be using as shown in figure 4, dynamic website live broadcast system be as resources requesting party first to dynamic website
Authorization server sends authorization message, with acquisition request access token, waits after authorization server checking authorization message, then receive
The token that authorization server is provided.
Step 204：A reptile robot is distributed for each token.
It is understood that each reptile machine possesses a different token per capita.
Step 205：The reptile robot of predetermined number is distributed for each pre-set user.
Wherein, predetermined number can voluntarily be set by designer according to practical scene and user's request in this step, such as
Need reptile robot to crawl microblog data in Sina weibo, then can be each to need to crawl the big V accounts distribution of microblogging
20 reptile robots, on the basis of ensuring to avoid Sina weibo to crawl the limitation of frequency to single reptile robot,
The newest microblogging of the big V accounts is crawled in real time, the delay live to the big V accounts is reduced.The present embodiment to this not by
Any limitation.
Step 206：Each corresponding reptile robot of pre-set user is obtained currently most in respective preset time successively
New data, and current latest data is stored to database.
It is understood that for the corresponding predetermined number of each pre-set user reptile robot to the pre-set user
The acquisition modes of current latest data, can as shown in this step predetermined number reptile robot by the order pre-set according to
It is secondary that obtaining for the current latest data in the time to the pre-set user is crawled in respective safety；Other can also be used to obtain
Mode is taken, such as the reptile robot of the corresponding predetermined number of each pre-set user is successively by default time interval to the default use
The progress of the current latest data at family is obtained.As long as the reptile robot of predetermined number is not being limited the base of access by dynamic website
On plinth, the current latest data of the pre-set user is obtained, the reptile of predetermined number corresponding for each pre-set user
Robot does not do any limitation to the acquisition modes of the current latest data of the pre-set user, the present embodiment.
It should be noted that this step can as indicated at 4, dynamic website live broadcast system passes through tool as resources requesting party
There is the reptile robot of access token to obtain target resource from the Resource Server of dynamic website, that is, pre-set user is current
Latest data.
Specifically, the corresponding data of pre-set user that reptile robot is obtained can be current latest data, such as Sina is micro-
Need to crawl the current newest microblogging of the big V accounts of microblogging in rich；The latest data that can also be updated for pre-set user, it is such as pre-
If the reptile robot of quantity only obtains the newest microblogging for the renewal that the big V accounts for crawling microblogging are needed in a Sina weibo,
As long as namely there is a reptile robot to obtain and store after newest microblogging, other reptile robots no longer obtain and store this
Newest microblogging.The present embodiment does not do any limitation to this.
Step 207：The current latest data stored in database is monitored, the newest number that pre-set user updates is obtained
According to, and latest data is sent to default live platform.
Wherein, the current latest data stored in database is monitored in this step, obtains what pre-set user updated
Latest data, can be with as shown in figure 5, the monitor module of dynamic website live broadcast system be connect by the WebService data provided
Mouthful, continuous polling data storehouse inquires about the current latest data stored in database and whether there is the newest number that pre-set user updates
According to if in the presence of returning to latest data.Design for the data-interface can use Restful technologies, due to application
The operation each time to database can regard the once request to data, services as, therefore, can be by each species
The data manipulation of type is mapped as the corresponding data request patterns in http agreements, and logarithm is realized using simple and clear url
According to operation.The present embodiment does not do any limitation to this.
It is understood that for latest data is sent to the concrete mode of default live platform, can be by designing
Personnel are correspondingly arranged according to the type of default live platform, and such as default live platform is the QQ group and wechat pre-established
Latest data, then can be published in group, realized live using the operation of shell script simulating keyboard by group automatically.The present embodiment
Any limitation is not done to this.
Step 208：By fixed platform website according to pre-set level whole latest datas corresponding to each pre-set user
Counted and show statistical result.
Wherein, this step can be that the latest data of whole pre-set users is counted and statistical result is shown, also may be used
Counted with the latest data to one or several pre-set users in whole pre-set users and show statistical result, this implementation
Example is unrestricted to this.
It is understood that for being counted according to pre-set level whole latest datas corresponding to each pre-set user
Concrete mode, can voluntarily be set according to practical scene and user's request by actual person, such as to the big V in stock market in smooth sea microblogging
Live whole microbloggings in position in storehouse information counted, the curve map of position in storehouse change is obtained, and by the curve map at one
Fixed platform website is shown.
In the present embodiment, the embodiment of the present invention by for each pre-set user distribute predetermined number reptile robot, often
The corresponding reptile robot of individual pre-set user obtains current latest data in respective preset time successively, and will be current newest
Data storage can be effectively prevented from the dynamic website of Sina weibo such as and the limit of frequency is crawled to reptile robot to database
System, improves live real-time；By fixed platform website according to pre-set level to the corresponding whole of each pre-set user most
New data is counted and shows statistical result, it is possible to use platform website carries out related statistics in the past live data,
Improve Consumer's Experience.
It refer to Fig. 6, a kind of structure chart of the live broadcast system for dynamic website that Fig. 6 is provided by the embodiment of the present invention.Should
System can include:
Access mandate module 100, for according to the authorization message that user is selected in dynamic website, obtaining the authorization message
The corresponding token for forever logging in the dynamic website；Wherein, the authorization message logs in dynamic including corresponding selected user
The log-on message of website；
Data acquisition module 200, for distributing a reptile robot for each token, utilizes the reptile machine
People obtains the current latest data of the pre-set user of the dynamic website, and the current latest data is stored to database
300；
The database 300, for storing the corresponding current latest data of each pre-set user；
Monitor module 400, for being monitored to the current latest data stored in the database 300, is obtained
The latest data for taking the pre-set user to update, and the latest data is sent to default live platform.
Optionally, the access mandate module 100, can include：
Token acquiring unit, for according to the authorization message, by being awarded described in Oauth2.0 protocol emulations user acquisition
Weigh the corresponding token for forever logging in the dynamic website of information.
Optionally, the access mandate module 100, can include：
Transmitting element, the authorization requests of the authorization message are obtained for being sent to the selected user；
Receiving unit, for obtaining the authorization message that the selected user returns.
Optionally, the data acquisition module 200, can include：
Allocation unit, the reptile robot for distributing predetermined number for each pre-set user；
Acquiring unit, is obtained in respective preset time successively for the corresponding reptile robot of each pre-set user
Take the current latest data.
Optionally, the system can also include：
Statistical module, for by fixed platform website according to pre-set level to the corresponding whole of each pre-set user
The latest data is counted and shows statistical result.
In the present embodiment, the embodiment of the present invention is awarded according to selecting user in dynamic website by access mandate module 100
Information is weighed, the corresponding permanent token for logging in dynamic website of authorization message is obtained, reptile robot can be made permanent by token
Dynamic website is logged in, access of the dynamic website to reptile robot is solved and limits, be each order by data acquisition module 200
Board distributes a reptile robot, and the current latest data of the pre-set user of dynamic website is obtained using reptile robot, can be with
The current latest data of the pre-set user of dynamic website is captured using multiple reptile robots, effectively avoided as new
The dynamic website of unrestrained microblogging crawls the limitation of frequency to reptile robot, improves live real-time, improves Consumer's Experience.
The embodiment of each in specification is described by the way of progressive, and what each embodiment was stressed is and other realities
Apply the difference of example, between each embodiment identical similar portion mutually referring to.For device disclosed in embodiment
Speech, because it is corresponded to the method disclosed in Example, so description is fairly simple, related part is referring to method part illustration
.
Professional further appreciates that, with reference to the unit of each example of the embodiments described herein description
And algorithm steps, can be realized with electronic hardware, computer software or the combination of the two, in order to clearly demonstrate hardware and
The interchangeability of software, generally describes the composition and step of each example according to function in the above description.These
Function is performed with hardware or software mode actually, depending on the application-specific and design constraint of technical scheme.Specialty
Technical staff can realize described function to each specific application using distinct methods, but this realization should not
Think beyond the scope of this invention.
Directly it can be held with reference to the step of the method or algorithm that the embodiments described herein is described with hardware, processor
Capable software module, or the two combination are implemented.Software module can be placed in random access memory (RAM), internal memory, read-only deposit
Reservoir (ROM), electrically programmable ROM, electrically erasable ROM, register, hard disk, moveable magnetic disc, CD-ROM or technology
In any other form of storage medium well known in field.
The live broadcasting method and system of dynamic website provided by the present invention are described in detail above.It is used herein
Specific case is set forth to the principle and embodiment of the present invention, and the explanation of above example is only intended to help and understands
The method and its core concept of the present invention.It should be pointed out that for those skilled in the art, not departing from this
On the premise of inventive principle, some improvement and modification can also be carried out to the present invention, these are improved and modification also falls into the present invention
In scope of the claims.
Claims (10)
1. a kind of live broadcasting method of dynamic website, it is characterised in that including：
According to the authorization message that user is selected in dynamic website, the acquisition authorization message is corresponding forever to log in the Dynamic Networks
The token stood；Wherein, the authorization message includes the log-on message that corresponding selected user logs in dynamic website；
A reptile robot is distributed for each token, the default of the dynamic website is obtained using the reptile robot
The current latest data of user, and the current latest data is stored to database；
The current latest data stored in the database is monitored, the newest number that the pre-set user updates is obtained
According to, and the latest data is sent to default live platform.
2. the live broadcasting method of dynamic website according to claim 1, it is characterised in that described to be selected according in dynamic website
The authorization message of user, obtains the corresponding token for forever logging in the dynamic website of the authorization message, including：
According to the authorization message, forever institute is logged in by the way that the Oauth2.0 protocol emulations user acquisition authorization message is corresponding
State the token of dynamic website.
3. the live broadcasting method of dynamic website according to claim 2, it is characterised in that described according to the authorization message,
Before the corresponding token for forever logging in the dynamic website of the Oauth2.0 protocol emulations user acquisition authorization message,
Also include：
The authorization requests for obtaining the authorization message are sent to the selected user；
Obtain the authorization message that the selected user returns.
4. the live broadcasting method of dynamic website according to claim 3, it is characterised in that described to utilize the reptile robot
The current latest data of the pre-set user of the dynamic website is obtained, including：
The reptile robot of predetermined number is distributed for each pre-set user；
Each the corresponding reptile robot of the pre-set user obtains the current newest number in respective preset time successively
According to.
5. the live broadcasting method of the dynamic website according to any one of Claims 1-4, it is characterised in that also include：
Each pre-set user corresponding whole latest data is carried out according to pre-set level by fixed platform website
Count and show statistical result.
6. a kind of live broadcast system of dynamic website, it is characterised in that including：
Access mandate module, for according to the authorization message that user is selected in dynamic website, obtaining the authorization message corresponding
Forever log in the token of the dynamic website；Wherein, the authorization message logs in dynamic website including corresponding selected user
Log-on message；
Data acquisition module, for distributing a reptile robot for each token, is obtained using the reptile robot
The current latest data of the pre-set user of the dynamic website, and the current latest data is stored to database；
The database, for storing the corresponding current latest data of each pre-set user；
Monitor module, for being monitored to the current latest data stored in the database, obtains described default
The latest data that user updates, and the latest data is sent to default live platform.
7. the live broadcast system of dynamic website according to claim 6, it is characterised in that the access mandate module, including：
Token acquiring unit, for according to the authorization message, obtaining described authorize by Oauth2.0 protocol emulations user and believing
The corresponding token for forever logging in the dynamic website of breath.
8. the live broadcast system of dynamic website according to claim 7, it is characterised in that the access mandate module, including：
Transmitting element, the authorization requests of the authorization message are obtained for being sent to the selected user；
Receiving unit, for obtaining the authorization message that the selected user returns.
9. the live broadcast system of dynamic website according to claim 8, it is characterised in that the data acquisition module, including：
Allocation unit, the reptile robot for distributing predetermined number for each pre-set user；
Acquiring unit, institute is obtained for the corresponding reptile robot of each pre-set user in respective preset time successively
State current latest data.
10. the live broadcast system of the dynamic website according to any one of claim 6 to 9, it is characterised in that also include：
Statistical module, for by fixed platform website according to pre-set level to described in the corresponding whole of each pre-set user
Latest data is counted and shows statistical result.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710278347.6A CN107103079B (en) | 2017-04-25 | 2017-04-25 | Live broadcast method and system for dynamic website |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710278347.6A CN107103079B (en) | 2017-04-25 | 2017-04-25 | Live broadcast method and system for dynamic website |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107103079A true CN107103079A (en) | 2017-08-29 |
| CN107103079B CN107103079B (en) | 2021-05-25 |
Family
ID=59657310
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710278347.6A Active CN107103079B (en) | 2017-04-25 | 2017-04-25 | Live broadcast method and system for dynamic website |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107103079B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110620670A (en) * | 2019-10-15 | 2019-12-27 | 深圳市小赢信息技术有限责任公司 | Token acquisition method, data acquisition system, proxy server, and storage medium |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101551813A (en) * | 2009-05-13 | 2009-10-07 | 腾讯科技（深圳）有限公司 | Network connection apparatus, search equipment and method for collecting search engine data source |
| US20110302147A1 (en) * | 2007-12-05 | 2011-12-08 | Yahoo! Inc. | Methods and apparatus for computing graph similarity via sequence similarity |
| CN103037010A (en) * | 2012-12-26 | 2013-04-10 | 人民搜索网络股份公司 | Distributed network crawler system and catching method thereof |
| CN103440139A (en) * | 2013-09-11 | 2013-12-11 | 北京邮电大学 | Acquisition method and tool facing microblog IDs (identitiesy) of mainstream microblog websites |
-
2017
- 2017-04-25 CN CN201710278347.6A patent/CN107103079B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20110302147A1 (en) * | 2007-12-05 | 2011-12-08 | Yahoo! Inc. | Methods and apparatus for computing graph similarity via sequence similarity |
| CN101551813A (en) * | 2009-05-13 | 2009-10-07 | 腾讯科技（深圳）有限公司 | Network connection apparatus, search equipment and method for collecting search engine data source |
| CN103037010A (en) * | 2012-12-26 | 2013-04-10 | 人民搜索网络股份公司 | Distributed network crawler system and catching method thereof |
| CN103440139A (en) * | 2013-09-11 | 2013-12-11 | 北京邮电大学 | Acquisition method and tool facing microblog IDs (identitiesy) of mainstream microblog websites |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110620670A (en) * | 2019-10-15 | 2019-12-27 | 深圳市小赢信息技术有限责任公司 | Token acquisition method, data acquisition system, proxy server, and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN107103079B (en) | 2021-05-25 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109246211B (en) | Resource uploading and resource requesting method in block chain | |
| CN103329113B (en) | Configuration is accelerated and custom object and relevant method for proxy server and the Dynamic Website of hierarchical cache | |
| Bent et al. | Characterization of a large web site population with implications for content delivery | |
| TWI557571B (en) | Method, computer-storage media, and server for optimizing web crawling with user history | |
| US10542124B2 (en) | Systems and methods of rate limiting for a representational state transfer (REST) application programming interface (API) | |
| CN105939361B (en) | Defend the method and device of CC attack | |
| CN109525602B (en) | Method, mobile terminal and system for accessing network based on flow-free platform | |
| CN103179148B (en) | A kind of processing method sharing adnexa in the Internet and system | |
| CN103716391A (en) | Implementation method for content caching and router | |
| CN106656959A (en) | Access request regulation and control method and device | |
| TW201824047A (en) | Attack request determination method, apparatus and server | |
| Datta et al. | World wide wait: A study of Internet scalability and cache-based approaches to alleviate it | |
| CN105450780A (en) | CDN system and source tracing method thereof | |
| CN104219069B (en) | access frequency control method, device and control system | |
| CN107196968B (en) | Crawler identification method | |
| CN107453900B (en) | Cloud analysis parameter setting management system and method for realizing parameter setting | |
| CN109672757A (en) | File access method and file access processing unit | |
| CN105959358A (en) | CDN server and method of CDN server of caching data | |
| CN109756584B (en) | Domain name resolution method, domain name resolution device and computer readable storage medium | |
| CN109787915A (en) | Flow control methods, device, electronic equipment and the storage medium of network access | |
| CN105871919A (en) | Network application firewall system and realization method thereof | |
| CN106789868A (en) | A kind of website user's Activity recognition and managing and control system | |
| CN105847277A (en) | Service account share management method and system used for third party application | |
| CN106844111A (en) | The access method of cloud storage NFS | |
| CN110166486A (en) | Website protection method, device and computer readable storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information |
Address after: 215347 7th floor, IIR complex, 1699 Weicheng South Road, Kunshan City, Suzhou City, Jiangsu Province Applicant after: Kunshan Microelectronics Technology Research Institute Address before: 215347 7th floor, complex building, No. 1699, Zuchongzhi South Road, Kunshan City, Suzhou City, Jiangsu Province Applicant before: KUNSHAN BRANCH, INSTITUTE OF MICROELECTRONICS OF CHINESE ACADEMY OF SCIENCES |
|
| CB02 | Change of applicant information | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |