CN110598135A - Network request processing method and device, computer readable medium and electronic equipment - Google Patents
Network request processing method and device, computer readable medium and electronic equipment Download PDFInfo
- Publication number
- CN110598135A CN110598135A CN201810505497.0A CN201810505497A CN110598135A CN 110598135 A CN110598135 A CN 110598135A CN 201810505497 A CN201810505497 A CN 201810505497A CN 110598135 A CN110598135 A CN 110598135A
- Authority
- CN
- China
- Prior art keywords
- page application
- network
- network access
- access request
- rendering
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Landscapes
- Information Transfer Between Computers (AREA)
Abstract
The embodiment of the invention provides a network request processing method and device for single-page application, a computer readable medium and electronic equipment. The method comprises the following steps: receiving a network access request for a single-page application; when the network access request comes from a search engine crawler, performing server-side rendering on the single-page application to obtain rendered webpage content; and returning the rendered webpage content. Performing server-side rendering on the single-page application comprises: calling an engine of a non-interface browser to open a website corresponding to the single-page application; monitoring a rendering completion event returned by the non-interface browser engine; and acquiring the rendered webpage content. The technical scheme of the embodiment of the invention can ensure that the content of the single-page application can be effectively captured by the search engine crawler.
Description
Technical Field
The present application relates to the field of computer technologies, and in particular, to a method and an apparatus for processing a network request of a single-page application, a computer-readable medium, and an electronic device.
Background
A Single Page Application (Single Page Application) is a web Application that uses asynchronous browser rendering on the browser side, such as a web Page developed using modern front-end frameworks (realjs, vuejs, etc.). The browser loads a single HTML page and then all interactions and effects are implemented within the single HTML page without the need to reload the HTML page.
The user does not need to reload the whole page to acquire the content on the single-page application, so the single-page application has good user experience and generates less stress on the server.
However, single-page applications do not support searching well. In other words, the search engine crawler cannot effectively capture the content in the single-page application, so that the web page content cannot be recorded by the search engine, and the content popularization and diffusion are affected.
Therefore, there is a need for a network request processing method for single page applications to facilitate search engine optimization.
It is to be noted that the information disclosed in the above background section is only for enhancement of understanding of the background of the present invention and therefore may include information that does not constitute prior art known to a person of ordinary skill in the art.
Disclosure of Invention
The embodiment of the invention relates to a network request processing method and device for single-page application, a computer readable medium and electronic equipment, which can overcome the problem that the content in the single-page application cannot be effectively captured by a search engine crawler to a certain extent.
Additional features and advantages of the invention will be set forth in the detailed description which follows, or may be learned by practice of the invention.
According to an aspect of the embodiments of the present invention, a method for processing a network request of a single-page application is provided, including: receiving a network access request for a single-page application; when the network access request comes from a search engine crawler, performing server-side rendering on the single-page application to obtain rendered webpage content; and returning the rendered webpage content, wherein the performing of the server-side rendering on the single-page application comprises: calling an engine of a non-interface browser to open a website corresponding to the single-page application; monitoring a rendering completion event returned by the non-interface browser engine; and acquiring the rendered webpage content.
According to another aspect of the embodiments of the present invention, there is provided a network request processing apparatus for a single-page application, including: a receiving unit, configured to receive a network access request for a single-page application; the rendering unit is used for rendering the single-page application at a server side when the network access request comes from a crawler of a search engine to obtain rendered webpage content; and a returning unit for returning the rendered web page content, wherein the rendering unit includes: the calling subunit is used for calling an engine of the non-interface browser to open the website corresponding to the single-page application; the monitoring subunit is used for monitoring a rendering completion event returned by the non-interface browser engine; and the acquisition subunit is used for acquiring the rendered webpage content.
According to still another aspect of embodiments of the present invention, there is provided a computer-readable medium on which a computer program is stored, wherein the program, when executed by a processor, implements the network access request processing method of the above-described single-page application.
According to still another aspect of embodiments of the present invention, there is provided an electronic apparatus including: one or more processors; a storage device for storing one or more programs, which when executed by the one or more processors, cause the one or more processors to implement the network access request processing method of the above-mentioned single-page application.
In the technical scheme provided by the embodiment of the invention, the non-interface browser engine is called at the server side to render the single-page application and the rendered web page content is returned to the search engine crawler, so that the content of the single-page application can be grabbed by the search engine crawler in a simple and effective manner.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the invention, as claimed.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and together with the description, serve to explain the principles of the invention. It is obvious that the drawings in the following description are only some embodiments of the invention, and that for a person skilled in the art, other drawings can be derived from them without inventive effort. In the drawings:
FIG. 1 illustrates a schematic diagram of an exemplary system architecture 100 of a network request processing method or apparatus for a single-page application to which embodiments of the present invention may be applied;
FIG. 2 illustrates a schematic structural diagram of a computer system suitable for use with the electronic device to implement an embodiment of the invention;
FIG. 3 schematically illustrates a flow chart of a method of network request processing for a single-page application according to an embodiment of the invention;
FIG. 4 schematically illustrates a flow diagram for server-side rendering by a single-page application according to an embodiment of the invention;
FIG. 5 is a diagram illustrating a network request processing method for a single-page application, such as invoking Headless-Chrome;
FIG. 6 is a flowchart of a method for processing a network request of a single-page application, such as invoking a Headless-Chrome;
FIG. 7 schematically shows a block diagram of a network request processing device for a single-page application according to an embodiment of the present invention;
fig. 8 schematically shows a block diagram of a rendering unit according to an embodiment of the invention.
Detailed Description
Example embodiments will now be described more fully with reference to the accompanying drawings. Example embodiments may, however, be embodied in many different forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the concept of example embodiments to those skilled in the art.
Furthermore, the described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided to provide a thorough understanding of embodiments of the invention. One skilled in the relevant art will recognize, however, that the invention may be practiced without one or more of the specific details, or with other methods, components, devices, steps, and so forth. In other instances, well-known methods, devices, implementations or operations have not been shown or described in detail to avoid obscuring aspects of the invention.
The block diagrams shown in the figures are functional entities only and do not necessarily correspond to physically separate entities. I.e. these functional entities may be implemented in the form of software, or in one or more hardware modules or integrated circuits, or in different networks and/or processor means and/or microcontroller means.
The flow charts shown in the drawings are merely illustrative and do not necessarily include all of the contents and operations/steps, nor do they necessarily have to be performed in the order described. For example, some operations/steps may be decomposed, and some operations/steps may be combined or partially combined, so that the actual execution sequence may be changed according to the actual situation.
Fig. 1 illustrates a schematic diagram of an exemplary system architecture 100 of a network request processing method or apparatus for a single-page application to which embodiments of the present invention may be applied.
As shown in fig. 1, the system architecture 100 may include one or more of terminal devices 101, 102, 103, a network 104, and one or more of servers 105, 106, and 107. Network 104 is the medium used to provide communication links between terminal devices 101, 102, 103, servers 105, 106, and 107. Network 104 may include various connection types, such as wired, wireless communication links, or fiber optic cables, to name a few.
It should be understood that the number of terminal devices, networks, and servers in fig. 1 is merely illustrative. There may be any number of terminal devices, networks, and servers, as desired for implementation.
The user may use the terminal devices 101, 102, 103 to interact with the servers 105, 106 and 107 via the network 104 to receive or send messages or the like. The terminal devices 101, 102, 103 may be various electronic devices having a display screen, including but not limited to smart phones, tablet computers, portable computers, desktop computers, and the like.
The servers 105, 106, and 107 may be servers that provide various services, and may interact with each other through the network 104.
For example, the server 105 may be a web server. The user can access the web server 105 through the terminal device 101 (or the terminal device 102 or 103) and acquire the web page code for the single-page application from the web server 105. For another example, the server 106 may be a search engine server that can automatically crawl web page content from the web site server 105 through a search engine crawler (also referred to as a web spider, web bot, a program or script that automatically crawls web information according to certain rules). As another example, as described below, web server 105 may call an interface-less browser, such as a thread-chrome, from server 107.
It should be noted that the network request processing method for the single-page application provided by the embodiment of the present invention is generally executed by the server 105, and accordingly, the network request processing apparatus for the single-page application is generally disposed in the server 105.
FIG. 2 illustrates a schematic structural diagram of a computer system suitable for use with the electronic device to implement an embodiment of the invention.
It should be noted that the computer system 200 of the electronic device shown in fig. 2 is only an example, and should not bring any limitation to the functions and the scope of the application of the embodiment of the present invention.
As shown in fig. 2, the computer system 200 includes a Central Processing Unit (CPU)201 that can perform various appropriate actions and processes in accordance with a program stored in a Read Only Memory (ROM)202 or a program loaded from a storage section 208 into a Random Access Memory (RAM) 203. In the RAM 203, various programs and data necessary for system operation are also stored. The CPU201, ROM 202, and RAM 203 are connected to each other via a bus 204. An input/output (I/O) interface 205 is also connected to bus 204.
The following components are connected to the I/O interface 205: an input portion 206 including a keyboard, a mouse, and the like; an output section 207 including a display such as a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and the like, and a speaker; a storage section 208 including a hard disk and the like; and a communication section 209 including a network interface card such as a LAN card, a modem, or the like. The communication section 209 performs communication processing via a network such as the internet. A drive 210 is also connected to the I/O interface 205 as needed. A removable medium 211 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 210 as necessary, so that a computer program read out therefrom is mounted into the storage section 208 as necessary.
In particular, according to an embodiment of the present invention, the processes described below with reference to the flowcharts may be implemented as computer software programs. For example, embodiments of the invention include a computer program product comprising a computer program embodied on a computer-readable medium, the computer program comprising program code for performing the method illustrated in the flow chart. In such an embodiment, the computer program may be downloaded and installed from a network through the communication section 209 and/or installed from the removable medium 211. The computer program executes various functions defined in the system of the present application when executed by a Central Processing Unit (CPU) 201.
It should be noted that the computer readable medium shown in the present invention can be a computer readable signal medium or a computer readable storage medium or any combination of the two. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present invention, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In the present invention, however, a computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, fiber optic cable, RF, etc., or any suitable combination of the foregoing.
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams or flowchart illustration, and combinations of blocks in the block diagrams or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The units described in the embodiments of the present invention may be implemented by software, or may be implemented by hardware, and the described units may also be disposed in a processor. Wherein the names of the elements do not in some way constitute a limitation on the elements themselves.
As another aspect, the present application also provides a computer-readable medium, which may be contained in the electronic device described in the above embodiments; or may exist separately without being assembled into the electronic device. The computer readable medium carries one or more programs which, when executed by an electronic device, cause the electronic device to implement the method as described in the embodiments below. For example, the electronic device may implement the steps shown in fig. 3 and 4.
As previously mentioned, web pages crawled by search engines require complete HTML and content, and therefore single page applications do not support searches well. That is, the search engine crawler cannot efficiently crawl dynamic content in the page that is updated with Ajax.
In the prior art, some single-page applications do not perform search engine optimization, and a request for a crawler is directly returned to a no-content template for browser rendering (for example, a browser obtains data asynchronously through AJAX and then renders HTML on a browser side). This approach results in the content of the single-page application not being available to the crawler, so that the crawler cannot search for web page content.
Another existing method is to multiplex the JS codes at the front and rear ends, so that the JS code running in the browser environment runs in the nodjs server environment, and render an HTML webpage at the server and return the HTML webpage to the crawler. This approach can result in complex and costly code writing. In addition, this introduces very intrusive changes to the existing code due to the need to be compatible with both execution environments.
The embodiment of the invention provides a search engine optimization problem which can be quickly and conveniently solved for single-page application, for example, whether a request comes from a crawler is judged through a User-Agent field of an HTTP protocol header, if so, a non-interface browser such as a thread-chrome is called to render webpage content and return the webpage content to the crawler, and therefore the crawler captures the content of the single-page application. For example, the network request processing method of the single page application provided according to the embodiment of the present invention may be performed in a server (e.g., the server 105).
Fig. 3 schematically shows a flowchart of a network request processing method of a single-page application according to an embodiment of the present invention.
As shown in fig. 3, the network request processing method of a single-page application according to an embodiment of the present invention includes the following steps.
In step S310, a network access request for a single-page application is received.
For example, server 105, which is a web server, may receive a network access request for a single-page application.
The network access request to access the website content may come from a search engine or user terminal, for example, from a search crawler of server 106, which is a search engine server, or may come from terminal device 101 (which may also be terminal devices 102 or 103).
A network access request, such as a request message of the HTTP protocol, includes an HTTP header and a message body. The request header field is a field used in a request message sent from the client to the server, and is used for supplementing the content such as the additional information of the request, the client information, the priority related to the response content, and the like, for example, Accept-charset, Host, Referer, User-Agent, Cookie, and the like, and may also include a custom field.
In step S320, when the web access request is from a search engine crawler, performing server-side rendering on the single-page application to obtain rendered web page content.
According to the embodiment of the invention, whether the network access request comes from the search engine crawler can be judged according to the network request field. For example, when a search engine crawler crawls web page content, a User Agent field (User-Agent) may be carried in the HTTP request protocol header to identify the crawler identity. For example, Google web crawlers are identified as GoogleBot, Baidu web crawlers are identified as BaiDuSpider, and Yahoo web crawlers are identified as Inktomi Slurp. The user agent field may also identify other information of the user making the network access, which may include, for example, a browser identification of the user, an identification of the operating system, a reference identification of the search engine, and the like.
According to the embodiment of the invention, the reference identifier of the search engine can be collected in advance and stored. In this case, when the stored reference identifier of the search engine contains the user agent field of the network access request, then it may be determined that the network access request is from a search engine crawler.
For example, reference identifications (e.g., keywords) of existing search engines may be stored in an array. When a network access request is received, the array is traversed to determine whether any reference identifier (e.g., a keyword) in the array is contained in the user agent field. If so, it may be determined that the network access request is from a search engine crawler; if not, it may be determined that the network access request is not from a search engine crawler.
In step S330, the rendered web page content is returned. The webpage content rendered through the process can be captured by a crawler of a search engine, and therefore search engine optimization of the single-page application website is completed.
FIG. 4 schematically shows a flowchart of server-side rendering by a single-page application according to an embodiment of the present invention.
As shown in fig. 4, performing server-side rendering on the single-page application may include steps S410 to S430.
In step S410, an interface-less browser engine is invoked to open a website corresponding to the single-page application.
Any existing non-interface browser engine may be used in accordance with embodiments of the present invention, such as Headless-Chrome, PhantomJS, etc. The invoked non-interface browser renders content of the single-page application, such as an html string, recognizable by a search engine crawler, like a normal browser, by running web page code for browser rendering.
According to an example embodiment of the present invention, an un-interfaced browser engine may be invoked from another server (e.g., server 107) to open a web address corresponding to a single-page application.
According to embodiments of the present invention, the automatic interaction with the non-interface browser engine may be through a remote control protocol.
For example, Chrome provides a websocket debugging interface for debugging the DOM, network, performance, storage, etc. of pages within the current Tab. A common developer tool is based on this interface, which also supports remote invocation, and adds-remote-debug-port 9222 to the startup parameter. The websocket is a new protocol of HTML5, full-duplex communication between the browser and the server is realized, only one handshake action is needed, a fast channel is formed between the browser and the server, and then data are transmitted to each other.
In addition, the chrome provides a remote control interface, and currently, a js code can be used for sending commands to the chrome for interaction through the chrome-remote-interface. The chrome-remote-interface provides a convenient way for compiling a debugging tool based on the chrome debugging protocol, and by utilizing the chrome-remote-interface, the debugging tool can be developed through the websocket without being based on the original protocol. In fact, the Chrome-remote-interface is a remote encapsulation for Chrome DevTools Protocol.
In step S420, a rendering completion event returned by the non-interface browser engine is monitored.
And monitoring that the rendering completion event represents that the content of the webpage is rendered, namely the called non-interface browser engine completes the rendering of the webpage.
For example, for a Headless-Chrome, after sending a Page.
According to an embodiment of the present invention, the rendering completion event may include a DOM content triggering event. For example, for Headless-Chrome, the render complete event may be a DomConentEventFired event. Ispageready can also be utilized for complex scenes.
In step S430, that is, after the rendering completion event returned by the non-interface browser engine is monitored, the rendered web page content may be acquired.
The above describes the case when the network access request is from a search engine crawler. According to an embodiment of the invention, when the network access request is not from the search engine crawler, returning webpage code for rendering by the requesting browser. For example, when the network access request comes from a browser of the user terminal, a web page code for a single-page application may be returned.
FIG. 5 is a diagram illustrating a network request processing method for a single-page application, for example, invoking Headless-Chrome. FIG. 6 is a flowchart of a method for processing a network request of a single-page application, for example, invoking Headless-Chrome.
As shown in fig. 5 and fig. 6, according to the network request processing method of the single-page application of the embodiment of the present invention, when it is determined that the network access request is from the search engine crawler, only the header-Chrome needs to be called to open the address to be accessed by the network access request, and the content rendered by the header-Chrome is returned to the search engine crawler.
Specifically, first, at step 610, a client request service is received, i.e., a network access request for a single-page application is received, such network access request may be from, for example, a search engine or a user terminal, as described above.
Then, in step 620, a User Agent field (UA) is read from the HTTP protocol header of the network access request; at step 630, it is determined by the UA whether the request is from a search engine crawler. For example, whether the request is from a search engine crawler may be determined by the methods described above.
When the network access request is not from a crawler, for example, when the network access request is from a browser of the user terminal, the web page code for browser rendering is returned, i.e., a web page purchase for single page use is returned, in step 640, thereby ending the request.
When the network access request comes from the crawler, the following steps are executed:
1) at step 650, start head-chrome;
2) in step 660, the website corresponding to the single-page application is opened by the help-chrome through page.
3) In step 670, waiting for the trigger of the domcondenteventfired event of the single-page application, at which time the content of the web page is rendered;
4) in step 680, the rendered content of the web page is obtained through dom.
At this point, the rendered web page content is returned, at step 690, thereby ending the request.
As can be seen from the above description with reference to the figures and the exemplary embodiments, the method may not make any changes to the original code of the web server, and thus non-intrusive access may be achieved.
In addition, the amount of computation required for detection of the user agent field is very small and therefore does not affect the performance of the web server.
Therefore, the method for processing the network request of the single-page application can enable the content of the single-page application to be grabbed by the search engine crawler in a simple and effective mode.
The following describes an embodiment of the apparatus of the present invention, which can be used to execute the network request processing method of the above-mentioned single-page application of the present invention. For details that are not disclosed in the embodiments of the apparatus of the present invention, refer to the embodiments of the network request processing method for single-page applications described above.
Fig. 7 schematically shows a block diagram of a network request processing device of a single-page application according to an embodiment of the present invention.
Referring to fig. 7, a network request processing apparatus 700 of a single page application according to an embodiment of the present invention may include: a receiving unit 710, configured to receive a network access request for a single-page application; a rendering unit 720, configured to perform server-side rendering on the single-page application when the network access request is from a search engine crawler, to obtain rendered web page content; and a returning unit 730, configured to return the rendered web page content.
Fig. 8 schematically shows a block diagram of a rendering unit according to an embodiment of the invention.
Referring to fig. 8, the rendering unit 720 may include: a calling subunit 721, configured to call an interface-less browser engine to open a website corresponding to the single-page application; a monitoring subunit 722, configured to monitor a rendering completion event returned by the non-interface browser engine; an obtaining subunit 723, configured to obtain the rendered web page content.
According to an embodiment of the present invention, the returning unit 730 may further be configured to: and when the network access request is not from the search engine crawler, returning webpage code for rendering by the request-side browser.
According to an embodiment of the present invention, in addition to the receiving unit 710, the rendering unit 720, and the returning unit 730 described above, the network request processing apparatus 700 of the single-page application may further include: and the determining unit is used for determining whether the network access request comes from a search engine crawler through a user agent field.
According to an embodiment of the present invention, in addition to the receiving unit 710, the rendering unit 720, and the returning unit 730 described above, the network request processing apparatus 700 of the single-page application may further include: a collecting unit for collecting reference identifiers of the search engine; and a storage unit for storing the reference identifier.
According to an embodiment of the present invention, the determining unit may determine that the network access request is from a search engine crawler when the stored reference identifier includes a user agent field of the network access request.
According to an embodiment of the present invention, the rendering unit 720 may further include: and the interaction subunit is used for automatically interacting with the non-interface browser engine through a remote control protocol.
It should be noted that although in the above detailed description several modules or units of the device for action execution are mentioned, such a division is not mandatory. Indeed, the features and functionality of two or more modules or units described above may be embodied in one module or unit, according to embodiments of the invention. Conversely, the features and functions of one module or unit described above may be further divided into embodiments by a plurality of modules or units.
Through the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein may be implemented by software, or by software in combination with necessary hardware. Therefore, the technical solution according to the embodiment of the present invention can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a usb disk, a removable hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which can be a personal computer, a server, a touch terminal, or a network device, etc.) to execute the method according to the embodiment of the present invention.
Other embodiments of the invention will be apparent to those skilled in the art from consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the invention following, in general, the principles of the invention and including such departures from the present disclosure as come within known or customary practice within the art to which the invention pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the invention being indicated by the following claims.
It will be understood that the invention is not limited to the precise arrangements described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the invention is limited only by the appended claims.
Claims (15)
1. A network request processing method for a single-page application is characterized by comprising the following steps:
receiving a network access request for a single-page application;
when the network access request comes from a search engine crawler, performing server-side rendering on the single-page application to obtain rendered webpage content; and
returning the rendered content of the web page,
wherein performing server-side rendering on the single-page application comprises:
calling an engine of a non-interface browser to open a website corresponding to the single-page application;
monitoring a rendering completion event returned by the non-interface browser engine;
and acquiring the rendered webpage content.
2. The network request processing method of claim 1, further comprising:
and when the network access request is not from the search engine crawler, returning webpage code for rendering by the request-side browser.
3. The network request processing method of claim 1, further comprising:
determining, by the user agent field, whether the network access request is from a search engine crawler.
4. The method of claim 3, further comprising:
collecting reference identifications of the search engine; and
storing the reference identifier.
5. The method of claim 4, wherein the network access request is determined to be from a search engine crawler when the stored reference identifier contains a user agent field of the network access request.
6. The method of claim 1, wherein the rendering completion event comprises a DOM content triggering event.
7. The network request processing method of claim 1, wherein performing server-side rendering on the single-page application further comprises:
automatically interacting with the non-interface browser engine via a remote control protocol.
8. A network request processing apparatus for a single-page application, the network request processing apparatus comprising:
a receiving unit, configured to receive a network access request for a single-page application;
the rendering unit is used for rendering the single-page application at a server side when the network access request comes from a crawler of a search engine to obtain rendered webpage content; and
a returning unit for returning the rendered web page content,
wherein the rendering unit includes:
the calling subunit is used for calling an engine of the non-interface browser to open the website corresponding to the single-page application;
the monitoring subunit is used for monitoring a rendering completion event returned by the non-interface browser engine;
and the acquisition subunit is used for acquiring the rendered webpage content.
9. The network request processing apparatus according to claim 8, wherein the returning unit is further configured to:
and when the network access request is not from the search engine crawler, returning webpage code for rendering by the request-side browser.
10. The network request processing device according to claim 8, wherein the network request processing device further comprises:
and the determining unit is used for determining whether the network access request comes from a search engine crawler through a user agent field.
11. The network request processing device according to claim 10, wherein the network request processing device further comprises:
a collecting unit for collecting reference identifiers of the search engine; and
and the storage unit is used for storing the reference identification.
12. The network request processing apparatus according to claim 11, wherein the determining unit is configured to: and when the stored reference identification contains the user agent field of the network access request, determining that the network access request comes from a search engine crawler.
13. The network request processing apparatus according to claim 8, wherein the rendering unit further comprises:
and the interaction subunit is used for automatically interacting with the non-interface browser engine through a remote control protocol.
14. A computer-readable medium on which a computer program is stored, the program implementing a network access request processing method for a single-page application according to any one of claims 1 to 7 when executed by a processor.
15. An electronic device, comprising:
one or more processors;
storage means for storing one or more programs which, when executed by the one or more processors, cause the one or more processors to implement the network access request processing method of the single-page application according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810505497.0A CN110598135A (en) | 2018-05-24 | 2018-05-24 | Network request processing method and device, computer readable medium and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810505497.0A CN110598135A (en) | 2018-05-24 | 2018-05-24 | Network request processing method and device, computer readable medium and electronic equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110598135A true CN110598135A (en) | 2019-12-20 |
Family
ID=68848572
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810505497.0A Pending CN110598135A (en) | 2018-05-24 | 2018-05-24 | Network request processing method and device, computer readable medium and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110598135A (en) |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111310005A (en) * | 2020-02-07 | 2020-06-19 | 北京达佳互联信息技术有限公司 | Network request processing method and device, server and storage medium |
| CN111680239A (en) * | 2020-04-14 | 2020-09-18 | 车智互联（北京）科技有限公司 | Page display method |
| CN111708600A (en) * | 2020-08-20 | 2020-09-25 | 腾讯科技（深圳）有限公司 | Page display method, device, equipment and computer readable storage medium |
| CN111881380A (en) * | 2020-06-28 | 2020-11-03 | 百度在线网络技术（北京）有限公司 | Search result processing method and device, electronic equipment and storage medium |
| CN112685667A (en) * | 2020-12-29 | 2021-04-20 | 浪潮软件科技有限公司 | Data interaction method and device, computer readable medium and interaction server |
| CN112836161A (en) * | 2021-01-29 | 2021-05-25 | 北京钛星数安科技有限公司 | High-speed rendering method implemented in remote browser system |
| CN111680239B (en) * | 2020-04-14 | 2024-05-31 | 车智互联（北京）科技有限公司 | Page display method |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102880607A (en) * | 2011-07-15 | 2013-01-16 | 舆情（香港）有限公司 | Dynamic network content grabbing method and dynamic network content crawler system |
| CN103164446A (en) * | 2011-12-14 | 2013-06-19 | 阿里巴巴集团控股有限公司 | Webpage request information response method and webpage request information response device |
| CN103970882A (en) * | 2014-05-19 | 2014-08-06 | 小米科技有限责任公司 | Method and device for rendering page |
| US8909646B1 (en) * | 2012-12-31 | 2014-12-09 | Google Inc. | Pre-processing of social network structures for fast discovery of cohesive groups |
| CN105912693A (en) * | 2016-04-22 | 2016-08-31 | 北京搜狗科技发展有限公司 | Network request processing method and apparatus, network data acquisition method, and server |
| CN107391775A (en) * | 2017-08-28 | 2017-11-24 | 湖北省楚天云有限公司 | A kind of general web crawlers model implementation method and system |
-
2018
- 2018-05-24 CN CN201810505497.0A patent/CN110598135A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102880607A (en) * | 2011-07-15 | 2013-01-16 | 舆情（香港）有限公司 | Dynamic network content grabbing method and dynamic network content crawler system |
| CN103164446A (en) * | 2011-12-14 | 2013-06-19 | 阿里巴巴集团控股有限公司 | Webpage request information response method and webpage request information response device |
| US8909646B1 (en) * | 2012-12-31 | 2014-12-09 | Google Inc. | Pre-processing of social network structures for fast discovery of cohesive groups |
| CN103970882A (en) * | 2014-05-19 | 2014-08-06 | 小米科技有限责任公司 | Method and device for rendering page |
| CN105912693A (en) * | 2016-04-22 | 2016-08-31 | 北京搜狗科技发展有限公司 | Network request processing method and apparatus, network data acquisition method, and server |
| CN107391775A (en) * | 2017-08-28 | 2017-11-24 | 湖北省楚天云有限公司 | A kind of general web crawlers model implementation method and system |
Non-Patent Citations (1)
| Title |
|---|
| 郑铁男等: "《数字编辑运营实训教程》", 知识产权出版社, pages: 214 * |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111310005A (en) * | 2020-02-07 | 2020-06-19 | 北京达佳互联信息技术有限公司 | Network request processing method and device, server and storage medium |
| CN111680239A (en) * | 2020-04-14 | 2020-09-18 | 车智互联（北京）科技有限公司 | Page display method |
| CN111680239B (en) * | 2020-04-14 | 2024-05-31 | 车智互联（北京）科技有限公司 | Page display method |
| CN111881380A (en) * | 2020-06-28 | 2020-11-03 | 百度在线网络技术（北京）有限公司 | Search result processing method and device, electronic equipment and storage medium |
| CN111708600A (en) * | 2020-08-20 | 2020-09-25 | 腾讯科技（深圳）有限公司 | Page display method, device, equipment and computer readable storage medium |
| CN112685667A (en) * | 2020-12-29 | 2021-04-20 | 浪潮软件科技有限公司 | Data interaction method and device, computer readable medium and interaction server |
| CN112836161A (en) * | 2021-01-29 | 2021-05-25 | 北京钛星数安科技有限公司 | High-speed rendering method implemented in remote browser system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110598135A (en) | Network request processing method and device, computer readable medium and electronic equipment | |
| CN110708346B (en) | Information processing system and method | |
| CN108415804B (en) | Method for acquiring information, terminal device and computer readable storage medium | |
| CN108334641B (en) | Method, system, electronic equipment and storage medium for collecting user behavior data | |
| US10084637B2 (en) | Automatic task tracking | |
| CN113010827B (en) | Page rendering method and device, electronic equipment and storage medium | |
| CN109829121B (en) | Method and device for reporting click behavior data | |
| CN110365724B (en) | Task processing method and device and electronic equipment | |
| CN113590974B (en) | Recommendation page configuration method and device, electronic equipment and computer readable medium | |
| CN111274512A (en) | Page loading method, device and medium | |
| CN110929128A (en) | Data crawling method, device, equipment and medium | |
| CN113076294A (en) | Information sharing method and device | |
| CN109862074B (en) | Data acquisition method and device, readable medium and electronic equipment | |
| US10931771B2 (en) | Method and apparatus for pushing information | |
| CN109325192B (en) | Advertisement anti-shielding method and device | |
| CN104954363A (en) | Method and device for generating interface document | |
| CN112395098A (en) | Application program interface calling method and device, storage medium and electronic equipment | |
| CN113971257A (en) | Function menu access method and device for hybrid application | |
| CN110321510A (en) | Page rendering method and system | |
| CN113742551A (en) | Dynamic data capture method based on script and puppeteer | |
| US9680697B2 (en) | Dynamic product installation based on user feedback | |
| CN111338928A (en) | Chrome-based browser testing method and device | |
| CN111666201A (en) | Regression testing method, device, medium and electronic equipment | |
| CN110232019A (en) | Page test method and Related product | |
| CN113590985B (en) | Page jump configuration method and device, electronic equipment and computer readable medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |