WO2020258479A1 - 网页帧的获取方法、服务器及存储介质 - Google Patents

网页帧的获取方法、服务器及存储介质 Download PDF

Info

Publication number
WO2020258479A1
WO2020258479A1 PCT/CN2019/102056 CN2019102056W WO2020258479A1 WO 2020258479 A1 WO2020258479 A1 WO 2020258479A1 CN 2019102056 W CN2019102056 W CN 2019102056W WO 2020258479 A1 WO2020258479 A1 WO 2020258479A1
Authority
WO
WIPO (PCT)
Prior art keywords
webpage
frame
target
snapshot data
web page
Prior art date
Application number
PCT/CN2019/102056
Other languages
English (en)
French (fr)
Inventor
黄学辉
Original Assignee
网宿科技股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 网宿科技股份有限公司 filed Critical 网宿科技股份有限公司
Priority to EP19802025.7A priority Critical patent/EP3783505A4/en
Priority to US16/717,807 priority patent/US20200410032A1/en
Publication of WO2020258479A1 publication Critical patent/WO2020258479A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/40—Information retrieval; Database structures therefor; File system structures therefor of multimedia data, e.g. slideshows comprising image and additional audio data
    • G06F16/48—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
    • G06F16/489—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using time information
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/90—Details of database functions independent of the retrieved data types
    • G06F16/95—Retrieval from the web
    • G06F16/957—Browsing optimisation, e.g. caching or content distillation
    • G06F16/9574—Browsing optimisation, e.g. caching or content distillation of access to content, e.g. by caching
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/90—Details of database functions independent of the retrieved data types
    • G06F16/95—Retrieval from the web
    • G06F16/958—Organisation or management of web site content, e.g. publishing, maintaining pages or automatic linking
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/47—End-user applications
    • H04N21/478—Supplemental services, e.g. displaying phone caller identification, shopping application
    • H04N21/4782—Web browsing, e.g. WebTV

Definitions

  • the embodiments of the present application relate to the field of network technology, and in particular to a method, server, and storage medium for obtaining webpage frames.
  • Webpage snapshot data needs to be cached when obtaining webpage frames.
  • the current caching method is usually to cache webpage snapshot data during webpage rendering, and process the cached webpage snapshot data when users need to obtain webpage frames, so as to obtain users The desired web page frame.
  • the related technology has at least the following problems: the current cache method for web page snapshot data is single, and the efficiency of obtaining web page frames cannot be improved.
  • the purpose of the embodiments of the present application is to provide a method, server, and storage medium for obtaining webpage frames, so as to improve the efficiency of obtaining webpage frames.
  • the implementation of this application provides a method for obtaining webpage frames, which is applied to a server, including: obtaining webpage addresses; obtaining webpage snapshot data in real time according to webpage addresses; and caching the obtained webpages in a preset cache mode Snapshot data; wherein the caching mode includes at least a periodic caching mode; the periodic caching mode is used to cache web page snapshot data within a period; the target web page frame is obtained according to the web page snapshot data cached in a preset caching mode.
  • the embodiment of the present application also provides a server, including: at least one processor; and a memory connected in communication with the at least one processor; wherein the memory stores instructions executable by at least one processor, and the instructions are The processor executes, so that at least one processor can execute the foregoing method for obtaining a webpage frame.
  • the embodiment of the present application also provides a storage medium storing a computer program, and when the computer program is executed by a processor, the method for obtaining the webpage frame described above is realized.
  • the embodiment of the present application caches the web page snapshot data in a preset periodic cache mode after real-time acquisition of web page snapshot data according to the obtained web page address, and the periodic cache mode can cache the web page snapshot data within a period. Therefore, for periodic web page snapshot data, there is no need to cache the web page snapshot data in the second cycle. You only need to obtain the corresponding web page snapshot data from the web page snapshot data in the first cycle. Data acquisition efficiency of the target webpage frame.
  • obtaining the target webpage frame according to the webpage snapshot data cached in the preset cache mode includes: decoding the cached webpage snapshot data to obtain the webpage frame; selecting the target from the webpage frame according to the timestamp of the cached webpage snapshot data Web page frame. In this way, all the cached webpage snapshot data can be decoded at one time, and there is no need to decode again after that, which greatly reduces the number of times of decoding.
  • acquiring the target webpage frame according to the webpage snapshot data cached in the preset cache mode is specifically: selecting target data from the cached webpage snapshot data according to the timestamp of the cached webpage snapshot data; decoding the selected target data, Get the target page frame. In this way, only the selected target data needs to be decoded each time, and the target web page frame is obtained after decoding, and the target web page frame can be directly obtained without selecting from the decoded data.
  • the method further includes: deleting the selected target data. After the selected target data is decoded and the target web page frame is obtained, the decoded target data is deleted, which reduces the memory occupied by the web page snapshot data.
  • caching the obtained webpage snapshot data in a preset cache mode specifically includes: selecting a target cache mode from a plurality of preset cache modes according to the webpage address; and caching the obtained webpage in the selected target cache mode Snapshot data. Every time a webpage frame is obtained according to the webpage address, the cache mode suitable for this webpage snapshot data can be selected from the multiple pre-stored cache modes according to the webpage address, so that the caching mode suitable for the webpage snapshot data is suitable for the webpage snapshot every time. The cache mode of the data.
  • the multiple cache modes further include: a single cache mode; if the target cache mode is a single cache mode, then the selected target cache mode is used to cache the obtained webpage snapshot data, which specifically includes: cache one Web page snapshot data; decode the cached web page snapshot data to obtain the target web page frame; while using the periodic cache mode for caching, the web page snapshot data can also be cached in the single cache mode.
  • the target webpage frame after obtaining the target webpage frame, it also includes: deleting the cached webpage snapshot data. This can reduce the memory occupied by cached web page snapshot data.
  • the target webpage frame is used to synthesize the input video obtained in advance; after the target webpage frame is obtained, between the target webpage frame and the input video obtained in advance, it also includes: the pixel format of the target webpage frame and the The pixel format of the composite video is adjusted to a consistent pixel format.
  • the obtained target webpage frame can be merged with the video to be synthesized, thereby adding text, pictures and animation effects to the video to be synthesized, enriching the video screen of the video to be synthesized, and making the video effect of the video to be synthesized more vivid.
  • Fig. 1 is a flowchart of a method for obtaining a webpage frame according to the first embodiment of the present application
  • FIG. 2 is a flowchart of obtaining webpage snapshot data according to the first embodiment of the present application
  • FIG. 3 is a flowchart of monitoring the status of the browser and the connection with the browser according to the first embodiment of the present application
  • FIG. 4 is a flowchart of a method for obtaining a webpage frame in the second embodiment of the present application
  • FIG. 5 is a flowchart of a method for obtaining a webpage frame when the cached data has not reached one cycle according to the second embodiment of the present application;
  • Fig. 6 is a flowchart of a method for obtaining a webpage frame in the third embodiment of the present application.
  • Fig. 7 is a flowchart of a method for obtaining a webpage frame in a fourth embodiment of the present application.
  • FIG. 8 is a flowchart of a method for obtaining a webpage frame in the fifth embodiment of the present application.
  • Fig. 9 is a schematic structural diagram of a server in a sixth embodiment according to the present application.
  • the first embodiment of the present application relates to a method for obtaining webpage frames, obtaining webpage addresses; obtaining webpage snapshot data in real time according to webpage addresses; caching the obtained webpage snapshot data in a preset cache mode; wherein the cache mode includes at least periodic caching Mode;
  • the periodic cache mode is used to cache web page snapshot data within a period; obtain the target web page frame according to the web page snapshot data cached in the preset cache mode, thereby improving the efficiency of obtaining web page frames according to specific web page snapshot data, and the specific process As shown in Figure 1.
  • Step 101 Obtain a web page address.
  • the obtained webpage address may be an Internet webpage address such as Baidu or Youku Video, or it may be a template address containing a specific webpage frame.
  • the specific webpage frames contained in the template address are specific layout effects or elements, and have a fixed layout style and element layout.
  • Step 102 Acquire web page snapshot data in real time according to the web page address.
  • the server when the server obtains the webpage snapshot data, it can control the browser to access the preset webpage address, control the browser to render the webpage screen and obtain the webpage snapshot data.
  • the controlled browser needs to support the development tool debugging protocol, such as the devtools protocol of Google Chrome, etc.
  • the server can control the browser behavior through the development tool debugging protocol, and obtain the browser page and Network and other related information.
  • the webpage snapshot data obtained in this embodiment is specifically webpage snapshot data with periodic characteristics. Since the webpage snapshot data is decoded, one or more webpage frames will be obtained.
  • the two webpage snapshot data corresponding to the webpage frame are also completely the same.
  • the images from the first webpage frame to the fifth webpage frame are different, but the image of the sixth webpage frame is exactly the same as the image of the first webpage frame mentioned above.
  • the picture is exactly the same as the picture of the second webpage frame above, etc.
  • the picture of the tenth webpage frame is exactly the same as the picture of the fifth webpage frame above
  • the picture of the eleventh webpage frame is again the same as the first webpage frame.
  • Step 201 Start the browser and establish a communication connection with the browser. Specifically, set debugging related parameters such as debugging port and security options as needed, start the browser process, and establish a communication connection with the browser according to the communication protocol of the development tool debugging protocol, so as to send control instructions to the browser and control Browser behavior and get information about browser behavior.
  • debugging related parameters such as debugging port and security options as needed
  • Step 202 Set the page attributes of the browser. Specifically, in order to synthesize the target webpage frame into the video to be synthesized, it is necessary to set the relevant attributes of the browser page.
  • the set page attributes mainly include resolution, background color information, and interactive settings, etc., to ensure that the webpage snapshot data meets the expected Set the page properties.
  • the resolution of the browser page is usually set to be the same as the resolution of the output video screen, and the background color can be set to a transparent color, so that the obtained web page snapshot data, except for text, animation, and input boxes, etc.
  • the background part of the web page is transparent, so that it is convenient to synthesize the obtained target webpage frame with the video to be synthesized, and reduce the processing of the obtained webpage screenshot before synthesis.
  • the operations related to human-computer interaction in the webpage may affect the quality of the webpage snapshot data. Therefore, in the specific implementation process, you can set to prohibit the human-computer interaction-related operations, such as prohibiting webpage pop-up dialogs or webpages not displaying scrolling Article etc.
  • Step 203 Visit the designated website. Specifically, after setting the relevant properties of the webpage, the operation of accessing the preset webpage address to obtain webpage snapshot data can be performed.
  • the server sends a control instruction to the browser to access the specified web address according to the preset webpage address, and the browser is receiving After reaching the control instruction, the designated website will be visited.
  • Step 204 tracking and processing the page process and the network process.
  • the browser accesses the specified web page address, it uses the development tool debugging protocol to send instructions to the browser to track the page process and network process, and receive information about the corresponding page process and network process.
  • the page process includes processes and related events such as page creation, page start loading, and loading completion;
  • the network process includes processes and related events such as the start of a network request, network request failure, network request success, and data reception.
  • step 205 it is judged whether the page is accessed successfully, if the judgment result is yes, go to step 206; if the judgment result is no, go back to step 203. Specifically, if the access fails, a certain number of retries can be performed, and if the failure still occurs after multiple retries, the external error information is fed back and the process is stopped. If the access is successful, you can perform page screenshot operations when the page is loaded to obtain the screenshot data, and save the screenshot data as web page snapshot data to ensure that the captured web page snapshot data is valid data.
  • Step 206 start the screenshot. Specifically, after tracing that the page access is successful and the page loading is completed, a screenshot instruction can be sent to the browser to obtain a screenshot of the current web page returned by the browser as the snapshot data of the current web page. Since the screen content of a web page may be constantly changing or may not change within a certain period of time, different screen content needs to be taken in different ways to take screenshots, so as to save resource consumption while obtaining web page snapshot data.
  • the interval time can be set to 40 milliseconds, so that each frame of the video to be synthesized can correspond to a screenshot of the web content. If the content of the web page does not change within a certain period of time, you can only get the web page snapshot data once during this period of time. You can request the browser through the development tool debugging protocol to set the browser when the content of the web page changes. Only then take the initiative to return to the new screenshot data. In doing so, there is no need to set an interval to send a screenshot command to the browser regularly. You only need to make a request to the browser once, so that when the content of the web page changes, the browser will actively return the screenshot of the latest changed web page content to avoid repeated interception. Content of the page.
  • Step 207 Receive the screenshot data and mark the time stamp. Specifically, after sending a screenshot instruction to the browser, the browser will return the screenshot data of the web page in the next process. Since web pages are generally dynamic, the latest web page screenshot data will be continuously generated. In order to distinguish the screenshot data at different times, the screenshot data needs to be time stamped. If the browser returns the time stamp of the screen corresponding to the screenshot data when returning to the web page page screenshot data, the time stamp returned by the received browser is set as the time stamp of the screenshot data.
  • the system timestamp at the moment when the browser returns the screenshot data of the web page can be used as the timestamp of the screenshot data, and the web page with the timestamp will be marked
  • the screenshot data is saved as the web page snapshot data at the current moment.
  • Step 208 Output the snapshot data.
  • Step 301 Monitor the browser status and connection status.
  • step 302 it is judged whether the browser ends abnormally, if the judgment result is yes, go to step 305 and then return to 301; if the judgment result is no, go to step 303.
  • step 303 it is judged whether the connection is disconnected, if the judgment result is yes, go to step 304; if the judgment result is no, go back to step 301.
  • Step 304 re-establish the connection.
  • Step 305 start the browser and establish a connection.
  • the browser when you control the browser to render the web page and obtain screenshots of the web page, the browser only needs to render the web page content and obtain the screenshot data of the web page, without displaying the web page content on the screen, so you can set the browser to In the non-graphical interface mode, the browser will not create a display window, which can break away from the dependence of the graphical interface, so that the above method for obtaining webpage snapshot data is also applicable to the server side without a graphical interface environment. In addition, since no visible window is created, resource consumption can be reduced.
  • step 103 is executed to cache the obtained webpage snapshot data in a preset cache mode.
  • the preset cache mode for this web page snapshot data also has certain special features. It is assumed that the obtained web page snapshot data is decoded. The number of post-repeat webpage frames is 5, that is, the sixth webpage frame is the same as the webpage picture of the first webpage frame, then the webpage snapshot data is cached in the preset cache mode, that is, the first webpage frame to the first webpage frame Web page snapshot data cache corresponding to five web page frames, this cache mode is called periodic cache mode.
  • Step 104 Obtain a target webpage frame according to the webpage snapshot data cached in a preset cache mode. Specifically, the target webpage frame is obtained from the cached webpage snapshot data. After all the cached webpage snapshot data is decoded, the target webpage frame can be selected from the decoded webpage frame. This only needs to be decoded once, which reduces the decoding time. Action: You can also select the target data in the web page snapshot data first, and then decode the target data to obtain the target web page frame. In this way, the decoded web page snapshot data needs less memory during decoding.
  • the embodiment of the present application caches the web page snapshot data in a preset periodic cache mode after real-time acquisition of web page snapshot data according to the obtained web page address, and the periodic cache mode can cache the web page snapshot data within a period. Therefore, for periodic web page snapshot data, there is no need to cache the web page snapshot data in the second cycle. You only need to obtain the corresponding web page snapshot data from the web page snapshot data in the first cycle. Data acquisition efficiency of the target webpage frame.
  • the second embodiment of the present application relates to a method for obtaining webpage frames.
  • one of the caching modes of the periodic caching mode is specifically described, as shown in FIG. 4.
  • Step 401 Obtain a web page address.
  • Step 402 Acquire web page snapshot data in real time according to the web page address.
  • Step 403 Cache the acquired webpage snapshot data in a periodic cache mode.
  • Step 404 Decode the cached webpage snapshot data to obtain a webpage frame.
  • Step 405 Select the target webpage frame from the webpage frame according to the timestamp.
  • the timestamp of each web page frame in the list is corresponding Generate the timestamp of the webpage snapshot data of the webpage frame, and select the target webpage frame one by one in the webpage frame list.
  • all the cached webpage snapshot data can be decoded at one time, and there is no need to decode the webpage snapshot data again in the subsequent process, only the target webpage frame needs to be selected from the webpage frame list according to the time stamp of the video frame to be synthesized. It can be seen that this implementation of the periodic buffer mode can save the computational resource consumption caused by decoding and format conversion.
  • the above description is for the process of obtaining the target webpage frame according to the cached webpage snapshot data after one cycle of webpage snapshot data is cached in the periodic cache mode.
  • the following is specifically for the case where the cached webpage snapshot data does not reach the one-cycle webpage snapshot data.
  • the process of obtaining the target webpage frame from the cached webpage snapshot data is explained, as shown in FIG. 5 in detail.
  • Step 501 Cache the received webpage snapshot data.
  • Step 502 Decode the latest webpage snapshot data to obtain a target webpage frame.
  • Step 503 Output the target webpage frame.
  • the latest cached webpage snapshot data is decoded to obtain the target webpage frame.
  • webpage frames can also be output during the caching process, without waiting for the cached data to meet a cycle, which can save time waiting for caching.
  • the third embodiment of the present application relates to a method for obtaining webpage frames.
  • the third embodiment further refines the first embodiment, and the main detail is: in the third embodiment of the present application, another caching method of the periodic caching mode is specifically described, as shown in FIG. 6.
  • Step 601 Obtain a web page address.
  • Step 602 Acquire webpage snapshot data in real time according to the webpage address.
  • Step 603 Cache the acquired webpage snapshot data in a periodic cache mode.
  • Step 604 Select target data from the cached web page snapshot data according to the timestamp.
  • Step 605 Decode the selected target data to obtain a target webpage frame.
  • the rendering process of the webpage is stopped, and the desired webpage snapshot data is selected from the cached webpage snapshot data for decoding, thereby obtaining the target webpage frame.
  • the selected target data can be determined according to the timestamp. For example, if the target webpage frame needs to be merged with the video to be synthesized, the target data is determined according to the timestamp of the video to be synthesized.
  • the rules for selecting web page snapshot data are as follows: Suppose the cached web page snapshot data are sorted as WTS 1 , WTS 2 ,...WTS n according to the timestamp, and the timestamps correspond to the web snapshot data of WTS 1 and WTS n
  • the time stamp of the video frame to be synthesized is ITS 1
  • the time stamp of the video frame to be synthesized is ITS At k
  • the target webpage frame is obtained, and the target webpage frame and the video frame to be synthesized are combined into the final output picture.
  • the above description is for the process of obtaining the target webpage frame according to the cached webpage snapshot data after one cycle of webpage snapshot data is cached in the periodic cache mode.
  • the cached webpage snapshot data does not reach one cycle of webpage snapshot data, according to the cached webpage snapshot data
  • the process of obtaining webpage frames from webpage snapshot data is the same as steps 501 to 503 in the second embodiment, and will not be repeated here.
  • the fourth embodiment of the present application relates to a method for obtaining webpage frames.
  • the fourth embodiment further refines the first embodiment, and the main detail is that: in the fourth embodiment of the present application, a single cache mode is also included, as shown in FIG. 7.
  • Step 701 Obtain a web page address.
  • Step 702 Select a target cache mode from a plurality of preset cache modes according to the webpage address.
  • Step 703 Cache the obtained webpage snapshot data in a single cache mode.
  • Step 704 Cache a webpage snapshot data.
  • Step 705 Decode the cached webpage snapshot data to obtain a webpage frame.
  • Step 706 Use the web page frame as the target web page frame.
  • the cached webpage snapshot data is decoded to obtain the webpage frame, and judge whether the obtained webpage frame is consistent with the target webpage frame. If they are the same, the obtained webpage frame will be regarded as the target webpage frame. If they are inconsistent, the next webpage snapshot data will be obtained, and the previously cached webpage snapshot data will be deleted to ensure that the cached webpage snapshot data is always the most recently obtained one Web page snapshot data.
  • the fifth embodiment of the present application relates to a method for obtaining webpage frames.
  • the pixel format of the acquired target webpage frame and the video to be synthesized is adjusted, as shown in FIG. 8.
  • Step 801 Obtain a web page address.
  • Step 802 Acquire webpage snapshot data in real time according to the webpage address.
  • Step 803 Cache the obtained webpage snapshot data in a preset cache mode.
  • Step 804 Obtain the target webpage frame according to the webpage snapshot data cached in the preset cache mode.
  • Step 805 Adjust the pixel format of the target webpage frame and the pixel format of the video to be synthesized into a consistent pixel format.
  • pixel format conversion is required. This can make the target web frame and the video to be synthesized can be combined at the pixel level according to a certain synthesis mode. Synthesis operation.
  • the most common video format to be synthesized is the YUV420P format
  • the web snapshot data is a PNG image
  • the naked video frame after decoding the web snapshot data is in the RGB24 format.
  • the RGB24 format web page frame needs to be pixel formatted and converted to YUV420P format, and then combined with the video to be combined into the final output video.
  • the target webpage frame may be cut out to obtain the webpage content that is finally merged with the video to be synthesized.
  • the background color of the web page of the target web page frame is not a transparent color, it is necessary to perform matting processing on the target web page frame.
  • the matting process can pre-set one or more colors (usually the background color). If the color of a certain pixel in the target webpage frame is within a certain range, the pixel is set to Fully transparent. In this way, the part of the screen corresponding to the meaningful page elements in the target webpage frame can be retained, and finally synthesized into the video to be synthesized.
  • Step 806 Combine the target webpage frame with the video to be synthesized.
  • the video to be synthesized may be a video obtained by means of camera shooting, screen capture, or decoding of the source video.
  • the target webpage frame obtained above is synthesized with the video to be synthesized to enrich the screen content of the video to be synthesized and enhance the expressive power of the video.
  • the target webpage frame can be a webpage frame that contains a special effect, a paragraph of text or a picture, etc.
  • the obtained target webpage frame is merged into the specified position of the video to be synthesized.
  • the final output video screen is simultaneously Contains the content of the video to be synthesized and the content contained in the target webpage frame.
  • each synthesized target webpage frame may be different, that is, the webpage address of the target webpage frame can be obtained during the playback of the synthesized video Real-time changes.
  • the first target webpage frame needs to be synthesized in the second minute of the video to be synthesized
  • the first target webpage frame is obtained from the Baidu webpage
  • the second target webpage frame needs to be synthesized in the fourth minute of the synthesized video.
  • the second target webpage frame is obtained from the Youku video webpage, then in the second minute of the video to be synthesized, the Baidu webpage address is obtained, and then the webpage snapshot data of the Baidu webpage is obtained, and the first one is obtained according to the webpage snapshot data.
  • the target webpage frame is combined with the obtained first target webpage frame and the second-minute video frame of the to-be-composited video.
  • the output video viewed by the user is the to-be-composited video combined with the first target webpage frame.
  • the video is played to the 4th minute, obtain the webpage address of Youku video, and then obtain webpage snapshot data of Youku webpage, obtain the second target webpage frame according to the webpage snapshot data, and combine the obtained second target webpage frame with the video to be synthesized
  • the video frames of the 4th minute of the video are merged, and the output video watched by the user at this time is the to-be-composited video merged with the second target webpage frame. From the output video screen, you can see that the video screen to be synthesized is superimposed with these webpage special effects, making the live broadcast screen more vivid.
  • the output video can be a live video or a video requested by the user. If the output video is a live video, you can use a streaming media server to cache the live video during the live broadcast, and obtain the target webpage frame when the synthesis operation is required, and synthesize the target webpage frame with the cached live video, and finally The synthesized live video output. If the output video is an on-demand video, the target webpage frame is obtained in the process of transcoding the on-demand video by the server, and the target webpage frame is synthesized with the on-demand video, and finally the synthesized on-demand video is output.
  • the target webpage frame When merging the target webpage frame with the video frame to be synthesized, the target webpage frame may be used as the foreground picture, and the video frame to be synthesized may be used as the background picture for synthesis.
  • the screen of the target webpage frame will obscure the image of the video to be synthesized, which can also be understood as the image of the video to be synthesized
  • the landing page frame screen is at the top.
  • the target webpage frame when merging the target webpage frame with the video to be synthesized, you can also use the target webpage frame as the background image and the to-be-composited video as the foreground image for synthesis, that is to say, the image of the to-be-composited video is at the top and the target webpage frame is placed at the bottom .
  • the synthesis method can be divided into full-screen synthesis and area synthesis.
  • Full-screen composition means that the occupied area of the target webpage frame in the final live output screen is equal to the area of the live output screen, that is, the target webpage frame covers the entire live output screen; area composition refers to the target webpage frame in the final live output screen The occupied area is smaller than the area of the live output screen, that is, the target webpage frame only occupies a part of the live output screen.
  • the resolution of the output screen is W o *H o , that is, the width of the live output screen is W o pixels and the height is H o pixels.
  • the resolution of the final live broadcast output screen is the same as the resolution of the video frame to be synthesized.
  • the position of the target webpage frame in the live output screen is (p, q)
  • the size is W w * H w , that is, the width of the target webpage frame in the output video screen is W w pixels and the height is H w pixels.
  • the upper left corner of the target webpage frame is located at a distance of p pixels to the right of the upper left pixel of the output video screen and q pixels below.
  • the transparency of each pixel of the target webpage frame and the video to be synthesized is adjusted, and the adjusted target webpage frame and each pixel of the video to be synthesized are synthesized.
  • the synthesis mode is full-frame synthesis, suppose the pixels in the final synthesized output video picture are Out i, j , where i and j are the position indexes of the pixel in the horizontal and vertical directions, and i represents the pixel in the horizontal The offset of the pixel from the upper left corner of the output video screen in the direction, j represents the offset of the pixel from the upper left corner of the output video screen in the vertical direction, 0 ⁇ i ⁇ W o and 0 ⁇ j ⁇ H o .
  • the pixel value of each pixel in the final output video screen is calculated by the following formula:
  • ⁇ i,j is the transparency value of the pixel at the position (i,j) in the scaled target webpage frame.
  • i,j is the pixel value of the pixel at the position (i,j) in the video to be synthesized
  • Web i,j is the pixel value of the pixel at the position (i,j) in the scaled target webpage frame.
  • each pixel of the target webpage frame is only fully transparent and fully opaque, that is, when the values of ⁇ i,j are only 0 and 1, the pixel value of each pixel in the output video screen is
  • the pixel value of the corresponding position in the live output screen is the pixel value of the pixel in the scaled target webpage frame; if zoomed After a certain pixel in the target webpage frame is completely transparent, the pixel value of the pixel in the corresponding position in the output video screen is the pixel value of the pixel in the corresponding position in the video frame to be synthesized.
  • the fully transparent pixels in the target webpage frame are obtained by matting the background color of the target webpage frame, or are obtained by setting the background color of the webpage to a transparent color when the webpage is rendered by the browser, Then, in the final output video picture, only the non-background parts such as animations and elements in the web page are retained and superimposed on the top of the video frame to be synthesized.
  • the synthesis method is specified area synthesis
  • the value of the pixels in other areas is equal to that of the pixels in the corresponding position of the video to be synthesized value.
  • a synthesis method similar to that of full-screen synthesis is adopted.
  • the pixel value of each pixel in the final live output screen is
  • Area S is the area occupied by the specified zoomed target webpage frame in the output video screen, that is, the area that meets the following restrictions:
  • the resolution of the final output video picture may not be equal to the resolution of the video frame to be synthesized.
  • the video to be synthesized needs to be scaled similar to the target web page frame; for example, the to be synthesized video can be used as the foreground image and the target web frame as Background image; or the video to be synthesized does not cover the entire output video screen, but only occupies a certain area of the final output video screen.
  • the video to be synthesized needs to be synthesized into the specified area according to the location of the area, without covering the entire output video screen .
  • the sixth embodiment of the present application relates to a server. As shown in FIG. 9, it includes at least one processor 901; and, a memory 902 communicatively connected with at least one processor 901; The instructions executed by 901 are executed by the at least one processor 901, so that the at least one processor 901 can execute the foregoing method for obtaining a webpage frame.
  • the memory 902 and the processor 901 are connected in a bus manner.
  • the bus may include any number of interconnected buses and bridges.
  • the bus connects one or more various circuits of the processor 901 and the memory 902 together.
  • the bus can also connect various other circuits such as peripheral devices, voltage regulators, power management circuits, etc., which are all well-known in the art, and therefore, no further description will be given herein.
  • the bus interface provides an interface between the bus and the transceiver.
  • the transceiver may be one element or multiple elements, such as multiple receivers and transmitters, providing a unit for communicating with various other devices on the transmission medium.
  • the data processed by the processor is transmitted on the wireless medium through the antenna, and further, the antenna also receives the data and transmits the data to the processor 901.
  • the processor 901 is responsible for managing the bus and general processing, and can also provide various functions, including timing, peripheral interfaces, voltage regulation, power management, and other control functions.
  • the memory 902 may be used to store data used by the processor 901 when performing operations.
  • the seventh embodiment of the present application relates to a computer-readable storage medium storing a computer program.
  • the computer program is executed by the processor, the above method embodiment is realized.
  • the program is stored in a storage medium and includes several instructions to enable a device ( It may be a single-chip microcomputer, a chip, etc.) or a processor (processor) to execute all or part of the steps of the methods described in the embodiments of the present application.
  • the aforementioned storage media include: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), magnetic disk or optical disk and other media that can store program code .

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Databases & Information Systems (AREA)
  • Data Mining & Analysis (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Library & Information Science (AREA)
  • Signal Processing (AREA)
  • Information Transfer Between Computers (AREA)

Abstract

一种网页帧的获取方法、服务器及存储介质。所述方法包括：获取网页地址(101)；根据网页地址实时获取网页快照数据(102)；以预设的缓存模式缓存获取的网页快照数据(103)；缓存模式至少包括周期缓存模式；周期缓存模式用于缓存一个周期内的网页快照数据；根据预设的缓存模式缓存的网页快照数据获取目标网页帧(104)。

Description

网页帧的获取方法、服务器及存储介质
交叉引用
本申请引用于2019年06月26日递交的名称为“网页帧的获取方法、服务器及存储介质”的第201910559472.3号中国专利申请，其通过引用被全部并入本申请
技术领域
本申请实施例涉及网络技术领域，特别涉及一种网页帧的获取方法、服务器及存储介质。
背景技术
随着HTML5、JavaScript等技术的发展以及各种浏览器前端技术的不断出现，网页帧中涵盖的内容呈现的效果越来越好，用户可以从网络中获取内容丰富的网页帧，并将获取的网页帧应用在日常生活中，比如，将获取的网页帧所涵盖的内容作为表情包发送给好友，等等。在获取网页帧时需要将网页快照数据缓存，目前的缓存方式通常为，在网页渲染过程中对网页快照数据进行缓存，在用户需要获取网页帧时对缓存的网页快照数据进行处理，从而得到用户想要的网页帧。
发明人发现相关技术中至少存在如下问题：目前对网页快照数据的缓存方式单一，无法提高网页帧的获取效率。
发明内容
本申请实施方式的目的在于提供一种网页帧的获取方法、服务器及存储介质，以提高网页帧的获取效率。
为解决上述技术问题，本申请的实施方式提供了一种网页帧的获取方法，应用于服务器，包括：获取网页地址；根据网页地址实时获取网页快照数据；以预设的缓存模式缓存获取的网页快照数据；其中，缓存模式至少包括周期缓存模式；周期缓存模式用于缓存一个周期内的网页快照数据；根据以预设的缓存模式缓存的网页快照数据获取目标网页帧。
本申请的实施方式还提供了一种服务器，包括：至少一个处理器；以及，与至少一个 处理器通信连接的存储器；其中，存储器存储有可被至少一个处理器执行的指令，指令被至少一个处理器执行，以使至少一个处理器能够执行上述网页帧的获取方法。
本申请的实施方式还提供了一种存储介质，存储有计算机程序，计算机程序被处理器执行时实现上述网页帧的获取方法。
本申请实施方式相对于现有技术而言，在根据获取的网页地址实时获取网页快照数据之后，以预设的周期缓存模式缓存网页快照数据，周期缓存模式可以缓存一个周期内的网页快照数据，使得针对呈周期性特性的网页快照数据，无需对第二个周期内的网页快照数据进行缓存，只需在第一个周期内的网页快照数据中获取对应的网页快照数据，从而提高根据网页快照数据获取目标网页帧的效率。
另外，根据以预设的缓存模式缓存的网页快照数据获取目标网页帧，具体包括：将缓存的网页快照数据解码，得到网页帧；根据缓存的网页快照数据的时间戳，在网页帧中选取目标网页帧。这样做可以一次性解码全部缓存的网页快照数据，此后无需再次进行解码操作，大大减少了解码的次数。
另外，根据以预设的缓存模式缓存的网页快照数据获取目标网页帧，具体为：根据缓存的网页快照数据的时间戳在缓存的网页快照数据中选取目标数据；将选取的目标数据进行解码，得到目标网页帧。这样做每次只需针对选取目标数据进行解码，解码后即获取目标网页帧，无需在解码的数据中进行选取便可直接获取目标网页帧。
另外，在得到所述目标网页帧之后，还包括：删除所选取的目标数据。在选取的目标数据解码后得到目标网页帧之后，删除所解码的目标数据，减少了网页快照数据占用的内存。
另外，以预设的缓存模式缓存获取的网页快照数据，具体包括：根据网页地址在预设的多个缓存模式中选取目标缓存模式；以选取的所述目标缓存模式，缓存所述获取的网页快照数据。在每次根据网页地址获取网页帧时，可以根据网页地址在预先存储的多个缓存模式中选择适合本次网页快照数据的缓存模式，使每次缓存网页快照数据的缓存模式均为适合网页快照数据的缓存模式。
另外，多个缓存模式还包括：单缓存模式；若所述目标缓存模式为单缓存模式，则所述以选取的所述目标缓存模式，缓存所述获取的网页快照数据，具体包括：缓存一个网页快照数据；将缓存的网页快照数据进行解码，得到目标网页帧；在利用周期缓存模式进行缓存的同时，还可以通过单缓存模式对网页快照数据进行缓存。
另外，在得到目标网页帧之后，还包括：将缓存的网页快照数据删除。这样做可以减少缓存的网页快照数据占用的内存。
另外，目标网页帧用于与预先获取的输入视频进行合成；在获取目标网页帧之后，在目标网页帧与预先获取的输入视频进行合成之间，还包括：将目标网页帧的像素格式与待合成视频的像素格式调整为一致的像素格式。这样做可以将获取的目标网页帧与待合成视频进行合并，从而在待合成视频中添加文字、图片及动画效果，丰富待合成视频的视频画面，使待合成视频的视频效果更加生动。
附图说明
一个或多个实施例通过与之对应的附图中的图片进行示例性说明，这些示例性说明并不构成对实施例的限定，附图中具有相同参考数字标号的元件表示为类似的元件，除非有特别申明，附图中的图不构成比例限制。
图1是根据本申请第一实施方式中的网页帧的获取方法的流程图；
图2是根据本申请第一实施方式中的获取网页快照数据的流程图；
图3是根据本申请第一实施方式中对浏览器的状态以及与浏览器的连接情况进行监控的流程图；
图4是根据本申请第二实施方式中的网页帧的获取方法的流程图；
图5是根据本申请第二实施方式中在缓存数据尚未达到一个周期时的网页帧的获取方法的流程图；
图6是根据本申请第三实施方式中的网页帧的获取方法的流程图；
图7是根据本申请第四实施方式中的网页帧的获取方法的流程图；
图8是根据本申请第五实施方式中的网页帧的获取方法的流程图；
图9是根据本申请第六实施方式中的服务器的结构示意图。
具体实施方式
为使本申请实施例的目的、技术方案和优点更加清楚，下面将结合附图对本申请的各实施方式进行详细的阐述。然而，本领域的普通技术人员可以理解，在本申请各实施方式中，为了使读者更好地理解本申请而提出了许多技术细节。但是，即使没有这些技术细节和基于以下各实施方式的种种变化和修改，也可以实现本申请所要求保护的技术方案。
以下各个实施例的划分是为了描述方便，不应对本申请的具体实现方式构成任何限定，各个实施例在不矛盾的前提下可以相互结合相互引用。
本申请的第一实施方式涉及一种网页帧的获取方法，获取网页地址；根据网页地址实 时获取网页快照数据；以预设的缓存模式缓存获取的网页快照数据；其中，缓存模式至少包括周期缓存模式；所述周期缓存模式用于缓存一个周期内的网页快照数据；根据预设的缓存模式缓存的网页快照数据获取目标网页帧，从而提高根据特定的网页快照数据获取网页帧的效率，具体流程如图1所示。
步骤101，获取网页地址。具体地说，获取的网页地址可以是百度或优酷视频等互联网的网页地址，也可以是包含特定网页帧的模板地址。模板地址中包含的特定网页帧具体为某些特定的布局效果或元素，具有固定的布局风格及元素排版。
步骤102，根据网页地址实时获取网页快照数据。具体地说，服务器在获取网页快照数据时，可以控制浏览器访问预设网页地址，控制浏览器渲染网页画面并获取网页快照数据。在控制浏览器访问预设网页地址时，所控制的浏览器需支持开发工具调试协议，比如谷歌浏览器的devtools协议等，服务器通过开发工具调试协议可以控制浏览器行为，并获得浏览器页面和网络等相关信息。本实施方式中获取的网页快照数据具体为周期性特性的网页快照数据，由于网页快照数据解码后会得到一个或多个网页帧，如果网页帧内的画面不同，则该网页帧对应的解码的网页快照数据也不同，而两个网页帧内的画面完全相同，则网页帧对应的两个网页快照数据也完全相同。现在有一组网页帧，第一个网页帧至第五个网页帧内的画面均不相同，但是第六个网页帧的画面与上述第一个网页帧的画面完全相同，第七个网页帧的画面与上述第二个网页帧的画面完全相同，等等，第十个网页帧的画面与上述第五个网页帧的画面完全相同，第十一个网页帧的画面再次与第一个网页帧的画面相同，以此类推，这一组网页帧对应的解码的网页快照数据即为本实施方式中具有周期性特性的网页快照数据。
以下针对控制浏览器渲染网页画面并获取网页快照数据进行说明，其过程如图2所示。
步骤201，启动浏览器并与浏览器建立通信连接。具体地说，根据需要设置调试端口、安全选项等调试相关参数，启动浏览器进程，同时根据开发工具调试协议的通信协议，建立与浏览器的通信连接，从而实现向浏览器发送控制指令，控制浏览器的行为并且获得浏览器行为的相关信息。
步骤202，设置浏览器的页面属性。具体地说，为了将目标网页帧合成到待合成视频中，需要设置浏览器页面的相关属性，所设置的页面属性主要包括分辨率、背景颜色信息和交互设置等，以保证网页快照数据满足预设的页面属性。在具体实施过程中，通常将浏览器页面的分辨率设置为和输出视频画面的分辨率相同，背景颜色可以设置为透明色，这样做使获得的网页快照数据，除文字、动画以及输入框等网页页面元素外，网页背景部分是透明的，从而方便将获取的目标网页帧与待合成视频进行合成，减少在合成之前对获取的网页截图的 处理。另外，网页中的人机交互的相关的操作，可能会影响获取网页快照数据的质量，因此在具体实施过程中，可以设置禁止人机交互相关操作，例如禁止网页弹出对话框或网页不显示滚动条等。
步骤203，访问指定网址。具体地说，在将网页相关属性设置完成后，即可进行访问预设网页地址获取网页快照数据的操作，服务器根据预设的网页地址向浏览器发送访问指定网址的控制指令，浏览器在收到该控制指令后，将访问指定的网址。
步骤204，追踪和处理页面过程与网络过程。具体地说，在浏览器访问指定网页地址的同时，使用开发工具调试协议向浏览器发送追踪页面过程和网络过程的指令，并接收处理相应的页面过程和网络过程相关的信息。所述页面过程包括：页面的创建、页面开始加载以及加载完成等过程和相关事件；所述网络过程包括：网络请求的开始、网络请求失败、网络请求成功和数据接收等过程和相关事件。通过追踪页面过程和网络过程，可以知道指定网页是否访问成功。
步骤205，判断页面是否访问成功，若判断结果为是，则进入步骤206；若判断结果为否，则返回步骤203。具体地说，如果访问失败，则可以进行一定次数的重试，如果重试多次后仍然失败，则反馈给外部错误信息并停止本过程。如果访问成功，那么可以在页面加载完成时进行页面截图操作，获取截图数据，并将截图数据作为网页快照数据保存，以保证截取的网页快照数据为有效数据。
步骤206，开启截图。具体地说，在追踪到页面访问成功且页面加载完成后，即可以向浏览器发送截图指令，获取浏览器返回的当前时刻的网页页面的截图，作为当前网页的快照数据。由于网页的画面内容可能是不断变化的，也可能在某段时间内没有变化，针对不同的画面内容，需要采用不同的截图方式，以便可以在获取网页快照数据的同时节省资源的消耗。
在网页的画面内容不断变化，比如网页可能包含动画或滚动的文字等时，可以定时地向浏览器发送截图指令，以获得最新的画面内容。定时的间隔可以根据待合成视频画面的帧率进行设置，比如待合成视频画面的帧率是每秒25帧时，那么一帧的持续时间就是1/25＝0.04秒即40毫秒。意味着每隔40毫秒就会有一帧输入视频帧。因为我们需要将网页内容合成到输入视频中，而每隔40毫秒就有一个输入视频帧，那么每隔40毫秒同时也就需要一个网页帧。所以可以将间隔时间设置为40毫秒，从而实现待合成视频的每一帧都可以对应一个网页内容截图。如果网页画面内容在某段时间内没有变化时，可以在这段时间内只获取一次网页快照数据即可，可以通过开发工具调试协议向浏览器请求，设置当网页页面内容发生变化时， 浏览器才主动返回新的画面截图数据。这样做无需设定一个间隔时间定时向浏览器发送截图指令，只需要向浏览器请求一次，即可使得当网页页面内容有变化时浏览器主动返回最新变化的网页页面内容的截图，避免截取重复的网页内容。
步骤207，接收截图数据并标记时间戳。具体地说，在向浏览器发送截图指令后，浏览器会在接下来的过程中返回网页页面的截图数据。由于网页一般是动态的，因此会不断地生成最新的网页页面截图数据，为了区分不同时刻的截图数据，需要对截图数据进行时间戳标记。若浏览器在返回网页页面截图数据的时候，同时返回浏览器渲染该截图数据对应的画面时刻的时间戳，则将接收到的浏览器传回的时间戳设置为该截图数据的时间戳。若浏览器返回网页页面截图数据时不附带对应的时间戳，则可以将收到浏览器返回该网页页面截图数据时刻的系统时间戳作为该截图数据的时间戳，将标记了时间戳的网页页面截图数据作为当前时刻的网页快照数据保存。
步骤208，输出快照数据。
另外，在启动浏览器后，对浏览器的状态以及与浏览器的连接情况进行监控。具体如图3所示。
步骤301，对浏览器状态和连接情况进行监控。
步骤302，判断浏览器是否异常结束，若判断结果为是，则进入步骤305后返回301；若判断结果为否，则进入步骤303。
步骤303，判断连接是否断开，若判断结果为是则进入步骤304；若判断结果为否，则返回步骤301。
步骤304，重新建立连接。
步骤305，启动浏览器并建立连接。
具体地说，在通过开发工具调试协议控制浏览器并获得网页页面截图数据的过程中，需要对浏览器进行监控并且对与浏览器的连接进行监控。如果浏览器正常运行但是监控到与浏览器的连接异常断开，则需要重新建立和浏览器的通信连接后，再继续原来的控制指令发送和响应接收操作。如果监控到浏览器异常结束，则需要重新启动浏览器，并再次检测浏览器的状态及与浏览器的连接情况。
在实际应用中，通过控制浏览器渲染网页并获取网页页面截图时，浏览器只需要渲染出网页内容，获得网页页面的截图数据，不需要将网页内容显示在屏幕中，因此可以设置浏览器为无图形界面模式，浏览器不会创建显示窗口，从而能够脱离图形界面的依赖，使上述获取网页快照数据的方放同样适用于没有图形界面环境的服务器端。另外，由于没有创建可 见窗口，因此能够降低资源消耗。
在获取网页快照数据之后，执行步骤103，以预设的缓存模式缓存获取的网页快照数据。具体地说，在上述说明中具体说明了本实施方式中获取的网页快照数据的周期性，针对这种网页快照数据预设的缓存模式也具有一定的特殊之处，假设获取的网页快照数据解码后重复网页帧的数量为5帧，即第六帧网页帧与第一帧网页帧的网页画面相同，那么以预设缓存模式对网页快照数据进行缓存，也就是将第一帧网页帧至第五帧网页帧对应的网页快照数据缓存，这种缓存模式称为周期缓存模式。
步骤104，根据预设的缓存模式缓存的网页快照数据获取目标网页帧。具体地说，在缓存的网页快照数据中获取目标网页帧，可以将缓存的网页快照数据全部解码后，在解码后的网页帧中选取目标网页帧，这样做只需解码一次，减少了解码的动作；也可以先在网页快照数据中选择目标数据，再将目标数据解码得到目标网页帧，这样做解码时解码的网页快照数据需要占用的内存较少。
本申请实施方式相对于现有技术而言，在根据获取的网页地址实时获取网页快照数据之后，以预设的周期缓存模式缓存网页快照数据，周期缓存模式可以缓存一个周期内的网页快照数据，使得针对呈周期性特性的网页快照数据，无需对第二个周期内的网页快照数据进行缓存，只需在第一个周期内的网页快照数据中获取对应的网页快照数据，从而提高根据网页快照数据获取目标网页帧的效率。
本申请的第二实施方式涉及一种网页帧的获取方法。在本申请第二实施方式中，对周期缓存模式的其中一种缓存方式进行具体说明，如图4所示。
步骤401，获取网页地址。
步骤402，根据网页地址实时获取网页快照数据。
步骤403，以周期缓存模式缓存获取的网页快照数据。
步骤404，将缓存的网页快照数据解码，得到网页帧。
步骤405，根据时间戳在网页帧中选取目标网页帧。
具体地说，一次性将所有缓存的网页快照数据解码，再对解码后的全部裸视频帧进行必要的像素格式转换后，形成一个网页帧列表，列表中每个网页帧的时间戳为对应的生成该网页帧的网页快照数据的时间戳，在网页帧列表中逐一挑选目标网页帧。这样做能够一次性解码完所有缓存的网页快照数据，在此后的过程中无需再次解码网页快照数据，只需要从网页帧列表中按照待合成视频帧的时间戳选择目标网页帧即可。可见，这种周期缓存模式的实施方式能够节省解码和格式转换所带来的计算资源消耗。
上述说明是针对周期缓存模式下缓存一个周期的网页快照数据后，根据缓存的网页快照数据获取目标网页帧的过程，以下具体针对缓存的网页快照数据未达到一周期网页快照数据的情况下，根据缓存的网页快照数据得到目标网页帧的过程进行说明，具体如图5所示。
步骤501，缓存接收的网页快照数据。
步骤502，解码最新的网页快照数据，得到目标网页帧。
步骤503，输出目标网页帧。
具体地说，在缓存数据未达到一个周期时，则将最新缓存的网页快照数据进行解码，从而得到目标网页帧。这样做在缓存过程中也可以输出网页帧，无需等待缓存数据满足一个周期，可以节省等待缓存的时间。
本申请的第三实施方式涉及一种网页帧的获取方法。第三实施方式将第一实施方式进一步细化，主要细化之处在于：在本申请第三实施方式中，对周期缓存模式的另外一种缓存方式进行具体说明，如图6所示。
步骤601，获取网页地址。
步骤602，根据网页地址实时获取网页快照数据。
步骤603，以周期缓存模式缓存获取的网页快照数据。
步骤604，根据时间戳在缓存的网页快照数据中选取目标数据。
步骤605，将选取的目标数据解码，得到目标网页帧。
具体地说，在以周期缓存模式缓存一个周期的网页快照数据后，停止网页的渲染过程，在缓存的网页快照数据中选择所需的网页快照数据进行解码，从而得到目标网页帧。在选择目标数据时可以根据时间戳决定所选择的目标数据，例如，若目标网页帧需要与待合成视频进行合并，则根据待合成视频的时间戳决定目标数据。选择网页快照数据的规则具体如下：假设缓存的网页快照数据按照时间戳从小到大排列依次为WTS 1，WTS 2，……WTS n，时间戳为WTS 1和WTS n的网页快照数据所对应的网页画面内容为相同的网页画面内容，代表一个周期的开始和结束，由此可知缓存的网页快照数据对应的网页画面持续时间T为T＝WTS n-WTS 1。以上述将目标网页帧与待合成视频进行合并为例，假设开始将目标网页帧合成到直播输出画面时，待合成视频帧的时间戳为ITS 1，那么当待合成视频帧的时间戳为ITS k时，从缓存的网页快照中，按照时间戳从大到小的顺序寻找第一个满足时间戳小于或等于mod(ITS k-ITS 1,T)+WTS 1的网页快照数据，作为目标数据进行解码，得到目标网页帧，将目标网页帧与待合成视频帧一起合成最终的输出画面。上述mod代表广义的求余数操作，即对于两个正数x和y，mod(x,y)为一个大于或等于0且小于y的数使得存在一个非负整数p满足 x＝p*y+mod(x,y)。为了减少内存的占用率，在实际应用中可以在将目标数据解码，得到目标网页帧之后，将目标数据删除，从而减少了缓存的网页快照数据占用的内存。
上述说明是针对周期缓存模式下缓存一个周期的网页快照数据后，根据缓存的网页快照数据获取目标网页帧的过程，针对缓存的网页快照数据未达到一周期网页快照数据的情况下，根据缓存的网页快照数据得到网页帧的过程与第二实施方式中步骤501至503一致，在此不再赘述。
本申请的第四实施方式涉及一种网页帧的获取方法。第四实施方式将第一实施方式进一步细化，主要细化之处在于：在本申请第四实施方式中，还包括单缓存模式，如图7所示。
步骤701，获取网页地址。
步骤702，根据网页地址在预设的多个缓存模式中选取目标缓存模式。
步骤703，以单缓存模式缓存获取的网页快照数据。
步骤704，缓存一个网页快照数据。
步骤705，将缓存的网页快照数据解码，得到网页帧。
步骤706，将网页帧作为目标网页帧。
具体地说，以单缓存模式缓存网页快照数据时，只需缓存最新的一个网页快照数据，并将缓存的一个网页快照数据解码，得到网页帧，判断得到的网页帧与目标网页帧是否一致，若一致，则将获取到的网页帧作为目标网页帧，若不一致，则获取下一个网页快照数据，并将之前缓存的网页快照数据删除，保证缓存的网页快照数据总是为最近得到的那一个网页快照数据。
本申请的第五实施方式涉及一种网页帧的获取方法。在本申请第五实施方式中，将获取的目标网页帧与待合成视频的像素格式做出了调整，具体如图8所示。
步骤801，获取网页地址。
步骤802，根据网页地址实时获取网页快照数据。
步骤803，以预设的缓存模式缓存获取的网页快照数据。
步骤804，根据预设的缓存模式缓存的网页快照数据获取目标网页帧。
步骤805，将目标网页帧的像素格式与待合成视频的像素格式调整为一致的像素格式。具体地说，当待合成视频的像素格式和目标网页帧不一致时，需要进行像素格式的转换，这样做可以使目标网页帧与待合成视频能够在像素级别上对两者按照一定的合成模式进行合成操作。例如：待合成视频格式最为常见的是YUV420P格式，而网页快照数据是PNG图像，网页快照数据解码后的裸视频帧为RGB24格式。此时需要将RGB24格式的网页帧进行像素 格式化转换，转换为YUV420P格式后，与待合成视频一起合成最终输出视频。
另外，在获得目标网页帧之后，可以通过对目标网页帧进行抠图，得到最终与待合成视频进行合并的网页内容。具体地说，如果目标网页帧的网页页面的背景色不是透明色，则需要对目标网页帧进行抠图处理。抠图处理可以预先设定一种或多种颜色(一般是背景颜色)，如果目标网页帧画面中某个像素点的颜色和设定颜色接近程度在一定范围内，则将该像素点设置为全透明。这样做可以保留目标网页帧中有意义的页面元素所对应的画面部分，最终合成到待合成视频中。
步骤806，将目标网页帧与待合成视频合并。
具体地说，待合成视频可以是由摄像机拍摄、屏幕采集或者解码源视频等途径得到的视频。将上述获取的目标网页帧与待合成视频进行合成，丰富待合成视频的画面内容，增强视频的表现力。例如，目标网页帧可以是包含一种特效，一段文字或一张图片等的网页帧，将获取的目标网页帧合并至待合成视频的指定位置，经过合成操作后，最终的输出视频画面中同时包含了待合成视频的内容以及目标网页帧包含的内容。由于在待合成视频中合成的目标网页帧的数量可能为多个，每个合成的目标网页帧可以是不同的，也就是说，获取目标网页帧的网页地址可以在待合成视频的播放过程中实时更改。例如，在待合成视频的第2分钟需要合成第一个目标网页帧，第一个目标网页帧是从百度网页中获取的，在待合成视频的第4分钟需要合成第二个目标网页帧，而第二个目标网页帧是从优酷视频网页中获取的，那么在待合成视频播放的第2分钟，获取百度的网页地址，进而获取百度网页的网页快照数据，根据网页快照数据得到第一个目标网页帧，并将获取的第一个目标网页帧与待合成视频的第2分钟的视频帧进行合并，这时用户观看的输出视频即为合并了第一个目标网页帧的待合成视频。在视频播放至第4分钟时，获取优酷视频的网页地址，进而获取优酷网页的网页快照数据，根据网页快照数据得到第二个目标网页帧，将获取的第二个目标网页帧与待合成视频的第4分钟的视频帧进行合并，这时用户观看的输出视频即为合并了第二个目标网页帧的待合成视频。从输出视频画面中可以观看到待合成视频画面叠加了这些网页特效效果，使得直播画面更为生动形象。
在实际应用中，输出视频可以为直播视频，也可以为用户点播的视频。若该输出视频为直播视频，则可以利用流媒体服务器，在直播过程对直播视频进行缓存，并在需要进行合成操作时获取目标网页帧，将目标网页帧与缓存的直播视频进行合成，最终将合成后的直播视频输出。若该输出视频为点播视频，则在服务器对点播视频转码过程中获取目标网页帧，并将目标网页帧与点播视频合成，最终将合成后的点播视频输出。
在将目标网页帧与待合成视频帧合并时，可以将目标网页帧作为前景画面，待合成视频帧作为背景画面进行合成。也就是说如果在输出画面的某个区域中待合成视频和目标网页帧的画面内容有重叠，那么目标网页帧的画面将会遮挡待合成视频的画面，也可以理解为待合成视频的画面在下方，目标网页帧画面在上方。同样，在将目标网页帧与待合成视频合并时，还可以将目标网页帧作为背景画面，待合成视频作为前景画面进行合成，也就是说待合成视频的画面在上方，目标网页帧放在下方。
根据目标网页帧在最终合成的输出画面中的大小，可以将合成方式分为全画面合成和区域合成。全画面合成是指，目标网页帧在最终直播输出画面中的占用面积等于直播输出画面的面积，即目标网页帧铺满整个直播输出画面；区域合成是指目标网页帧在最终直播输出画面中的占用面积小于直播输出画面的面积，即目标网页帧只占据直播输出画面的一部分区域。
在实际应用中，假设输出画面的分辨率为W o*H o，即直播输出画面的宽为W o个像素、高为H o个像素。一般而言，最终直播输出画面的分辨率大小和待合成视频帧的分辨率大小一致。设目标网页帧在直播输出画面中的位置为(p，q)，大小为W w*H w，即目标网页帧在输出视频画面中的宽为W w个像素、高为H w个像素，且目标网页帧左上角像素点位于输出视频画面左上角像素点右侧p个像素点、下方q个像素点的距离。假设目标网页帧中每个像素点的透明度归一化到[0，1]区间，其中透明度为0代表全透明，透明度为1代表完全不透明。使目标网页帧和待合成输出视频帧合成最终直播输出画面的步骤如下：
首先，对待合成的网页内容进行缩放。若合成方式为全画面合成，则在目标网页帧的分辨率大小与输出视频画面的分辨率大小不一致时，将目标网页帧缩放到和输出视频画面一样的大小，即使得W w＝W o,H w＝H o；若合成方式为指定区域合成，则根据配置的区域大小，将目标网页帧缩放到适配指定的区域大小。
其次，对目标网页帧和待合成视频的各像素点的透明度进行调节，并将调节后的目标网页帧和待合成视频的各像素点进行合成。在合成方式为全画面合成时，设最终合成的输出视频画面中的像素点为Out i,j，其中i和j分别为像素点在水平和竖直方向的位置索引，i代表像素点在水平方向上距离输出视频画面左上角像素点的偏移，j代表像素点在竖直方向上距离输出视频画面左上角像素点的偏移，0≤i<W o且0≤j<H o。最终的输出视频画面中的各像素点的像素值通过以下公式进行计算：
Out i,j＝(1-α i,j)*In i,j+α i,j*Web i,j，
其中，α i,j为缩放后的目标网页帧中位置为(i，j)的像素点的透明度值。In i,j为待合成 视频中位置为(i，j)的像素点的像素值，Web i,j为缩放后的目标网页帧中位置为(i，j)的像素点的像素值。
例如，当目标网页帧每个像素的透明度只有全透明和完全不透明两种情况时，即α i,j的取值只有0和1两种情况时，输出视频画面中各像素点的像素值为
Figure PCTCN2019102056-appb-000001
由上述公式可知，如果缩放后的目标网页帧中某个像素点完全不透明，则直播输出画面中相应位置的像素点的像素值为缩放后的目标网页帧中该像素点的像素值；如果缩放后的目标网页帧中某个像素点完全透明，则输出视频画面中相应位置的像素点的像素值为待合成视频帧中相应位置的像素点的像素值。如果目标网页帧中全透明的像素点是对目标网页帧中的背景颜色进行抠图得到的，或者是在使用浏览器渲染网页页面画面时，将网页的背景颜色设置为了透明色而得到的，那么最终输出视频画面中只有网页页面中的动画、元素等非背景部分保留了下来，叠加到了待合成视频帧的上方。
另外，在合成方式为指定区域合成时，在最终合成的输出视频画面中，除了指定的缩放后的目标网页帧所占据的区域，其它区域的像素点的值等于待合成视频对应位置像素点的值。在指定的缩放后的目标网页帧所占据的区域，采用和全画面合成类似的合成方法。最终的直播输出画面中各像素点的像素值为
Figure PCTCN2019102056-appb-000002
区域S为输出视频画面中，指定的缩放后的目标网页帧所占据的区域，即满足以下限制条件的区域：
p≤i<p+W w且q≤j<q+H w。
值得一提的是，以上实例为了对本申请实施方式的合成原理进行具体说明所做的假设，在实际生活中不限于上述合成方式，可以根据具体的需求，进行不同的合成方案。比如最终输出视频画面的分辨率也可以不等于待合成视频帧的分辨率，此时需要对待合成视频进行类似目标网页帧的缩放；再比如也可以把待合成视频作为前景画面，目标网页帧作为背景画面；或者待合成视频并不铺满整个输出视频画面，只占据最终输出视频画面的某个区域，此时需要将待合成视频根据区域位置合成到指定区域内，无需铺满整个输出视频画面。
上面各种方法的步骤划分，只是为了描述清楚，实现时可以合并为一个步骤或者对某些步骤进行拆分，分解为多个步骤，只要包括相同的逻辑关系，都在本专利的保护范围内；对算法中或者流程中添加无关紧要的修改或者引入无关紧要的设计，但不改变其算法和流程 的核心设计都在该专利的保护范围内。
本申请第六实施方式涉及一种服务器，如图9所示，包括至少一个处理器901；以及，与至少一个处理器901通信连接的存储器902；其中，存储器902存储有可被至少一个处理器901执行的指令，指令被至少一个处理器901执行，以使至少一个处理器901能够执行上述网页帧获取方法。
其中，存储器902和处理器901采用总线方式连接，总线可以包括任意数量的互联的总线和桥，总线将一个或多个处理器901和存储器902的各种电路连接在一起。总线还可以将诸如外围设备、稳压器和功率管理电路等之类的各种其他电路连接在一起，这些都是本领域所公知的，因此，本文不再对其进行进一步描述。总线接口在总线和收发机之间提供接口。收发机可以是一个元件，也可以是多个元件，比如多个接收器和发送器，提供用于在传输介质上与各种其他装置通信的单元。经处理器处理的数据通过天线在无线介质上进行传输，进一步，天线还接收数据并将数据传送给处理器901。
处理器901负责管理总线和通常的处理，还可以提供各种功能，包括定时、外围接口、电压调节、电源管理以及其他控制功能。而存储器902可以被用于存储处理器901在执行操作时所使用的数据。
本申请第七实施方式涉及一种计算机可读存储介质，存储有计算机程序。计算机程序被处理器执行时实现上述方法实施例。
即，本领域技术人员可以理解，实现上述实施例方法中的全部或部分步骤是可以通过程序来指令相关的硬件来完成，该程序存储在一个存储介质中，包括若干指令用以使得一个设备(可以是单片机，芯片等)或处理器(processor)执行本申请各个实施例所述方法的全部或部分步骤。而前述的存储介质包括：U盘、移动硬盘、只读存储器(ROM，Read-Only Memory)、随机存取存储器(RAM，Random Access Memory)、磁碟或者光盘等各种可以存储程序代码的介质。
本领域的普通技术人员可以理解，上述各实施方式是实现本申请的具体实施例，而在实际应用中，可以在形式上和细节上对其作各种改变，而不偏离本申请的精神和范围。

Claims (10)

  1. 一种网页帧的获取方法，应用于服务器，其中，包括：
    获取网页地址；
    根据所述网页地址实时获取网页快照数据；
    以预设的缓存模式缓存所述获取的网页快照数据；其中，所述缓存模式包括周期缓存模式；所述周期缓存模式用于缓存一个周期内的网页快照数据；
    根据以所述预设的缓存模式缓存的网页快照数据获取目标网页帧。
  2. 根据权利要求1所述的网页帧的获取方法，其中，所述根据以所述预设的缓存模式缓存的网页快照数据获取目标网页帧，具体包括：
    将所述缓存的网页快照数据解码，得到网页帧；
    根据所述缓存的网页快照数据的时间戳，在所述网页帧中选取目标网页帧。
  3. 根据权利要求1所述的网页帧的获取方法，其中，所述根据以所述预设的缓存模式缓存的网页快照数据获取目标网页帧，具体为：
    根据所述缓存的网页快照数据的时间戳在所述缓存的网页快照数据中选取目标数据；
    将所述选取的目标数据进行解码，得到所述目标网页帧。
  4. 根据权利要求3所述的网页帧的获取方法，其中，在所述得到所述目标网页帧之后，还包括：
    删除所述选取的目标数据。
  5. 根据权利要求1所述的网页帧的获取方法，其中，所述以预设的缓存模式缓存所述获取的网页快照数据，具体包括：
    根据所述网页地址在预设的多个缓存模式中选取目标缓存模式；
    以选取的所述目标缓存模式，缓存所述获取的网页快照数据。
  6. 根据权利要求5所述的网页帧的获取方法，其中，所述多个缓存模式至少包括：单缓存模式；
    若所述目标缓存模式为单缓存模式，则所述以选取的所述目标缓存模式，缓存所述获取的网页快照数据，具体包括：
    缓存一个网页帧对应的网页快照数据；
    将所述缓存的网页快照数据进行解码，得到所述目标网页帧。
  7. 根据权利要求6所述的网页帧的获取方法，其中，在所述得到所述目标网页帧之后，还包括：
    将缓存的所述网页快照数据删除。
  8. 根据权利要求1至7中任一项所述的网页帧的获取方法，其中，所述目标网页帧用于与预先获取的输入视频进行合成；
    在所述获取目标网页帧之后，在所述目标网页帧与预先获取的输入视频进行合成之间，还包括：
    将所述目标网页帧的像素格式与待合成视频的像素格式调整为一致的像素格式。
  9. 一种服务器，其中，包括：
    至少一个处理器；以及，
    与所述至少一个处理器通信连接的存储器；其中，
    所述存储器存储有可被所述至少一个处理器执行的指令，所述指令被所述至少一个处理器执行，以使所述至少一个处理器能够执行如权利要求1至8中任一所述的网页帧的获取方法。
  10. 一种计算机可读存储介质，存储有计算机程序，其中，所述计算机程序被处理器执行时实现权利要求1至8中任一项所述的网页帧的获取方法。
PCT/CN2019/102056 2019-06-26 2019-08-22 网页帧的获取方法、服务器及存储介质 WO2020258479A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
EP19802025.7A EP3783505A4 (en) 2019-06-26 2019-08-22 METHOD OF DETECTING WEBPAGE FRAMES, SERVER AND STORAGE MEDIUM
US16/717,807 US20200410032A1 (en) 2019-06-26 2019-12-17 Method for acquiring webpage frame, server, and storage medium

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201910559472.3A CN110457623A (zh) 2019-06-26 2019-06-26 网页帧的获取方法、服务器及存储介质
CN201910559472.3 2019-06-26

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US16/717,807 Continuation US20200410032A1 (en) 2019-06-26 2019-12-17 Method for acquiring webpage frame, server, and storage medium

Publications (1)

Publication Number Publication Date
WO2020258479A1 true WO2020258479A1 (zh) 2020-12-30

Family

ID=68481103

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/102056 WO2020258479A1 (zh) 2019-06-26 2019-08-22 网页帧的获取方法、服务器及存储介质

Country Status (3)

Country Link
EP (1) EP3783505A4 (zh)
CN (1) CN110457623A (zh)
WO (1) WO2020258479A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114143490A (zh) * 2021-11-29 2022-03-04 云门(深圳)技术有限公司 一种在网页中预览sdi视频信号的方法

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105808659A (zh) * 2016-02-29 2016-07-27 努比亚技术有限公司 移动终端及其网页截图方法
CN106533926A (zh) * 2016-12-27 2017-03-22 武汉斗鱼网络科技有限公司 一种网页信息传播方法以及装置
CN109271092A (zh) * 2018-09-21 2019-01-25 真相网络科技（北京）有限公司 网页快照保存方法
US20190171351A1 (en) * 2008-05-21 2019-06-06 Please Don't Go, LLC. Messaging window overlay for a browser
CN109889882A (zh) * 2019-01-24 2019-06-14 北京亿幕信息技术有限公司 一种视频剪辑合成方法和系统

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8891939B2 (en) * 2009-12-22 2014-11-18 Citrix Systems, Inc. Systems and methods for video-aware screen capture and compression
CN106780685B (zh) * 2017-03-23 2018-09-04 维沃移动通信有限公司 一种动态图片的生成方法及终端

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20190171351A1 (en) * 2008-05-21 2019-06-06 Please Don't Go, LLC. Messaging window overlay for a browser
CN105808659A (zh) * 2016-02-29 2016-07-27 努比亚技术有限公司 移动终端及其网页截图方法
CN106533926A (zh) * 2016-12-27 2017-03-22 武汉斗鱼网络科技有限公司 一种网页信息传播方法以及装置
CN109271092A (zh) * 2018-09-21 2019-01-25 真相网络科技（北京）有限公司 网页快照保存方法
CN109889882A (zh) * 2019-01-24 2019-06-14 北京亿幕信息技术有限公司 一种视频剪辑合成方法和系统

Non-Patent Citations (4)

* Cited by examiner, † Cited by third party
Title
ANGRY BANANA: "SiteShoter Green Chinese Version 1.42 I Webpage screenshot (support timing and batch processing", SOFTWARE NO1, 3 February 2016 (2016-02-03), pages 1 - 3, XP055700435, Retrieved from the Internet <URL:https://www.rjno1.com/siteshoter-make-webpage-screenshot-support-batch-and-timing> *
ANGRY BANANA: "SiteShoter Green Chinese Version 1.42 I Webpage screenshot (support timing and batch processing", SOFTWARE NO1,20160203, 3 February 2016 (2016-02-03), pages 1 - 8, XP055700435 *
ANONYMOUS: "Selenium – testclass.cn", SELENIUM SCREENSHOT OF THE CURRENT BROWSER WINDOW, 29 January 2019 (2019-01-29), pages 1 - 6, XP055705239, Retrieved from the Internet <URL:https://www.testclass.cn/selenium_screenshot.html> *
See also references of EP3783505A4 *

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114143490A (zh) * 2021-11-29 2022-03-04 云门(深圳)技术有限公司 一种在网页中预览sdi视频信号的方法

Also Published As

Publication number Publication date
EP3783505A1 (en) 2021-02-24
CN110457623A (zh) 2019-11-15
EP3783505A4 (en) 2021-02-24

Similar Documents

Publication Publication Date Title
WO2020258480A1 (zh) 视频生成方法、装置、服务器及存储介质
US10984067B2 (en) Video generating method, apparatus, server, and storage medium
US20210193184A1 (en) Image information processing method and apparatus, and storage medium
JP4852674B2 (ja) 特定のクライアント装置タイプに対して最適化された動的ビューポート階層化を提供する画像システム
US8667054B2 (en) Systems and methods for networked, in-context, composed, high resolution image viewing
US20080222273A1 (en) Adaptive rendering of web pages on mobile devices using imaging technology
US10423527B2 (en) Memory management and image display for mobile devices
US20220007083A1 (en) Method and stream-pushing client for processing live stream in webrtc
CN103514179B (zh) 网络浏览器切换历史网页的方法及网络浏览器
KR20150091132A (ko) 페이지 렌더링 방법 및 장치
US11494071B2 (en) Method for displaying handwritten input content, electronic device and computer storage medium
CN102684986B (zh) 一种终端显示图片的方法和系统
US20190230311A1 (en) Video interface display method and apparatus
US10630755B2 (en) Selective consumption of web page data over a data-limited connection
CN106162301A (zh) 一种信息推送方法
CN111246272A (zh) 视频封面图的显示方法及装置
WO2020258479A1 (zh) 网页帧的获取方法、服务器及存储介质
US20200410032A1 (en) Method for acquiring webpage frame, server, and storage medium
JP2013073506A (ja) 画像処理装置、その制御方法、及びプログラム
CN112419456B (zh) 一种特效画面生成方法和装置
CN106557529B (zh) 页面跳转中的跳转页面展示方法和装置
JP2008123234A (ja) 翻訳サーバ装置、翻訳システム、翻訳方法、翻訳プログラム
CN113852757B (zh) 视频处理方法、装置、设备和存储介质
CN109309874A (zh) 一种焦点更新方法及装置
CN107277639A (zh) 一种移动终端视频和信息叠加显示的方法

Legal Events

Date Code Title Description
ENP Entry into the national phase

Ref document number: 2019802025

Country of ref document: EP

Effective date: 20191122

121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19802025

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE