CN117454038A

CN117454038A - Page detection method and device

Info

Publication number: CN117454038A
Application number: CN202311153789.XA
Authority: CN
Inventors: 董桂明
Original assignee: Jingdong Technology Information Technology Co Ltd
Current assignee: Jingdong Technology Information Technology Co Ltd
Priority date: 2023-09-07
Filing date: 2023-09-07
Publication date: 2024-01-26

Abstract

The invention discloses a method and a device for page detection, and relates to the technical field of computers. One embodiment of the page detection method comprises the following steps: responding to the received page address, and acquiring a target page depth corresponding to the page address; determining a target page corresponding to the page address according to the target page depth, and generating a page image of the target page; and identifying the page image by using a preset algorithm model, and generating a detection result corresponding to the page address. According to the embodiment, the page image to be detected is determined according to the target page depth, the page detection range is flexibly determined, and the page detection efficiency is improved.

Description

Page detection method and device

Technical Field

The present invention relates to the field of computer technologies, and in particular, to a method and apparatus for page detection.

Background

Detecting the correctness of page rendering is a primary task of front-end testing. In general, when detecting whether a page is rendered correctly, a tester determines a target page to be detected according to a page address, then inputs page elements (e.g., buttons, input boxes, popups, etc.) included in the target page into a page verification tool one by one, and the page verification tool judges whether the input page elements are rendered correctly according to preset elements.

In carrying out the present invention, the inventors have found that at least the following problems exist in the prior art:

the page detection range is too dependent on manually-entered page elements, and page contents beyond manual entry cannot be detected, so that the page detection efficiency is low.

Disclosure of Invention

In view of this, the embodiments of the present invention provide a method and apparatus for detecting a page, which can flexibly determine a page detection range and improve page detection efficiency.

To achieve the above object, according to a first aspect of an embodiment of the present invention, there is provided a method for detecting a page, including:

responding to the received page address, and acquiring a target page depth corresponding to the page address;

determining a target page corresponding to the page address according to the target page depth, and generating a page image of the target page;

and identifying the page image by using a preset algorithm model, and generating a detection result corresponding to the page address.

Optionally, determining, according to the target page depth, a target page corresponding to the page address includes:

determining a first rendered page rendered by the page address, determining a jump address included in the first rendered page, and determining a second rendered page rendered by the jump address;

Judging whether the difference of the page depths of the first rendering page and the second rendering page is larger than the target page depth;

if yes, the first rendering page is used as a target page corresponding to the page address;

otherwise, repeating the step of updating the second rendering page until the difference between the page depths of the first rendering page and the updated second rendering page is consistent with the target page depth, and taking the first rendering page and each second rendering page as the target page corresponding to the page address; the step of updating the second rendered page comprises: and taking the rendered page rendered by the jump address included in the second rendered page as a new second rendered page.

Optionally, determining the jump address included in the first rendered page includes:

acquiring a document object model of the first rendering page;

matching the document object model with a document label included in the document object model by using a preset label set;

and obtaining the jump address included in the successfully matched document label from the document object model.

Optionally, the page image is identified by using a preset algorithm model, and a detection result corresponding to the page address is generated, which includes:

Identifying the page image by using a preset algorithm model to obtain an identification result;

and extracting a detection result corresponding to the page address from the identification result by using preset prompt information.

Optionally, before the page image is identified using the preset algorithm model, the method further includes:

storing the page image and generating a public network address of the page image;

and responding to the received identification request, determining a corresponding target public network address, acquiring a target page image corresponding to the target public network address, and inputting the target page image into the algorithm model.

Optionally, the method further comprises:

generating a detection report according to the page image corresponding to the page address and the detection result;

the detection report is updated in response to receiving the detail analysis request.

Optionally, the method further comprises:

training the algorithm model by using the updated detection report and the page image corresponding to the page address, and updating the component model library corresponding to the algorithm model.

According to a second aspect of an embodiment of the present invention, there is provided an apparatus for detecting a page, including:

The acquisition module is used for responding to the received page address and acquiring the target page depth corresponding to the page address;

the generation module is used for determining a target page corresponding to the page address according to the target page depth and generating a page image of the target page;

and the detection module is used for identifying the page image by using a preset algorithm model and generating a detection result corresponding to the page address.

acquiring a document object model of the first rendering page;

Optionally, the apparatus further comprises:

the storage module is used for storing the page image and generating a public network address of the page image;

and the input module is used for responding to the received identification request, determining a corresponding target public network address, acquiring a target page image corresponding to the target public network address, and inputting the target page image into the algorithm model.

Optionally, the apparatus further comprises:

The report module is used for generating a detection report according to the page image corresponding to the page address and the detection result;

and the updating module is used for updating the detection report in response to receiving the detail analysis request.

Optionally, the apparatus further comprises:

and the training module is used for training the algorithm model by using the updated detection report and the page image corresponding to the page address, and updating the component model library corresponding to the algorithm model.

According to a third aspect of an embodiment of the present invention, there is provided an electronic apparatus including:

one or more processors;

storage means for storing one or more programs,

the one or more processors implement the method of any of the embodiments described above when the one or more programs are executed by the one or more processors.

According to a fourth aspect of embodiments of the present invention, there is provided a computer readable medium having stored thereon a computer program which, when executed by a processor, implements a method according to any of the embodiments described above.

One embodiment of the above invention has the following advantages or benefits: the page image to be detected is determined according to the depth of the target page, so that the page detection range can be flexibly determined, and the page detection efficiency is improved; the algorithm model is used for identifying the page image, and a detection result corresponding to the page address is generated, so that the accuracy of page detection can be improved, and the cost of manpower and material resources can be reduced; the target page is determined according to the difference of the page depths of the first rendering page and the second rendering page, the target page to be detected can be flexibly covered, the page detection range is automatically determined, and the page detection efficiency is improved; the document label with the jump address can be rapidly determined by matching the preset label set with the document label included in the document object model, and the jump address can be accurately and efficiently obtained; the detection result corresponding to the page address is extracted from the identification result according to the prompt information, the identification result can be simplified, and the detection result meeting the requirements is obtained from the identification result; the public network address of the page image is generated, the page image is downloaded by using the public network address, so that the safety of the page address can be improved, the asynchronous detection of the page is realized, and the page image is efficiently acquired; updating the detection report according to the detail analysis request, so that a more accurate detection report can be generated; the updated inspection report and the corresponding page image are used for training the algorithm model, so that the accuracy of the algorithm model can be improved, a component model library is perfected, and the generalization capability of the algorithm model is improved.

Further effects of the above-described non-conventional alternatives are described below in connection with the embodiments.

Drawings

The drawings are included to provide a better understanding of the invention and are not to be construed as unduly limiting the invention. Wherein:

FIG. 1 is a schematic diagram of the main flow of a method of page detection according to an embodiment of the present invention;

FIG. 2 is a schematic diagram of a targeted page of a method of page detection according to one referenceable embodiment of the invention;

FIG. 3 is a schematic diagram of a training flow of a method of page detection according to one referenceable embodiment of the invention;

FIG. 4 is a schematic diagram of the overall flow of a method of page detection according to one referenceable embodiment of the invention;

FIG. 5 is a schematic diagram of the main flow of a method of page detection according to one referenceable embodiment of the invention;

FIG. 6 is a schematic diagram of the main flow of a method of page detection according to another referenceable embodiment of the invention;

FIG. 7 is a schematic diagram of the main flow of a method of page detection according to yet another referenceable embodiment of the invention;

FIG. 8 is a schematic diagram of the main modules of an apparatus for page detection according to an embodiment of the present invention;

FIG. 9 is an exemplary system architecture diagram in which embodiments of the present invention may be applied;

Fig. 10 is a schematic diagram of a computer system suitable for use in implementing an embodiment of the invention.

Detailed Description

Exemplary embodiments of the present invention will now be described with reference to the accompanying drawings, in which various details of the embodiments of the present invention are included to facilitate understanding, and are to be considered merely exemplary. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the invention. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.

In the technical scheme of the invention, the related processes of collecting, using, storing, sharing, transferring and the like of the personal information of the user accord with the regulations of related laws and regulations, the user needs to be informed and obtain the consent or the authorization of the user, and when the personal information of the user is applicable, the technical processes of de-identification and/or anonymization and/or encryption are performed on the personal information of the user.

The page detection range is too dependent on manually-entered page elements, and page contents beyond manual entry cannot be detected, so that the page detection efficiency is low, and a large amount of labor cost is consumed; and the preset elements are not updated timely, so that the preset elements are low in effectiveness and reusability.

In view of this, according to a first aspect of an embodiment of the present invention, there is provided a method of page detection.

Fig. 1 is a schematic diagram of a main flow of a method of page detection according to an embodiment of the present invention. As shown in fig. 1, the method for detecting a page according to an embodiment of the present invention mainly includes the following steps S101 to S103.

Step S101, responding to received page addresses, and acquiring target page depths corresponding to the page addresses.

The page address is a network address pointing to a page from which a corresponding page can be accessed, e.g., the page address is a URL (Uniform Resource Locator ). The target page depth is used to determine the extent of page detection, i.e., which pages should be detected. The execution body of the embodiment of the invention detects the page related to the received page address according to the target page depth, wherein the page related to the received page address comprises: a page rendered by a page address, a page rendered after the page address is redirected, a page pointed to by other page addresses contained on the rendered page, and so forth.

After receiving the page address, a corresponding target page depth is determined. For example, the target page depth is preset, and the target page depth corresponding to the page address is directly determined from a configuration file, a cache or a database. For another example, the received page detection request is parsed to obtain a page address and a corresponding target page depth.

Illustratively, the received page detect request is "host= 'a1.a2/a3' & deep=3", where "host" represents a method interface that processes the page detect request, "url= 'a1.a2/a3'" represents a page address of "a1.a2/a3", and "deep=3" represents a target page depth of 3. Further exemplary, after receiving the page address A4, the configuration file is traversed to find the target page depth corresponding to the page address A4, or the corresponding target page depth is obtained from the database using the page address A4 as a search condition.

The depth of the target page corresponding to the page address is determined, so that the range of page detection is conveniently and flexibly determined, the condition that only the page corresponding to the page address can be detected each time is avoided, and the diversified page detection requirements can be met.

Step S102, determining a target page corresponding to the page address according to the target page depth, and generating a page image of the target page.

The target page can be one page or a plurality of pages, and after the depth of the target page is determined, the target page corresponding to the page address is determined according to the depth of the target page. Illustratively, each page has an absolute page depth, and the absolute page depth of each page is fixed, e.g., jump from a preset root page (e.g., website top page or website landing page, etc.) twice to page B1, then the absolute page depth of page B1 is 2; for another example, the absolute page depth of page B2 is preset in a configuration file or database; determining a target page according to the absolute page depth and the target page depth, for example, the absolute page depth of a first rendered page B2 rendered by a page address is 1, the target page depth is 5, the page B2 is added into a target page set because the absolute page depth is smaller than or equal to the target page depth, the page B2 can jump to a page B3 and a page B4, wherein the absolute page depth of the page B3 is 3, the absolute page depth of the page B4 is 5, the page B3 and the page B4 are added into the target page set because the absolute page depth is smaller than or equal to the target page depth, and then continuing to determine jump pages corresponding to the page B3 and the page B4, and adding the jump pages into the target page set until the absolute page depth of the jump pages is larger than the target page depth or no jump pages exist; after the target page set is obtained, a page image of each target page included in the target page set is generated, for example, the target page is loaded and screenshot is performed by using a tool such as selenum.

Still further exemplary, each page has a relative page depth, the relative page depth of each page being non-fixed; after the reference page is determined, the relative page depth of each page relative to the reference page can be determined, specifically, the relative page depth of the reference page is 0, and the relative page depth of the page reached by the reference page by one jump is 1; thus, the relative page depth of each page changes as the reference page changes. And taking the first rendered page rendered by the page address as a reference page, determining a page with the relative page depth equal to 2 under the condition that the target page depth is equal to 2, namely jumping to the page with the reference page twice, taking the page with the relative page depth equal to 2 as the target page, and generating a page image of the target page after determining the target page.

It should be noted that, in the case that the target page corresponding to the page address cannot be determined, that is, the target page pointed by the page address cannot be accessed, for example, the page address cannot be identified, the time for identifying the page address is too long, the system file containing the target page is deleted, and the like, the page detection result is directly generated, which indicates that the detection result of the current page detection is abnormal.

The page detection range can be flexibly determined through the depth of the target page and the page address, so that various page detection requirements are met; the determined page detection range comprises one or more page depths, and pages in the one or more page depths are used as target pages, so that batch detection of the pages can be realized, and the page detection efficiency is improved.

According to the consultable embodiment of the invention, when the target page corresponding to the page address is determined according to the target page depth, the page address is rendered, and a first rendered page corresponding to the page address is obtained. The first rendered page is parsed, a jump address included in the first rendered page is determined, the jump address can be one or more, for example, source codes of the first rendered page are obtained, the source codes are traversed, and the jump address is searched from the source codes. Rendering the jump address to obtain a second rendering page corresponding to the jump address. Determining the page depth of the first rendering page and the page depth of the second rendering page, subtracting the page depths of the first rendering page and the second rendering page, and taking an absolute value to obtain the difference between the page depths of the first rendering page and the second rendering page. Judging whether the difference of the page depths is larger than the target page depth; if yes, the first rendering page is used as a target page corresponding to the page address; otherwise, repeating the step of updating the second rendered page until the difference between the page depths of the first rendered page and the updated second rendered page is consistent with the target page depth, wherein the step of updating the second rendered page comprises the following steps: taking a rendered page rendered by a jump address included in the second rendered page as a new second rendered page, wherein the difference of the page depths is consistent with the target page depth, and the method comprises the following steps: the difference in page depths is less than or equal to the target page depth. And under the condition that the difference of the page depths of the first rendering page and the updated second rendering page is consistent with the target page depth, taking the first rendering page and each second rendering page as the target page corresponding to the page address.

Fig. 2 is a schematic diagram of determining a target page according to a method of page detection according to a reference embodiment of the present invention. Illustratively, as shown in FIG. 2, each circle represents a page, the numbers within the circles represent the numbers of the page, e.g., circles labeled with the number "0" represent page 0; the arrows between circles represent the skip relationship between pages, e.g., page 0 has an arrow pointing to page 1, indicating that page 0 can skip to page 1, i.e., page 0 has a skip address of page 1 within it; setting the page depth of a first rendering page corresponding to the page address to be 0, and adding 1 to the page depth once per jump; after obtaining the page address of the page 1 and the target page depth of 1, determining the corresponding target page includes: page 1, page 4, and page 5; after obtaining the page address of the page 2 and the target page depth of 1, determining the corresponding target page includes: page 2, page 6, page 7, and page 8; after obtaining the page address of the page 3 and the target page depth of 0, determining the corresponding target page includes: page 3.

Illustratively, the target page depth is 3, the page depth of the first rendered page C1 is 1, and the page depth of the rendered page rendered by the jump address is specified to be equal to the page depth of the rendered page containing the jump address plus 1; therefore, the first rendering page C1 includes two jump addresses, the two jump addresses are respectively rendered to obtain a second rendering page C2 and a second rendering page C3, wherein the page depths of the second rendering page C2 and the second rendering page C3 are both 2 (i.e., 1+1), the second rendering page C2 includes one jump address, the second rendering page C3 does not include a jump address, the jump address included in the second rendering page C2 is rendered to obtain a third rendering page C4, the page depth of the third rendering page C4 is 3 (i.e., 2+1), and since the third rendering page C4 includes three jump addresses, the page depths of the fourth rendering page obtained by the three jump addresses are all equal to 4 and greater than the target page depth, the rendering is stopped, and the first rendering page C1, the second rendering page C2, the second rendering page C3 and the third rendering page C4 are regarded as target pages.

According to the depth of the target pages and the page addresses, the target pages to be detected can be flexibly determined, the number of the target pages is effectively limited, excessive target pages are prevented from being detected at one time, the page detection efficiency is improved, and if necessary, the number threshold of the target pages can be set, so that the target pages to be detected are further limited.

According to another embodiment of the present invention, when determining a jump address included in a first rendered page, a document object model (Document Object Model, DOM) of the first rendered page is obtained, a preset tag set is obtained, the tag set includes a plurality of target document tags capable of carrying jump links, and the target document tags include: < a > tags, < link > tags, and the like. Matching a target document tag included in the tag set with a document tag included in the document object model; and under the condition that the target document label is the same as the document label included in the document object model, judging that the matching is successful, and acquiring an attribute value of a label attribute 'heref' from the successfully matched document label, wherein the attribute value represents a jump address included in the first rendering page.

Illustratively, the method "getElementByName (" d1 ")" is used to obtain the document object model of the page element named "d1", and the method "getElementByClass (" d2 ")" is used to obtain the document object model of the page element named "d 2"; the obtained document object model comprises a label E1"< a heref=" https: d3.Com ">" and a label E2"< link heref=" https: d4.Com ">", wherein the label E1 is matched with a target document label < a >, and the label E2 is matched with the target document label < link >, so that a jump link is obtained from the label attribute "heref", and the jump link "https: d3.Com" and the jump link "https: d4.Com" are obtained.

The jump address included in the first rendered page is obtained by using the document object model and the tag set, the first rendered page can be converted into a standard page tag, the search range of the jump address is reduced according to the tag set, and the jump address included in the page tag is accurately and efficiently obtained.

And step S103, identifying the page image by using a preset algorithm model, and generating a detection result corresponding to the page address.

After obtaining the page image of the target page, inputting the page image into a preset algorithm model, carrying out image recognition on the page image by using the algorithm model, extracting image characteristics included in the page image, and judging whether the page image is normally displayed or not according to the image characteristics included in the page image. Algorithm models suitable for use in embodiments of the present invention include: a TensorFlow algorithm model, and an algorithm model obtained after optimizing the TensorFlow algorithm.

Illustratively, the target page is a merchandise browse page, and thus the page image of the target page should include: commodity pictures, page turning buttons, search boxes, search buttons, commodity attribute selection boxes, and the like; inputting a page image of a target page into a TensorFlow algorithm model, wherein the output result comprises a commodity picture of 0.9'; ' Page turning button ': 0.85'; ' search box ': 0.88'; ' commodity attribute selection box ': 0.82'; 'search button': 0.95'}, wherein,' commodity picture ': 0.9' ″ means that the probability of detecting that "commodity image" is included in the page image is 0.9, the rest of the results and the like; the preset checking threshold value is 0.75, namely, the page image is judged to actually comprise the corresponding detection object under the condition that the probability is larger than 0.75; since the probability of each detected object in the output results is greater than 0.75, the target page is judged to be normally displayed, and a detection result corresponding to the page address is generated, wherein the detection result indicates that the target page corresponding to the page address passes detection.

The algorithm model is used for automatically identifying the page image of the target page, so that page elements such as buttons and text boxes which are included in the page image can be prevented from being manually input, labor cost is reduced, and detection efficiency and accuracy of the target page are improved.

According to a referenceable embodiment of the present invention, a preset algorithm model is used to identify a page image, and a detection result corresponding to a page address is generated, including: identifying the page image by using a preset algorithm model, and determining various graphs included in the page image, wherein the various graphs comprise: right angle text boxes, rounded corner text boxes, rectangular buttons, circular buttons, image borders, text borders, hyperlink patterns, title patterns, and the like, and taking the determined graph as a recognition result. Obtaining preset prompt information, wherein the prompt information can be one or more keywords or sentences, for example, the prompt information is: judging whether the page image of the target page is normally rendered or not, or if the prompt information is: rectangular text boxes, rectangular buttons, etc.

According to the prompt information, extracting a detection result corresponding to the page address from the identification result, wherein the identification result comprises: the circular text box and the circular button, the prompt message is: the right-angle text box and the rectangular button are compared, the identification result is determined to be different from the prompt information, namely, the detection object included in the prompt information is not identified, so that the right-angle text box and the rectangular button are judged to be incorrectly rendered by the page image, the corresponding detection result is further generated, and the detection result indicates that the target page corresponding to the page address fails to be detected.

For another example, the recognition result includes: a circular text box (0.75), a title (0.8), a subtitle (0.4) and a button (0.85), wherein the circular text box (0.75) represents that the recognition rate of the circular text box is 0.75, namely the probability of the page containing the circular text box is 0.75; the prompt information is as follows: please judge whether the recognition rate of the current picture content display is greater than 0.7; accordingly, the recognition rate of each recognition result is compared with 0.7, and it is determined that the recognition rate of the "subtitle" is less than 0.7, and thus, hint information is generated, which indicates that the target page corresponding to the page address has not passed detection.

The recognition result of the target page is determined by using the algorithm model, the detection result is generated from the recognition result by using the prompt information, and the algorithm model and the prompt information are used together, so that different detection requirements can be met, and the page detection efficiency is improved.

According to another exemplary embodiment of the present invention, before the page image is identified using the preset algorithm model, the method further includes: the page image of the target page is stored, for example, using a mirror server, or using OSS (Object Storage Service, object store service). The public network address of the page image is generated, the public network address is a public address for storing the page image, and the corresponding public network address is generated according to the storage position of the page image, for example, the storage address (i.e. the public network address) of the page image on the mirror server or the OSS server is generated. After receiving the identification request, analyzing the identification request, determining a corresponding target public network address, acquiring a target page image corresponding to the target public network address from a mirror server or an OSS server according to the public network address, inputting the target page image into an algorithm model, and identifying the target page by using the algorithm model.

Illustratively, the OSS is used to store the page image, specifically, the public network address of the page image F1 is "oss.host/F1", the public network address of the page image F2 is "oss.host/F2", and the public network address of the page image F3 is "oss.host/F3"; analyzing the received identification request to obtain a target public network address of 'OSS/host/F/F2', determining a page image to be identified as a page image F2 according to the target public network address, downloading the page image F2 corresponding to the target public network address from the OSS, inputting the page image F2 into an algorithm model, and performing page detection on the page image F2 by using the algorithm model.

According to the identification request, the page image corresponding to the target public network address is identified, so that the page can be further flexibly detected, and the page can be detected in a targeted manner. The method is equivalent to backing up the page image, and is convenient for improving the accuracy of the algorithm model by using the page image. The public network address corresponding to the page image is generated, so that only the corresponding page image can be obtained during page detection, page address leakage is avoided, and the safety of the target page is improved.

According to a further referenceable embodiment of the invention, the method further comprises: generating a detection report according to the page image corresponding to the page address and the detection result, wherein the detection report comprises: page address, number of detected pages, page image, whether a page passes detection, problem with a page if the page does not pass detection, and so forth. After the detection report is generated, prompt information is generated to prompt a worker to audit the detection report, for example, the worker is prompted by means of short messages, telephones, mails, public numbers and the like, or the detection report is sent to the worker by means of mails, short messages, multimedia messages and the like. When a worker considers that the detection report has errors, a detail analysis request is sent to an execution main body of the embodiment of the invention, and after the execution main body of the embodiment of the invention receives the detail analysis request, the detail analysis request is analyzed, correction data of the detection report corrected by the worker is obtained, and the detection report is updated by using the correction data; and under the condition that the staff considers that the detection report is not wrong, sending an audit confirmation request to the execution body of the embodiment of the invention, and confirming that the detection report is accurate.

Illustratively, the detection report includes a page G1 and a page G2, wherein the detection result of the page G1 is a failed detection and the detection result of the page G2 is a passed detection; analyzing the received detail analysis request to obtain "{ 'page G1': correct '}", wherein "' page G1': correct' }" means that the page G1 is judged to be normal in display after manual examination, and the detection should be passed, so that the detection result of the page G1 in the detection report is modified to be passed.

After each page detection, a detection report is generated, so that the detection results of each page detection can be summarized and counted, and the detection results of each page detection can be more intuitively and clearly known. The manual auditing is introduced, the detection report is updated, the accuracy of the detection report can be ensured, and the labor cost is reduced.

According to a further embodiment of the invention, the method further comprises: and taking the page image corresponding to the updated detection report and the page address as training data, marking different labels on the page image passing through the page detection and the page image not passing through the page detection, training the algorithm model by using the training data, and optimizing the weight and bias of the algorithm model. The method comprises the steps of identifying page elements in a page image by using an algorithm model, such as buttons, text boxes, drop-down menus, popups, page styles and the like, comparing the identified page elements with page elements included in a component model library corresponding to the algorithm model, and adding the identified page elements which are not in the component model library into the component model library, so that the component model library corresponding to the algorithm model is updated.

FIG. 3 is a schematic diagram of a training flow of a method of page detection according to one referenceable embodiment of the invention. As shown in fig. 3, the training process mainly includes two stages of initial training data accuracy and model training and page detection, specifically, firstly, starting from "start 1", in the initial training data preparation stage, acquiring a UI design drawing or video and acquiring a manual test screenshot or video; in the model training stage, each frame of the acquired video is converted into an image by using OpenCV, common element components such as buttons, input frames, drop-down menus, popups and the like are identified from the design drawing, the test screenshot and the image obtained by video conversion, a preset algorithm model is trained, a component model library of the algorithm model is generated, a preliminary training process of the algorithm model is completed at this time, and the algorithm model obtained by training can be used for page detection. And then starting from the beginning 2, responding to the received page address, carrying out page detection on a target page corresponding to the page address, judging whether the page detection is normal, if so, marking the target page as a normal page, carrying out normal content training on the algorithm model by using the target page, updating a component model library of the algorithm model, otherwise, marking the target page as an abnormal page, carrying out abnormal content training on the algorithm model by using the target page, and updating the component model library of the algorithm model. In the page detection process, judging whether the page is normally displayed, providing training data for the algorithm model, continuously optimizing the algorithm model according to the detection result of the page detection, and updating a component model library of the algorithm model.

The algorithm model is optimized based on the page detection result, the modification and the update of the page can be automatically adapted, the rapid iteration of the algorithm model is facilitated, and the accuracy and the generalization capability of the algorithm model are improved.

Fig. 4 is a schematic diagram of the overall flow of a method of page detection according to one referenceable embodiment of the invention. Illustratively, as shown in FIG. 4, in response to receiving a page address, a preset target page depth is obtained; rendering a page address, resolving a jump address set from the rendered page according to the depth of the target page, judging whether the page address and the jump address can be accessed or not, and if the page address and the jump address can be accessed, acquiring the target page corresponding to the page address and the jump address, and generating a page image of the target page; storing the page address, the corresponding page image, the jump address and the corresponding page image into an image storage server, and generating a public network address of the page image; transmitting the public network address to the algorithm model, so that the algorithm model downloads the corresponding page image from the public network address; identifying the page image by using an algorithm model to obtain a detection result; if the page address and the jump address can not be accessed, the page address or the jump address is treated as an abnormal result; after the detection result is obtained, a detection report is generated according to the detection result, and manual auditing is prompted; and updating the detection report according to the manual auditing result.

Fig. 5 is a schematic diagram of the main flow of a method for page detection according to one referenceable embodiment of the present invention. As shown in fig. 5, the method for detecting a page may include:

step S501, responding to the received page address, and acquiring the page depth corresponding to the page address;

step S502, determining a first rendered page rendered by a page address, determining a jump address included in the first rendered page, and determining a second rendered page rendered by the jump address;

step S503, determining the difference between the page depths of the first rendering page and the second rendering page;

step S504, judging whether the difference of the page depths is larger than the target page depth, if so, jumping to step S505, otherwise, jumping to step S506;

step S505, the first rendering page is used as a target page corresponding to the page address;

step S506, repeatedly executing the step of updating the second rendering page until the difference between the page depths of the first rendering page and the updated second rendering page is consistent with the target page depth;

step S507, the first rendering page and each second rendering page are used as target pages corresponding to the page addresses;

step S508, screenshot is carried out on the target page, and a page image of the target page is generated;

Step S509, a preset algorithm model is used for identifying the page image, and a detection result corresponding to the page address is generated.

The specific implementation of the method for detecting a page according to the above embodiment of the present invention is described in detail in the above method for detecting a page, and thus the description thereof will not be repeated here.

Fig. 6 is a schematic diagram of the main flow of a method of page detection according to another referenceable embodiment of the invention. As shown in fig. 6, the method for detecting a page may include:

step S601, responding to received page addresses, and acquiring corresponding page depths;

step S602, determining a first rendered page rendered by a page address;

step S603, a document object model of the first rendering page is obtained, and a preset tag set is used for matching with document tags included in the document object model;

step S604, judging whether the label set and the document label are successfully matched, if so, jumping to step S605, otherwise, jumping to step S610;

step S605, obtaining a jump address included in the successfully matched document label from the document object model, and determining a second rendered page rendered by the jump address;

Step S606, determining the difference between the page depths of the first rendered page and the second rendered page;

step S607, judging whether the difference of the page depths is larger than the target page depth, if yes, jumping to step S610, otherwise jumping to step S608;

step S608, repeating the step of updating the second rendered page until the difference between the page depths of the first rendered page and the updated second rendered page is consistent with the target page depth;

step S609, taking the first rendering page and each second rendering page as a target page corresponding to the page address;

step S610, the first rendering page is used as a target page corresponding to the page address;

step S611, screenshot is carried out on the target page, and a page image of the target page is generated;

step S612, the page image is identified by using a preset algorithm model, and a detection result corresponding to the page address is generated.

The implementation of the page detection method according to the above embodiment of the present invention is described in detail in the above page detection method, and thus the description thereof will not be repeated here.

Fig. 7 is a schematic diagram of the main flow of a method for page detection according to still another embodiment of the present invention. As shown in fig. 7, the method for detecting a page may include:

Step S701, responding to received page addresses, and acquiring page depths corresponding to the page addresses;

step S702, determining a target page corresponding to a page address according to the page depth, and generating a page image of the target page;

step S703, storing the page image and generating a public network address of the page image;

step S704, in response to receiving the identification request, determining a corresponding target public network address, and obtaining a target page image corresponding to the target public network address;

step S705, identifying the target page image by using a preset algorithm model to obtain an identification result;

step S706, extracting a detection result corresponding to the page address from the identification result by using preset prompt information;

step S707, generating a detection report according to the target page image and the detection result;

step S708, in response to receiving the detail analysis request, updating the detection report;

step S709, training the algorithm model by using the updated detection report and the page image corresponding to the page address, and updating the component model library corresponding to the algorithm model.

The implementation of the method for detecting a page according to the above embodiment of the present invention is described in detail in the above method for detecting a page, and thus the description thereof will not be repeated here.

According to a second aspect of an embodiment of the present invention, there is provided an apparatus for page detection.

Fig. 8 is a schematic diagram of main modules of an apparatus for page detection according to an embodiment of the present invention, and as shown in fig. 8, an apparatus 800 for page detection mainly includes:

an obtaining module 801, configured to obtain, in response to receiving a page address, a target page depth corresponding to the page address;

a generating module 802, configured to determine a target page corresponding to the page address according to the target page depth, and generate a page image of the target page;

and the detection module 803 is configured to identify the page image by using a preset algorithm model, and generate a detection result corresponding to the page address.

According to a referenceable embodiment of the present invention, determining a target page corresponding to the page address according to the target page depth includes:

According to another referenceable embodiment of the present invention, determining a jump address included in the first rendered page includes:

acquiring a document object model of the first rendering page;

According to another embodiment of the present invention, the method for identifying the page image by using a preset algorithm model, and generating a detection result corresponding to the page address includes:

According to a further exemplary embodiment of the present invention, the apparatus 800 for detecting a page further includes:

According to yet another embodiment of the present invention, the apparatus 800 for detecting a page further includes:

According to a referenceable embodiment of the present invention, the apparatus 800 for page detection further includes:

It should be noted that, in the embodiment of the present invention, the specific implementation of the apparatus for detecting a page has been described in detail in the method for detecting a page, so that the description is not repeated here.

According to the technical scheme provided by the embodiment of the invention, the page image to be detected is determined according to the depth of the target page, so that the page detection range can be flexibly determined, and the page detection efficiency is improved; the algorithm model is used for identifying the page image, and a detection result corresponding to the page address is generated, so that the accuracy of page detection can be improved, and the cost of manpower and material resources can be reduced; the target page is determined according to the difference of the page depths of the first rendering page and the second rendering page, the target page to be detected can be flexibly covered, the page detection range is automatically determined, and the page detection efficiency is improved; the document label with the jump address can be rapidly determined by matching the preset label set with the document label included in the document object model, and the jump address can be accurately and efficiently obtained; the detection result corresponding to the page address is extracted from the identification result according to the prompt information, the identification result can be simplified, and the detection result meeting the requirements is obtained from the identification result; the public network address of the page image is generated, the page image is downloaded by using the public network address, so that the safety of the page address can be improved, the asynchronous detection of the page is realized, and the page image is efficiently acquired; updating the detection report according to the detail analysis request, so that a more accurate detection report can be generated; the updated inspection report and the corresponding page image are used for training the algorithm model, so that the accuracy of the algorithm model can be improved, a component model library is perfected, and the generalization capability of the algorithm model is improved.

According to a third aspect of an embodiment of the present invention, there is provided an electronic device for page detection, including: one or more processors; and the storage device is used for storing one or more programs, and when the one or more programs are executed by the one or more processors, the one or more processors are enabled to realize the method provided by the first aspect of the embodiment of the invention.

According to a fourth aspect of embodiments of the present invention, there is provided a computer readable medium having stored thereon a computer program which when executed by a processor implements the method provided by the first aspect of embodiments of the present invention.

Fig. 9 illustrates an exemplary system architecture 900 of a page detection method or apparatus to which embodiments of the present invention may be applied.

As shown in fig. 9, system architecture 900 may include terminal devices 901, 902, 903, a network 904, and a server 905. The network 904 is the medium used to provide communications links between the terminal devices 901, 902, 903 and the server 905. The network 904 may include various connection types, such as wired, wireless communication links, or fiber optic cables, among others.

A user may interact with the server 905 over the network 904 using the terminal devices 901, 902, 903 to receive or send messages, etc. Various communication client applications can be installed on the terminal devices 901, 902, 903, such as: page detection class applications, model training class applications, search class applications, instant messaging tools, mailbox clients, social platform software, and the like (just examples).

Terminal devices 901, 902, 903 may be a variety of electronic devices having a display screen and supporting web browsing, including but not limited to smartphones, tablets, laptop and desktop computers, and the like.

The server 905 may be a server that provides various services, such as a background management server (merely an example) that supports upstream requests for page detection by the terminal apparatuses 901, 902, 903. The background management server can respond to the received page address and acquire the target page depth corresponding to the page address; determining a target page corresponding to the page address according to the target page depth, and generating a page image of the target page; identifying the page image by using a preset algorithm model, and generating a detection result corresponding to the page address; and feeds back the page detection situation (only by way of example) to the terminal device.

It should be noted that, the method for detecting a page provided by the embodiment of the present invention is generally executed by the server 905, and accordingly, the device for detecting a page is generally disposed in the server 905. The method for page detection provided by the embodiment of the invention can also be performed by the terminal equipment 901, 902, 903, and correspondingly, the device for page detection can be arranged in the terminal equipment 901, 902, 903.

It should be understood that the number of terminal devices, networks and servers in fig. 9 is merely illustrative. There may be any number of terminal devices, networks, and servers, as desired for implementation.

Referring now to FIG. 10, there is illustrated a schematic diagram of a computer system 1000 suitable for use in implementing an embodiment of the present invention. The terminal device shown in fig. 10 is only an example, and should not impose any limitation on the functions and the scope of use of the embodiment of the present invention.

As shown in fig. 10, the computer system 1000 includes a Central Processing Unit (CPU) 1001, which can execute various appropriate actions and processes according to a program stored in a Read Only Memory (ROM) 1002 or a program loaded from a storage section 1008 into a Random Access Memory (RAM) 1003. In the RAM 1003, various programs and data required for the operation of the system 1000 are also stored. The CPU 1001, ROM 1002, and RAM 1003 are connected to each other by a bus 1004. An input/output (I/O) interface 1005 is also connected to bus 1004.

The following components are connected to the I/O interface 1005: an input section 1006 including a keyboard, a mouse, and the like; an output portion 1007 including a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), etc., and a speaker, etc.; a storage portion 1008 including a hard disk or the like; and a communication section 1009 including a network interface card such as a LAN card, a modem, or the like. The communication section 1009 performs communication processing via a network such as the internet. The drive 1010 is also connected to the I/O interface 1005 as needed. A removable medium 1011, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like, is installed as needed in the drive 1010, so that a computer program read out therefrom is installed as needed in the storage section 1008.

In particular, according to embodiments of the present disclosure, the processes described above with reference to flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method shown in the flow chart. In such an embodiment, the computer program may be downloaded and installed from a network via the communication portion 1009, and/or installed from the removable medium 1011. The above-described functions defined in the system of the embodiment of the present invention are performed when the computer program is executed by a Central Processing Unit (CPU) 1001.

It should be noted that, the computer readable medium shown in the embodiments of the present invention may be a computer readable signal medium or a computer readable storage medium, or any combination of the two. The computer readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples of the computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In embodiments of the present invention, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In contrast, in embodiments of the present invention, the computer-readable signal medium may comprise a data signal propagated in baseband or as part of a carrier wave, with the computer-readable program code embodied therein. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination of the foregoing. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, fiber optic cable, RF, etc., or any suitable combination of the foregoing.

The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer programs according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams or flowchart illustration, and combinations of blocks in the block diagrams or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

The modules involved in the embodiments of the present invention may be implemented in software or in hardware. The described modules may also be provided in a processor, for example, as: a processor includes an acquisition module, a generation module, and a detection module, where the names of the modules do not in some cases define the module itself, and for example, the acquisition module may also be described as "a module that acquires a target page depth corresponding to a page address".

As another aspect, the embodiment of the present invention also provides a computer-readable medium that may be contained in the apparatus described in the above embodiment; or may be present alone without being fitted into the device. The computer readable medium carries one or more programs which, when executed by a device, implement the method of: responding to the received page address, and acquiring a target page depth corresponding to the page address; determining a target page corresponding to the page address according to the target page depth, and generating a page image of the target page; and identifying the page image by using a preset algorithm model, and generating a detection result corresponding to the page address.

The above detailed description should not be construed as limiting the scope of the embodiments of the present invention. It will be apparent to those skilled in the art that various modifications, combinations, sub-combinations and alternatives can occur depending upon design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the embodiments of the present invention should be included in the scope of the embodiments of the present invention.

Claims

1. A method of page detection, comprising:

2. The method of claim 1, wherein determining the target page corresponding to the page address according to the target page depth comprises:

3. The method of claim 2, wherein determining the jump address included in the first rendered page comprises:

acquiring a document object model of the first rendering page;

4. The method of claim 1, wherein identifying the page image using a preset algorithm model, generating a detection result corresponding to the page address, comprises:

5. The method of claim 1, wherein prior to identifying the page image using a pre-set algorithmic model, the method further comprises:

6. The method according to claim 1, wherein the method further comprises:

7. The method of claim 6, wherein the method further comprises:

8. An apparatus for detecting a page, comprising:

9. An electronic device, comprising:

one or more processors;

storage means for storing one or more programs,

the one or more processors implement the method of any of claims 1-7 when the one or more programs are executed by the one or more processors.

10. A computer readable medium, on which a computer program is stored, characterized in that the program, when being executed by a processor, implements the method according to any of claims 1-7.