CN116719524A

CN116719524A - Page development processing method and device, electronic equipment and storage medium

Info

Publication number: CN116719524A
Application number: CN202310629118.XA
Authority: CN
Inventors: 谢姗删
Original assignee: Bank of China Ltd
Current assignee: Bank of China Ltd
Priority date: 2023-05-30
Filing date: 2023-05-30
Publication date: 2023-09-08

Abstract

The application provides a page development processing method, a page development processing device, electronic equipment and a storage medium. The method comprises the following steps: obtaining a development request, wherein the development request comprises the following steps: web UI pictures to be identified; according to the development request, analyzing and acquiring the category and the coordinate of each UI component in the Web UI picture to be identified by adopting a pre-trained identification model; the category and the coordinates of each UI component are output to realize the development processing of the Web page. The method of the application can improve the development efficiency of Web pages.

Description

Page development processing method and device, electronic equipment and storage medium

Technical Field

The present application relates to a target detection and recognition technology, and in particular, to a page development processing method, a device, an electronic apparatus, and a storage medium.

Background

Web page development generally includes several stages, such as demand analysis, prototype graph (UI) design, front-end development, testing and repairing, and online publishing, where the prototype graph design refers to drawing a page layout, a functional module, and the like according to a result of demand analysis, and designing colors, styles, layouts, icons, pictures, and the like of the page. Front-end development refers to that front-end personnel complete front-end development work of a website and construct a web page by utilizing technologies such as HTML, CSS, javaScript and the like according to the category, the coordinates and the style of each UI component in a prototype graph.

In a conventional manner, a front-end developer typically views a prototype graph, analyzes UI components in the prototype graph, and determines the category and coordinates of each UI component. In particular, when a front-end developer determines the positions of a plurality of UI components in a prototype graph, the front-end developer is required to hover a mouse around each UI component to determine the accurate position of each UI component.

However, in the case of more prototype graphs, the efficiency of manually analyzing UI components in the prototype graphs is low, which results in a technical problem of low development efficiency of web pages.

Disclosure of Invention

The application provides a page development processing method, a device, electronic equipment and a storage medium, which are used for solving the technical problem that in the prior art, UI components in a prototype graph are manually analyzed to influence the development efficiency of a front-end page.

In a first aspect, the present application provides a page development processing method, including:

obtaining a development request, wherein the development request comprises the following steps: the method comprises the steps of identifying a Web UI picture to be identified, wherein the Web UI picture to be identified is a prototype picture;

according to the development request, analyzing and acquiring the category and the coordinate of each UI component in the Web UI picture to be identified by adopting a pre-trained identification model;

the category and the coordinates of each UI component are output to realize the development processing of the Web page.

In a second aspect, the present application provides a page development processing apparatus, including:

the prototype graph obtaining module is used for obtaining a development request, wherein the development request comprises the following components: the method comprises the steps of identifying a Web UI picture to be identified, wherein the Web UI picture to be identified is a prototype picture;

the UI component identification module is used for analyzing and acquiring the category and the coordinate of each UI component in the Web UI picture to be identified by adopting a pre-trained identification model according to the development request;

and the identification output module is used for outputting the category and the coordinates of each UI component so as to realize development processing of the front-end page.

In a third aspect, the present application provides an electronic device comprising: a processor, and a memory communicatively coupled to the processor;

the memory stores computer-executable instructions;

the processor executes computer-executable instructions stored in the memory to implement the method as described in the first aspect.

In a fourth aspect, the present application provides a computer-readable storage medium having stored therein computer-executable instructions for performing the method according to the first aspect when executed by a processor.

In a fifth aspect, the application provides a computer program product comprising a computer program which, when executed by a processor, implements the method according to the first aspect.

The application provides a page development processing method, a device, electronic equipment and a storage medium, and a development request is acquired, wherein the development request comprises the following components: the method comprises the steps of identifying a Web UI picture to be identified, wherein the Web UI picture to be identified is a prototype picture; according to the development request, analyzing and acquiring the category and the coordinate of each UI component in the Web UI picture to be identified by adopting a pre-trained identification model; the category and the coordinates of each UI component are output to realize the development processing of the Web page. Aiming at the Web UI picture to be identified, the Web UI picture to be identified is input into a pre-trained identification model, the category and the coordinate of each UI component of the Web UI picture can be identified through the pre-trained identification model, and the whole process does not need to be manually analyzed, so that under the condition that the prototype picture is more, the efficiency of analyzing the UI components in the prototype picture can be improved, and the efficiency of developing the Web page can be improved to a certain extent.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the application and together with the description, serve to explain the principles of the application.

FIG. 1 is an application scenario diagram of a page development processing method implementing an embodiment of the present application;

FIG. 2 is a flow chart illustrating a method for implementing page development processing according to an embodiment of the present application;

FIG. 3 is a flowchart illustrating a method for implementing a page development processing according to another embodiment of the present application;

FIG. 4 is a schematic diagram of a structure for implementing a page development processing method according to the present application;

fig. 5 is a schematic structural diagram of an electronic device for implementing the page development processing method.

Specific embodiments of the present application have been shown by way of the above drawings and will be described in more detail below. The drawings and the written description are not intended to limit the scope of the inventive concepts in any way, but rather to illustrate the inventive concepts to those skilled in the art by reference to the specific embodiments.

Detailed Description

Reference will now be made in detail to exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, the same numbers in different drawings refer to the same or similar elements, unless otherwise indicated. The implementations described in the following exemplary examples do not represent all implementations consistent with the application. Rather, they are merely examples of apparatus and methods consistent with aspects of the application as detailed in the accompanying claims.

The page development processing method, the device, the electronic equipment and the storage medium of the application can be used in the financial field and can also be used in any field except the financial field, and the application fields of the page development processing method, the device, the electronic equipment and the storage medium of the application are not limited.

For a clear understanding of the technical solutions of the present application, the prior art solutions will be described in detail first.

Web development generally includes several stages, such as demand analysis, prototype graph (UI) design, front-end development, testing and repairing, and online publishing, where the prototype graph design refers to drawing a page layout, a functional module, and the like according to a result of demand analysis, and designing colors, styles, layouts, icons, pictures, and the like of the page. Front-end development refers to that front-end personnel complete front-end development work of websites and construct front-end pages by utilizing technologies such as HTML, CSS, javaScript and the like according to the category, the coordinates and the style of each UI component in the prototype graph.

In a conventional manner, a front-end developer typically views a prototype graph, analyzes UI components in the prototype graph, and determines the category and coordinates of each UI component. In particular, when a front-end developer determines the positions of a plurality of UI components in a prototype graph, the front-end developer is required to hover a mouse around each UI component to determine the accurate position of each UI component. However, in the case of more prototypes, the efficiency of manually analyzing UI components in the prototypes is low, which in turn results in a lower web page development efficiency.

Therefore, in the face of the technical problems in the prior art, the inventor discovers that the web page development time is shortened in order to improve the web page development efficiency after creative research. Therefore, the Web UI components are identified and classified and the coordinates are determined by adopting the pre-trained identification model, so that help can be provided for front-end developers to analyze and understand the prototype graph more quickly, and further development efficiency of the front-end developers on the Web page is improved.

As shown in fig. 1, an application scenario of the page development processing method provided by the embodiment of the present application includes an electronic device 10 and a terminal 20 in a corresponding network architecture in the application scenario, where the electronic device 10 and the terminal 20 communicate with each other. When the front-end developer needs to acquire the category and the coordinates of each UI component in the Web UI picture, the front-end developer initiates a development request to the electronic device 10 through the terminal 20, where the development request includes the Web UI picture to be identified. In the actual development process, the Web UI pictures included in the development request are multiple, and after the electronic device 10 receives the development request, the stored and pre-trained recognition model is called, and the category and the coordinates of each UI component in the Web UI pictures to be recognized are analyzed and obtained. Thereafter, the electronic device 10 may output the category and coordinates of each UI component in the Web UI picture in the form of a log or document for the front-end personnel to view and write the code development Web page accordingly.

The following describes the technical scheme of the present application and how the technical scheme of the present application solves the above technical problems in detail with specific embodiments. The following embodiments may be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments. Embodiments of the present application will be described below with reference to the accompanying drawings.

Fig. 2 is a diagram illustrating a page development processing method according to an embodiment of the present application, and as shown in fig. 2, an execution subject of the page development processing method according to the embodiment is an electronic device. The page development processing method provided by the embodiment comprises the following steps:

step 101, acquiring a development request, wherein the development request comprises the following steps: and (5) identifying the Web UI picture to be identified.

The development request is a request initiated to the electronic equipment through the terminal when a front-end developer needs to identify a large number of Web UI pictures before entering a Web page development stage.

The development request comprises a Web UI picture to be identified, and the Web UI picture to be identified is a prototype graph. In actual Web page development, there are a plurality of Web UI pictures to be identified.

And 102, analyzing and acquiring the category and the coordinates of each UI component in the Web UI picture to be identified by adopting a pre-trained identification model according to the development request.

Specifically, the electronic device responds to the development request, and adopts a pre-trained recognition model to analyze and recognize the Web UI pictures to be recognized in the development request, so that the category and the coordinates of each UI component in the Web UI pictures to be recognized can be obtained. The categories of UI components include nine categories of Button, divider, flag, header, icon, image, input, label, list.

And step 103, outputting the category and the coordinates of each UI component to realize development processing on the Web page.

The electronic device determines the sum of the category and the coordinates of each UI component in the Web UI picture to be identified, and can output the category and the coordinates of each UI component in the Web UI picture in a log or document mode so as to be used for a front-end personnel to view and write codes to develop a Web page according to the category and the coordinates.

In the application, a development request is obtained, wherein the development request comprises the following steps: the method comprises the steps of identifying a Web UI picture to be identified, wherein the Web UI picture to be identified is a prototype picture; according to the development request, analyzing and acquiring the category and the coordinate of each UI component in the Web UI picture to be identified by adopting a pre-trained identification model; the category and the coordinates of each UI component are output to realize the development processing of the Web page. Aiming at the Web UI picture to be identified, the Web UI picture to be identified is input into a pre-trained identification model, the category and the coordinate of each UI component of the Web UI picture can be identified through the pre-trained identification model, and the whole process does not need to be manually analyzed, so that under the condition that the prototype picture is more, the efficiency of analyzing the UI components in the prototype picture can be improved, and the efficiency of developing the Web page can be improved to a certain extent.

As an alternative embodiment, the pre-trained recognition model (i.e., fast R-CNN) includes a trained feature extraction network, a trained region generation network, a trained region of interest pooling layer, and a trained classification network; as shown in fig. 3, in this embodiment, step 102 includes the following steps:

and step 201, extracting features of the Web UI picture to be identified through the trained feature extraction network to obtain a feature map.

Specifically, for a Web UI picture input into a pre-trained recognition model, feature extraction is first performed. In the process of extracting the characteristics of the Web UI picture, the characteristic extraction is carried out through a plurality of convolution layers, a ReLU layer (which can be called as a nonlinear activation layer) and a pooling layer, and finally the characteristic diagram of the Web UI picture to be identified is obtained.

And 202, generating a plurality of recommended candidate areas of the feature map through the trained area generating network.

The trained regional generation network refers to a trained RPN network.

Multiple candidate regions of the feature map may be generated by the trained region generation network. The recommended candidate region is further screened from a plurality of candidate regions, and the recommended candidate region is a candidate region with a high possibility of containing the UI component in the candidate region.

And 203, pooling each recommended candidate region through the trained region-of-interest pooling layer to obtain a plurality of region feature maps with the same size.

Wherein the trained region of interest Pooling layer is a trained ROI Pooling.

Since the sizes of the recommended candidate regions may be different, and the classification network has a fixed input dimension, the sizes of the recommended candidate regions are unified before classification. And carrying out pooling operation on a plurality of recommended candidate areas of the feature map through the trained interesting area pooling layer, so that a plurality of area feature maps with the same size can be obtained.

And 204, inputting the regional feature diagrams into the trained classification network for classification and identification, and determining the category and the coordinates of each UI component of the Web UI picture to be identified.

Wherein the trained classification network is a trained CNN convolutional neural network.

And inputting the characteristic images of the areas into a trained classification network for classification and identification, and determining the category and the coordinates of each UI component of the Web UI picture to be identified.

In this embodiment, feature extraction is performed on the Web UI image to be identified through the trained feature extraction network, so as to obtain a feature map; generating a plurality of recommended candidate areas of the feature map through the trained area generating network; pooling each recommended candidate region through the trained region-of-interest pooling layer to obtain a plurality of region feature maps with the same size; and inputting each regional characteristic diagram into the trained classification network for classification and identification, and determining the category and the coordinates of each UI component of the Web UI picture to be identified. The trained networks are used for respectively carrying out corresponding processing to obtain the category and the coordinate of each UI component of the Web UI picture to be identified, so that the accuracy of the category and the coordinate of each UI component of the Web UI picture to be identified can be improved.

As an optional implementation manner, in this embodiment, the trained classification network includes a full convolution network, a first classification full connection layer, and a first regression full connection layer; step 204, comprising the steps of:

step 301, inputting each regional characteristic map into the full convolution network, and mapping each regional characteristic map into a first characteristic vector through the full convolution network.

Wherein the full convolution network is the full convolution network of the trained classification network. The full convolution network may map each region feature map to a first feature vector, respectively.

Step 302, inputting each first feature vector into a first classification full-connection layer and a first regression full-connection layer respectively, determining the category of the UI component in each regional feature map through the first classification full-connection layer, and determining the coordinate of the UI component in each regional feature map through the second regression full-connection layer.

The first feature vector is input to the first classification full-connection layer, and the classification of the UI component in each region feature map can be determined. The first feature vector is input to a first regression full connection layer, and the coordinates of the UI components in each region feature map can be determined.

Step 303, determining the category and the coordinates of each UI component in the Web UI picture to be identified based on the category and the coordinates of the UI component in each region feature map.

Specifically, for each UI component in the Web UI picture to be identified, the category and the coordinates may be obtained from the region feature map to which each UI component matches, and so on, the category and the coordinates of all UI components in the Web UI picture to be identified may be obtained.

In this embodiment, each of the regional feature maps is input into the full convolution network, and each of the regional feature maps is mapped into a first feature vector through the full convolution network; inputting the first feature vectors into a first classification full-connection layer and a first regression full-connection layer respectively, determining the category of the UI component in each regional feature map through the first classification full-connection layer, and determining the coordinate of the UI component in each regional feature map through the second regression full-connection layer; and determining the category and the coordinates of each UI component in the Web UI picture to be identified based on the category and the coordinates of the UI component in each region feature diagram.

As an alternative implementation manner, in this embodiment, step 202 includes the following steps:

step 401, sliding a window with a size of n×n on the feature map by a preset step length through the trained region generating network, acquiring features of the window corresponding to a position when the window slides to the position, and determining a plurality of candidate regions corresponding to the position; wherein n is a positive integer.

That is, the RPN slides a window of n×n on the feature map, where n may be 3 or 5. Each time a position is slid to, the RPN generates k=9 anchor points in the window corresponding to that position. The 9 anchor points differ in scale and size, respectively, in three dimensions (128 x 128, 256 x 256, 512 x 512) and three aspect ratios (1:1, 1:2, 2:1). The purpose of this design is to be able to cover UI components of different sizes and aspect ratios.

For a plurality of positions in the feature map, the following steps are performed:

and step 402, mapping the features of the window corresponding to the position into a second feature vector through the convolution layer of the trained region generation network.

Specifically, the features of the window corresponding to the position are mapped into a low-dimensional feature space through a convolution layer of the trained region generation network, and a second feature vector is obtained.

Step 403, inputting the second feature vector into a second classification full-connection layer and a second regression full-connection layer, determining the coordinates of each candidate region of the position through the second regression full-connection layer, and determining the background probability and the foreground probability of each candidate region of the position through the second classification full-connection layer.

Wherein the second regression full-connection layer is a reg full-connection layer. The second classification full-connection layer is cls full-connection layer. The foreground probability refers to the probability that a candidate region may contain a target (UI component), and the background probability refers to the probability that a candidate region may be background, which does not contain a target.

Specifically, the second feature vector is input to the second classification full connection layer, which may output the background probability and the foreground probability of each candidate region of the location. The second feature vector is input to a second regression full connection layer, which may output coordinates of each candidate region for the location.

And step 404, taking each candidate region with the foreground probability larger than the background probability as a recommended candidate region.

And taking the candidate region with the foreground probability larger than the background probability at the position as a recommended candidate region. Steps 402-404 are performed for all locations in the feature map, and therefore recommended candidate regions for each location in the feature map may be obtained.

In this embodiment, the trained area generating network slides a window with a size of n×n on the feature map with a preset step length, and when the window slides to a position, the feature of the window corresponding to the position is obtained, and a plurality of candidate areas corresponding to the position are determined; wherein n is a positive integer; for a plurality of positions in the feature map, the following steps are performed: mapping the features of the window corresponding to the position into a second feature vector through a convolution layer of the trained region generation network; inputting the second feature vector into a second classification full-connection layer and a second regression full-connection layer, determining the coordinates of each candidate region of the position through the second regression full-connection layer, and determining the background probability and the foreground probability of each candidate region of the position through the second classification full-connection layer; and taking the candidate areas with the foreground probability being greater than the background probability as recommended candidate areas. Since the foreground probability is the probability that the candidate region may contain the UI component, and the background probability is the probability that the candidate region may be background, the candidate region with the foreground probability greater than the background probability is used as the recommended candidate region, so that the calculation amount and the accuracy of subsequently determining the UI component category and the coordinate can be reduced.

As an alternative implementation manner, in this embodiment, step 203 includes the following steps:

step 501, dividing each recommended candidate region into a plurality of sub-regions with the same number as the preset output dimension for each recommended candidate region.

Wherein the size of each sub-region is the same. The preset output dimension is the same as the number of UI component categories, as previously described, the UI component includes 9 categories, so here the preset output dimension is 9 dimensions.

Step 502, performing a maximum pooling operation on each sub-region of each recommended candidate region, so as to obtain a plurality of region feature maps with the same size.

And performing maximum pooling operation (max pooling) on each sub-region of each recommended candidate region respectively, so as to obtain a plurality of region feature maps with the same size. Wherein each recommended candidate region corresponds to a region feature map.

In this embodiment, for each of the recommended candidate areas, each of the recommended candidate areas is divided into a plurality of sub-areas equal in number to a preset output dimension; wherein the size of each sub-region is the same; and respectively carrying out maximum pooling operation on each subarea of each recommended candidate area to obtain a plurality of area feature maps with the same size. And as each recommended candidate region is divided into a plurality of sub-regions and the maximum pooling operation is carried out, the region feature diagrams corresponding to each recommended candidate region are obtained, and the region feature diagrams have the same size, and are favorable for being input into a trained classification network for classification.

As an alternative implementation manner, in this embodiment, before step 101, the method further includes the following steps:

step 601, obtain an included element set from the Semantic UI library.

Wherein the SemanticUI is a CSS-based UI library providing a predefined set of CSS classes and javascript components for creating a modern web UI interface. The exclusive type data refers to a set of collections containing some elements that can be used in combination to create a more responsible UI component.

For example, each UI component may constitute an exclusive type set according to different values of its attributes (colors, sizes).

Step 602, mapping the included element set into a set of HTML attribute class values, and obtaining a web UI component data set.

Wherein the class attribute in HTML is an attribute for specifying one or more class names for an HTML element. Class names may be used to specify one or more CSS styles. For example, to add a specific style to a button, a class attribute may be used to specify the class to which the button belongs, and then the style of the class is defined in the CSS.

In particular, the names of the included type data are converted to corresponding class values so that the HTML uses these classes to identify and locate web UI components. For example, the included type data contains buttons, tables, and form elements, mapped to class values, such as ui button, ui table, and ui form.

And step 603, performing random combination on the Web UI component data sets to obtain a plurality of Web UI picture samples.

For example, taking a white picture with a fixed size of 750×1334 pixels as a background, randomly selecting 1-15 UI components and placing the UI components at any position on the background picture for synthesis to obtain a large Web UI picture.

And step 604, performing regression training on each Web UI picture sample in the initial recognition model to obtain a pre-trained recognition model.

That is, the training sample of the pre-trained recognition model is Web UI pictures, and regression training is performed on a large number of Web UI picture samples, so that the pre-trained recognition model can be obtained. The training method is the prior art and will not be described in detail.

In this embodiment, an included element set is acquired from a Semantic UI library; mapping the included element set into a set of HTML attribute class values to obtain a web UI component data set; any combination is carried out on the Web UI component data sets, and a plurality of Web UI picture samples are obtained; and carrying out regression training on each Web UI picture sample in the initial recognition model to obtain a pre-trained recognition model. By constructing and obtaining a Web UI picture sample and obtaining a pre-trained recognition model based on Web UI picture training, accuracy of recognition categories and coordinates of UI components contained in the Web UI picture to be recognized can be improved when the Web UI picture to be recognized is input later.

As an alternative implementation, in this embodiment, step 204 includes the following steps:

and 701, inputting each regional characteristic diagram into the trained classification network for classification and identification, and obtaining a plurality of probabilities of each regional characteristic diagram, which are the same as the number of the preset output dimensions.

Wherein the plurality of probabilities in each region feature graph characterizes probabilities that the UI components in the region feature graph are classes of UI components. That is, each region feature map includes a plurality of probabilities, the number of probabilities is the same as the preset output dimension, and there are 9 dimensions as the preset output dimension, where each region feature map has 9 probabilities, and the 9 probabilities represent probabilities of 9 UI component categories, respectively.

Step 702, using the UI component category with the largest probability value in each region feature map as the category of each UI component in the Web UI picture to be identified, and using the coordinates corresponding to each region feature map in the Web UI picture to be identified as the coordinates of each UI component in the Web UI picture to be identified.

For example, there are 9 probability values in the a region feature map, the probability of being a Button component is 0.2, the probability of being a provider component is 0.1, the probability of being a Flag component is 0.12, the probability of being a Header component is 0.31, the probability of being a Icon component is 0.11, the probability of being Image is 0.22, the probability of being an Input component is 0.34, the probability of being a Label component is 0.17, and the probability of being a List component is 0.3, so the UI component type with the largest probability value in the a region feature map is the Input component, and the Input component can be used as the UI component of the a region feature map at the corresponding position in the Web UI picture to be identified. The position of the A region feature map in the Web UI picture to be identified can be used as the coordinate of the Input component.

In this embodiment, each region feature map is input into the trained classification network to perform classification recognition, so as to obtain a plurality of probabilities of each region feature map, where the probabilities are the same as the number of the preset output dimensions; wherein the plurality of probabilities in each region feature graph characterizes the probabilities that the UI components in the region feature graph are the classes of the UI components; and taking the UI component category with the maximum probability value in each regional characteristic diagram as the category of each UI component in the Web UI picture to be identified, and taking the corresponding coordinate of each regional characteristic diagram in the Web UI picture to be identified as the coordinate of each UI component in the Web UI picture to be identified. Since the category and the coordinates of each UI component of the Web UI picture to be identified are determined based on the UI component category with the largest probability value of the region feature map, accuracy can be improved.

Fig. 4 is a schematic structural diagram of a page development processing apparatus according to an embodiment of the present application, and as shown in fig. 4, if the page development processing apparatus 40 according to the present embodiment is located in an electronic device, the page development processing apparatus 40 according to the present embodiment includes:

the prototype-drawing obtaining module 41 is configured to obtain a development request, where the development request includes: the method comprises the steps of identifying a Web UI picture to be identified, wherein the Web UI picture to be identified is a prototype picture;

the UI component identifying module 42 is configured to analyze and acquire, according to the development request, a category and coordinates of each UI component in the Web UI picture to be identified by using a pre-trained identifying model;

the recognition output module 43 is configured to output the category and the coordinates of each UI component, so as to implement development processing on the front-end page.

Optionally, the pre-trained recognition model includes a trained feature extraction network, a trained region generation network, a trained region of interest pooling layer, and a trained classification network; the UI component identifying module 42 is specifically configured to, when using the pre-trained identifying model to analyze and obtain a category and coordinates of each UI component in the Web UI picture to be identified:

extracting the characteristics of the Web UI picture to be identified through the trained characteristic extraction network to obtain a characteristic diagram;

generating a plurality of recommended candidate areas of the feature map through the trained area generating network;

pooling each recommended candidate region through the trained region-of-interest pooling layer to obtain a plurality of region feature maps with the same size;

and inputting each regional characteristic diagram into the trained classification network for classification and identification, and determining the category and the coordinates of each UI component of the Web UI picture to be identified.

Optionally, the trained classification network includes a full convolution network, a first classification full connection layer, and a first regression full connection layer; the UI component identifying module 42 is specifically configured to, when inputting each of the regional feature diagrams into the trained classification network to perform classification identification, determine a category and coordinates of each UI component of the Web UI picture to be identified:

inputting each regional characteristic map into the full convolution network, and mapping each regional characteristic map into a first characteristic vector through the full convolution network;

inputting the first feature vectors into a first classification full-connection layer and a first regression full-connection layer respectively, determining the category of the UI component in each regional feature map through the first classification full-connection layer, and determining the coordinate of the UI component in each regional feature map through the second regression full-connection layer;

and determining the category and the coordinates of each UI component of the Web UI picture to be identified based on the category and the coordinates of the UI component in each region feature diagram.

Optionally, the UI component identifying module 42 is specifically configured to, when generating, through the trained area generating network, a plurality of recommended candidate areas of the feature map:

sliding a window with the size of n multiplied by n on the feature map by a preset step length through the trained region generation network, acquiring the feature of the window corresponding to the position when the window slides to one position, and determining a plurality of candidate regions corresponding to the position; wherein n is a positive integer;

mapping the features of the window corresponding to the position into a second feature vector through a convolution layer of the trained region generation network;

inputting the second feature vector into a second classification full-connection layer and a second regression full-connection layer, determining the coordinates of each candidate region of the position through the second regression full-connection layer, and determining the background probability and the foreground probability of each candidate region of the position through the second classification full-connection layer;

and taking the candidate areas with the foreground probability being greater than the background probability as recommended candidate areas.

Optionally, the UI component identifying module 42 is specifically configured to, when pooling each recommended candidate region by the trained region of interest pooling layer to obtain a plurality of region feature maps with the same size:

dividing each recommended candidate region into a plurality of sub-regions which are the same in number as preset output dimensions for each recommended candidate region; wherein the size of each sub-region is the same;

and respectively carrying out maximum pooling operation on each subarea of each recommended candidate area to obtain a plurality of area feature maps with the same size.

Optionally, the page development processing device further comprises a model building module, configured to:

acquiring an included element set from a Semantic UI library;

mapping the included element set into a set of HTML attribute class values to obtain a web UI component data set;

any combination is carried out on the Web UI component data sets, and a plurality of Web UI picture samples are obtained;

and carrying out regression training on each Web UI picture sample in the initial recognition model to obtain a pre-trained recognition model.

Optionally, the UI component identifying module 42 is specifically configured to, when inputting each of the regional feature diagrams into the trained classification network to perform classification identification, determine a category and coordinates of each UI component of the Web UI picture to be identified:

inputting each regional feature map into the trained classification network for classification and identification, and obtaining a plurality of probabilities of each regional feature map, which are the same as the number of the preset output dimensions; wherein the plurality of probabilities in each region feature graph characterizes the probabilities that the UI components in the region feature graph are the classes of the UI components;

and taking the UI component category with the maximum probability value in each regional characteristic diagram as the category of each UI component in the Web UI picture to be identified, and taking the corresponding coordinate of each regional characteristic diagram in the Web UI picture to be identified as the coordinate of each UI component in the Web UI picture to be identified.

Fig. 5 is a block diagram of an electronic device, which may be the electronic device shown in fig. 5, according to an exemplary embodiment, including: a memory 51, a processor 52; memory 51 is a memory for storing processor-executable instructions; the processor 52 is operative to execute computer programs or instructions to implement the page development processing method as provided in any one of the embodiments above.

The memory 51 is used for storing programs. In particular, the program may include program code including computer-operating instructions. The memory 51 may comprise a high-speed RAM memory or may further comprise a non-volatile memory (non-volatile memory), such as at least one disk memory.

The processor 52 may be a central processing unit (Central Processing Unit, abbreviated as CPU), or an application specific integrated circuit (Application Specific Integrated Circuit, abbreviated as ASIC), or one or more integrated circuits configured to implement embodiments of the present disclosure.

Wherein the transceiver 54 transmits and receives data and instructions.

Alternatively, in a specific implementation, if the memory 51, the processor 52 and the transceiver 54 are implemented independently, the memory 51, the processor 52 and the transceiver 54 may be connected to each other and communicate with each other through the bus 53. The bus 53 may be an industry standard architecture (Industry Standard Architecture, abbreviated ISA) bus 53, an external device interconnect (Peripheral Component, abbreviated PCI) bus 53, or an extended industry standard architecture (Extended Industry Standard Architecture, abbreviated EISA) bus 53, among others. The bus 53 may be classified into an address bus 53, a data bus 53, a control bus 53, and the like. For ease of illustration, only one thick line is shown in fig. 5, but not only one bus 53 or one type of bus 53.

Alternatively, in a specific implementation, if the memory 51, the processor 52 and the transceiver 54 are implemented on a single chip, the memory 51, the processor 52 and the transceiver 54 may perform the same communication through an internal interface.

A non-transitory computer readable storage medium, which when executed by a processor of an electronic device, causes the electronic device to perform the page development processing method of the electronic device.

Other embodiments of the application will be apparent to those skilled in the art from consideration of the specification and practice of the application disclosed herein. This application is intended to cover any variations, uses, or adaptations of the application following, in general, the principles of the application and including such departures from the present disclosure as come within known or customary practice within the art to which the application pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the application being indicated by the following claims.

It is to be understood that the application is not limited to the precise arrangements and instrumentalities shown in the drawings, which have been described above, and that various modifications and changes may be effected without departing from the scope thereof. The scope of the application is limited only by the appended claims.

Claims

1. A method for processing page development, the method comprising:

2. The method of claim 1, wherein the pre-trained recognition model comprises a trained feature extraction network, a trained region generation network, a trained region of interest pooling layer, and a trained classification network; the analyzing and acquiring the category and the coordinate of each UI component in the Web UI picture to be identified by adopting the pre-trained identification model comprises the following steps:

3. The method of claim 2, wherein the trained classification network comprises a full convolution network, a first classification full connection layer, and a first regression full connection layer; inputting each region feature map into the trained classification network for classification and identification, and determining the category and the coordinates of each UI component of the Web UI picture to be identified, wherein the method comprises the following steps:

4. The method of claim 2, wherein the generating the plurality of recommended candidate regions of the feature map via the trained region generation network comprises:

5. The method according to claim 2, wherein the pooling each recommended candidate region by the trained region of interest pooling layer to obtain a plurality of region feature maps with the same size comprises:

6. The method of claim 1, wherein prior to the fetching of the development instruction, further comprising:

acquiring an included element set from a Semantic UI library;

7. The method according to claim 5, wherein the inputting the region feature map into the trained classification network for classification and identification, determining the category and coordinates of each UI component of the Web UI picture to be identified, comprises:

8. A page development processing apparatus, the apparatus comprising:

9. An electronic device, comprising: a processor, a transceiver, and a memory communicatively coupled to the processor;

the transceiver receives and transmits data and instructions;

the memory stores computer-executable instructions;

the processor executes computer-executable instructions stored in the memory to implement the method of any one of claims 1-7.

10. A computer readable storage medium having stored therein computer executable instructions which when executed by a processor are adapted to carry out the method of any one of claims 1-7.

11. A computer program product comprising a computer program which, when executed by a processor, implements the method of any of claims 1-7.