WO2020140932A1

WO2020140932A1 - Method and device for generating interface codes

Info

Publication number: WO2020140932A1
Application number: PCT/CN2020/070059
Authority: WO
Inventors: 涂挺鹏
Original assignee: 阿里巴巴集团控股有限公司
Priority date: 2019-01-06
Filing date: 2020-01-02
Publication date: 2020-07-09
Also published as: CN111414165A; CN111414165B

Abstract

Provided are a method and device for generating interface codes. Wherein, the method comprises the following steps: determining interface element information of an image file to be processed (101); determining layout information of an interface based on the interface element information (102); and generating interface codes according to the layout information (103). In the technical solution provided by the method, the automatic generation of the interface codes through the processing of the image file to be processed not only improves development efficiency and development cycle, but also reduces burden on programmers.

Description

Method and equipment for generating interface code

This application requires the priority of the Chinese patent application with the application number 201910010012.5 and the invention titled "Interface Code Generation Method and Equipment" submitted on January 06, 2019, the entire content of which is incorporated by reference in this application.

Technical field

This application relates to the field of electronic technology, in particular to a method and device for generating interface codes.

Background technique

At present, business under the Internet is developing rapidly, and various products are iterating rapidly. It is an important mission to improve the efficiency of user interface development and quickly connect to the rapid development of Internet services.

However, in the current user interface development process, designers need to rely on the design tools to design the design draft of the user interface first. The programmers obtain the interface code of the user interface by manually writing the code according to the design draft designed by the designer. The development efficiency of this development method is low and the development cycle is long.

Summary of the invention

In view of the above problems, the present application is proposed to provide a method and device for generating an interface code that solves the above problems or at least partially solves the above problems.

Therefore, in an embodiment of the present application, a method for generating interface codes is provided. The method includes:

Determine the interface element information of the image file to be processed;

Determine the layout information of the interface based on the interface element information;

According to the layout information, an interface code is generated.

In another embodiment of the present application, a method for generating interface code is provided. The method includes:

Display operation interface for users to input image data to be processed;

In response to an interface code generation instruction triggered by the user through the operation interface, an interface code is generated based on the image data to be processed.

In yet another embodiment of the present application, an electronic device is provided. The electronic device includes:

A first memory and a first processor, wherein,

The first memory is used to store programs;

The first processor, coupled to the first memory, is used to execute the program stored in the first memory for:

Determine the interface element information of the image file to be processed;

According to the layout information, an interface code is generated.

A second memory, a second processor and a second display, wherein,

The second memory is used to store programs;

The second display, coupled with the second processor, is used to display an operation interface for the user to input image data to be processed;

The second processor, coupled to the second memory, is used to execute the program stored in the second memory for:

In the technical solution provided by the embodiment of the present application, the interface element information is first derived from the image file to be processed, the layout information of the interface is determined according to the interface element information, and then the interface code is automatically generated according to the layout information. It can be seen that, in the technical solution provided by the embodiments of the present application, the interface code is automatically generated by processing the image file to be processed, which not only improves the development efficiency and development cycle, but also reduces the burden on the programmer.

BRIEF DESCRIPTION

In order to more clearly explain the embodiments of the present invention or the technical solutions in the prior art, the following will briefly introduce the drawings used in the description of the embodiments or the prior art. Obviously, the drawings in the following description These are some embodiments of the present invention. For those of ordinary skill in the art, without paying any creative work, other drawings can be obtained based on these drawings.

FIG. 1 is a schematic flowchart of a generation method provided by an embodiment of this application;

2 is a schematic flowchart of a generating method provided by another embodiment of this application;

3 is a structural block diagram of a generating device provided by an embodiment of the present application;

4 is a structural block diagram of a generation device provided by an embodiment of the present application;

5 is a structural block diagram of an electronic device provided by an embodiment of the present invention;

6 is a structural block diagram of an electronic device provided by another embodiment of the present invention;

7 is a schematic flowchart of a generating method provided by an embodiment of the present application;

8 is a schematic flowchart of a layout method provided by an embodiment of this application;

FIG. 9 is an operation interface diagram provided by an embodiment of the present application.

detailed description

In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be described clearly and completely in conjunction with the drawings in the embodiments of the present invention.

Some processes described in the specification, claims, and drawings of the present invention include multiple operations that occur in a specific order. These operations may not be performed in the order in which they appear in this document or in parallel. The sequence numbers of operations such as 101 and 102 are only used to distinguish different operations, and the sequence numbers themselves do not represent any execution order. In addition, these processes may include more or fewer operations, and these operations may be performed sequentially or in parallel. It should be noted that the descriptions of "first" and "second" in this article are used to distinguish different messages, devices, modules, etc., and do not represent the order, nor limit "first" and "second". Are different types.

In the prior art, after the designer designs the UI (User Interface, user interface) design draft, the UI design draft is delivered to the programmer. The programmer writes the code according to the UI design draft to get the interface code. After the programmer writes the interface code, the designer can see the real UI effect of the code for visual walkthrough. This development method has low development efficiency and a long development cycle.

In order to improve development efficiency and shorten the development cycle, embodiments of the present application provide a method for automatically generating interface codes based on image files to be processed.

The technical solutions in the embodiments of the present invention will be described clearly and completely in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, but not all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work fall within the protection scope of the present invention.

FIG. 1 shows a schematic flowchart of a method for generating an interface code provided by an embodiment of the present application. As shown in Figure 1, the method includes:

101. Determine interface element information of an image file to be processed.

102. Determine the layout information of the interface based on the interface element information.

103. Generate interface codes according to the layout information.

In the above 101, the image file to be processed may be a design draft or a design drawing. among them. The design draft refers to the original file created by the UI designer through the design software. The original file usually contains multiple layers; the design drawing refers to the image generated based on the design draft, for example: a screenshot. During specific implementation, the image file to be processed may be a design draft or design drawing corresponding to a UI module.

Designers commonly used design software such as: Sketch, Photoshop, etc. These design software usually open an application program interface (ie plug-in) to the user, and can export interface element information related to the design draft through the application program interface of the design software, and cache the exported interface element information in a cache space, for example: Clipboard. During the export process, the plug-in will traverse all layers of the image file to be processed, and perform type analysis and data extraction on all layers.

Among them, the interface element information may include: attribute information of each interface element. The attribute information may include at least one of the following items: name, type, coordinates, width and height, drawing order, fill content, and style of fill content. The types of interface elements may include but are not limited to: text type, picture type, shape type, input type, textArea type.

In the above 102, interface element information can be derived from the cache space, and layout identification can be performed to obtain layout information. The layout information includes: the parent-child node relationship between each interface element and the distribution information of each child node under each parent node.

The interface element corresponding to the parent node is located below the interface element corresponding to its child node, that is, the drawing order of the interface element corresponding to the parent node is prior to the drawing order of the interface element corresponding to its child node. The specific relationship between the parent node and the child node may include: inclusion relationship and cross relationship. Inclusion relationship means that all the interface elements corresponding to the child nodes overlap with the interface elements corresponding to the parent node. The cross relationship means that the interface elements corresponding to the child nodes only partially overlap the interface elements corresponding to the parent node.

In the above 103, code conversion is performed on the layout information of the interface to obtain the interface code. Interface codes corresponding to multiple code language types can be generated according to actual needs. Code language types include but are not limited to: HTML (HyperText Markup Language, Hyper Text Markup Language), Vue, Rax.

In a practical solution, the "determining the layout information of the interface based on the interface element information" in 102 above may be specifically implemented by the following steps:

1021: Based on attribute information of each interface element in the interface element information, determine a parent-child node relationship between each interface element and distribution information of at least one child node of each parent node.

1022. Determine the layout information according to the parent-child node relationship and distribution information of at least one child node of each parent node.

In the above 1021, the parent-child node relationship between each interface element may be determined based on the position information and drawing order information of each interface element. Among them, the position information includes coordinates and width and height.

Specifically, it can be determined whether there is an overlap between any two interface elements according to the position information of any two interface elements in each interface element; when there is an overlap between the two interface elements, according to the drawing order information of the two interface elements The interface element drawn first among the two interface elements is used as the parent node, and the interface element drawn later is used as the child node of the parent node; after traversing all interface elements, the parent-child node relationship between all interface elements can be obtained.

In order to improve the rationality of the determined parent-child node relationship, “determining the parent-child node relationship between each interface element based on the attribute information of each interface element in the interface element information” in 1021 above may be implemented by the following steps:

S11. Determine the initial parent-child node relationship between the interface elements based on the position information and drawing sequence information of the interface elements.

S12. In the initial parent-child node relationship, if a designated interface element exists as a parent node, a first container node is added to the initial parent node relationship to accommodate the designated interface element and the Specify the child node of the class interface element to obtain the parent-child node relationship.

The specific determination process of the initial parent-child node relationship in the above S11 may refer to the corresponding content above, and will not be repeated here.

In the above S12, the designated interface elements may be: picture interface elements and text interface elements. It is unreasonable to use picture interface elements and text interface elements as parent nodes. Therefore: in the initial parent-child node relationship, if a designated interface element exists as a parent node, a first container node is added to the initial parent node relationship to accommodate the designated interface element and the Specify the child node of the class interface element to obtain the parent-child node relationship.

Specifically: In the initial parent node relationship: the child nodes of node A have B and C, and the parent node of node A is node Q. Node A is a picture-type interface element, so a first container node D is added, and D is the parent node of A, B, and C, and the parent node of D is Q.

In the parent-child node relationship, except for the root node and the leaf node, each node acts as both a parent node and a child node. The root node can only be a parent node, and the leaf node can only be a child node. There are usually two types of parent nodes in the parent-child node relationship: the first type of parent node and the second type of parent node. The first type of parent node refers to: its child node is the parent node of the above-mentioned containing relationship; the second type of parent node refers to: its child node is the parent node of the above-mentioned cross-relationship.

The “determining the distribution information of at least one first child node of the first type parent node” in 1021 above specifically includes determining the row and column information of the at least one first child node according to the position information of the at least one first child node And information about the distance between the first child nodes in the at least one first child node.

Wherein, the distribution information of at least one first child node of the first type of parent node includes: row and column information of the at least one first child node and information about the surrounding distance of each first child node in the at least one first child node.

In a specific implementation, the row and column information of at least one first child node may be obtained by row and column scanning; and then the boundary scan is performed on the at least one first child node according to the row and column information to obtain information about the surrounding distance of each first child node. Generally, the four-week spacing information includes: left spacing, right spacing, top spacing, and bottom spacing.

The “determining the distribution information of at least one second child node of the second type parent node” in 1021 above specifically includes: according to the position information of the second type parent node and the at least one second child node, check The at least one second child node performs an absolute positioning layout to obtain distribution information of the at least one second child node.

The interface element corresponding to the parent node of the second type and the interface element corresponding to the second child node have a cross relationship, so the calculation is performed according to the position information of the parent node of the second type and the position information of at least one second child node to obtain at least one The absolute coordinate information of the second child node relative to the parent node of the second type.

In the above 1022, the relationship between the parent-child node and the distribution information of at least one child node of each parent node may be combined to obtain layout information of the interface.

In order to improve the accuracy of the parent-child node relationship between each interface element, it is necessary to modify the possible overflow and small cross in the interface element information. For example, when the overlapping degree of the boundary between the two interface elements is lower than the overlap threshold, the boundary repair can be performed on the two interface elements so that the boundary between the two interface elements does not overlap.

In order to be more close to the scenario of manually writing code in actual development, and to improve the actual usability and maintainability of the interface code, a scalable layout attribute (that is, a flex layout attribute) can be added to the layout information. Specifically, the above method may further include:

104. Determine, according to the surrounding information of the first child nodes in the at least one first child node, the stretchable layout attributes of each row and the stretchable layout attributes of each column.

105. Determine a correction scheme for the layout information according to the scalable layout attributes of each row and the scalable layout attributes of each column.

In the above 104, the telescopic layout attributes include: horizontal centering, vertical centering, and equal-spaced layouts (that is, there are multiple spacings in a row or column, and the difference between any two spacings in the multiple spacings is less than the first error value). The spacing rules corresponding to various types of scaling layout attributes can be set in advance according to various types of scaling layout attributes in practical applications; subsequently, the spacing rules corresponding to various types of scaling layout attributes can be determined according to the surrounding space information of the first child node on each row Matching, the telescopic layout attributes corresponding to the spacing rules on the match are used as the telescopic layout attributes of each row; according to the surrounding space information of the first child node on each column, to match the spacing rules corresponding to various types of telescopic layout attributes, The scaling layout attribute corresponding to the spacing rule on the match is used as the scaling layout attribute of each column. .

For example: the spacing rule corresponding to horizontal centering is: the difference between the left spacing of the row and the right spacing of the row is less than the second error value; the first row includes the first child nodes A and B arranged from left to right, the first child The left pitch Z1 of the node A is the left pitch of the first row, and the right pitch Z2 of the first child node B is the right pitch of the first row. When the difference between Z1 and Z2 is less than the second error value, the telescopic layout attribute of the first row is horizontally centered.

In the above 105, when the scalable layout attributes of each row are consistent, in the layout information, a scalable layout attribute of any row is added to the first-type parent node.

When the telescopic layout attributes of the columns are consistent, add the telescopic layout attributes of any column to the parent node of the first type in the layout information.

When the telescopic layout attributes of the rows are inconsistent and the telescopic layout attributes of the columns are also inconsistent, in the layout information, a second container node is added to each row to accommodate the first child node on each row, and for each row The newly added second container node adds a scalable layout attribute corresponding to each row, wherein the second container node of each row is a child node of the parent node of the first type. In addition, the second container node of each row is the parent node of the first child node on each row.

After adding a second container node to each row to accommodate the first child node on each row, and adding the scaling layout attribute corresponding to each row to the second container node newly added to each row, the redundant scaling layout under certain rules can be eliminated Attributes, for example, left-aligned are usually default telescopic layout attributes, so left-aligned telescopic layout attributes can be eliminated.

Further, the "generating interface code according to the layout information" in 103 above may be specifically implemented by the following steps:

1031. According to the layout information, corresponding code layout information is matched from the code base.

1032. According to the attribute information of each interface element in the layout information, match the attribute information of each node in the code layout information at the code level from the code library.

1033: Add attribute information of each node at the code level to the code layout information to obtain the interface code.

Code coding may be performed according to various possible layout information in advance to obtain code layout information. Code encoding may be performed according to attribute information of various possible interface elements in advance to obtain attribute information at the code level. Store code layout information corresponding to various possible layout information and code level attribute information corresponding to various possible interface element attribute information into the code library.

In this way, subsequent code layout information can be matched from the code base according to the layout information. According to the attribute information of each interface element in the layout information, the code level attribute information of each node in the code layout information is matched from the code library. Among them, the interface elements in the layout information correspond one-to-one with the nodes in the code layout information.

In a feasible solution, each of the interface elements includes: a first interface element. In the above 1032, "according to the attribute information of the first interface element, the attribute information at the code level of the first node corresponding to the first interface element in the code layout information is matched from the code library", specifically Is: according to the type attribute of the first interface element, the type attribute of the first node at the code level is matched in the code library; according to the display attribute of the first interface element, in the code library The display attributes of the first node at the code level are matched. Among them, the display attributes may include: the filling content and the style of the filling content.

In order to increase the readability of the generated interface code, the naming of each interface element in the layout information needs to be taken as the naming of the corresponding node in the code layout information. In order to more closely compile the code developed under the real scene, the "generating interface code based on the layout information" in 103 above also includes:

1034. Before acquiring corresponding code layout information according to the layout information, rename each interface element in the layout information according to a semantic naming rule.

The semantic naming is mainly to re-assign the naming of each interface element, and the naming is obtained by parsing the attribute information of the interface element. The semantic naming rules include: default rules, grouping rules, NLP (Neuro-Linguistic Programming), natural language analysis, image recognition, and business rules.

In addition, users can also manually modify the corresponding fields of the corresponding interface elements. Specifically, the field to be handwritten input by the user and the field to be modified manually bound by the user can be received; the field to be modified manually bound by the user is replaced with the field written by the user to complete the automatic mapping of the field.

In order to simplify the code, the layout information can be scanned to identify the structure loop body; and the structure loop body is marked in the layout information; in this way, according to the layout information, the code layout simplified by the loop syntax can be matched from the code base information.

Further, the "determining the interface element information of the image file to be processed" in the above 101 can be specifically implemented by the following steps:

1011. Export layer attribute information of each layer from the image file to be processed.

1012. Perform preset rule processing on the layer attribute information of each layer.

1013. Use layer attribute information of each layer processed by the prediction rule as attribute information of each interface element in the interface element information.

Wherein, the layer attribute information includes name, type, coordinates, width and height, drawing order, filling content and style of filling content.

In the above 1011, the layer attribute information of each layer can be exported by using the application program interface provided by the design software.

In the above 1012, the step of performing preset rule processing on the layer attribute information of each layer includes at least one of the following:

The first item: delete layer attribute information of useless layers by performing layer redundancy and sequence check on each layer.

Among them, the useless layer refers to a layer completely covered by other layers or a layer that does not cause a change in the user interface during the drawing of the user interface.

The second item: correct the layer type of the layer with the wrong layer type by checking the layer type of each layer.

For example: shape type layer is easy to be recognized as a picture type layer, so it needs to be corrected.

Item 3: By performing layer merge verification on the layers, merge multiple layers with merge association into one layer.

Specifically: According to the connection status and pixel differences of multiple layers, it is determined whether the multiple layers have a merged association. For example, if there are overlapping parts at the boundaries of multiple layers and the pixel difference of multiple layers is less than a preset threshold, it is determined that multiple images have merge relevance.

It should be added that the first, second, and third items above can also use OpenCV to implement layer verification.

Item 4: Modify the layer attribute information of each layer in combination with the design rules of the design tool of the image file to be processed.

The design rules of different design tools are different. Therefore, the layer attribute information of each layer needs to be modified according to the design rules of different design tools.

Item 5: Convert the layer attribute information of each layer into data in a preset format.

Specifically, the preset format may be a Json format. Json format is a lightweight data exchange format. It needs to be added that: before converting the layer attribute information of each layer into data in a preset format, upload the picture corresponding to the picture type interface element to the server, for example: CDN (Content Delivery Network, content distribution Network), and receive the picture storage address returned by the server, and use the picture storage address as part of the attribute information of the picture type interface element.

When the to-be-processed image file is a design drawing, “exporting layer attribute information of each layer from the to-be-processed image file” in 1011 above may be specifically implemented by using the following steps:

S31. Perform text and image recognition on the design drawing to separate multiple layers from the design drawing.

S32. Perform layer type identification on the multiple layers to determine the layer types of the multiple layers.

Through OCR (Opticalcharacterrecognition, optical character recognition) to identify and analyze the text to start, separate the text type layer. Further expand the diffuse water to find other types of contours, and fit the layer type corresponding to each contour and other attribute data of the layer.

In specific implementation, OCR recognition can be performed on the design drawing to obtain multiple text layers. Specifically, the overall OCR recognition is performed on the design drawing to obtain multiple text ROI (region of interest). From each text ROI, extract the words through the inner and outer contours. After the words are extracted, the words are processed: color aggregation and extraction of the word information such as text color, position, font size and so on. You can bind the text ROI and the word information extracted from the text ROI. After extracting all the words in the design drawing, look for the connected area of the word, and bind each connected area with the word information contained in each connected area. After obtaining multiple Unicom areas, you can merge similar Unicom areas based on the text color, position, and font size of the words in the Unicom area. Treat each connected area as a text-like layer.

Determine whether the font in the text-like layer is a standard font model. If so, determine that the text-like layer is a text-type layer; if not, use the text-like layer as a picture-like layer.

Aggregate multiple text layers into a text layer; remove the text layer in the design drawing, and obtain a stable picture layer after preprocessing such as noise removal, edge enhancement, and continuous area filtering; determine from the stable picture layer through image recognition Contour information of multiple class picture layers; cut the design drawing according to the contour information of multiple class picture layers to obtain multiple class picture layers.

If there is no text in the class picture layer, it is judged whether the class picture layer contains a floating layer; if it contains a floating layer, the floating layer is separated, and the class picture layer after the floating layer is separated is determined as a picture type layer.

If there is text in the image-like layer, identify the block containing the text to determine whether the block is a shape-like layer; when the block is a shape-like layer, further determine whether the block is an input or textarea layer; if If it is not an input or textarea layer, the block is determined to be a shape type layer. In addition, the block is removed from the picture-like layer, and image restoration is performed on the picture-like layer. Determine whether the repaired class picture layer contains a floating layer; if it contains a floating layer, separate the floating layer, and determine the class picture layer after separating the floating layer as the picture type layer.

In addition, after the interface code is obtained, the UI module vision can be directly rendered by executing the interface code to achieve the effect seen in the visual draft, and can ensure the degree of visual restoration, improve the efficiency of the initial module development of the development, and shorten the development and vision. Coordination cycle.

FIG. 2 shows a schematic flowchart of a method for generating an interface code provided by another embodiment of the present application. As shown in Figure 2, the method includes:

201. Display an operation interface for the user to input image data to be processed.

202. In response to an interface code generation instruction triggered by the user through the operation interface, generate an interface code based on the image data to be processed.

In the above 201, the user can import image data to be processed on the operation interface. The image data to be processed may be an image file to be processed or interface element information derived from the image file to be processed. The image file to be processed may be a design draft or a design drawing. The design draft refers to the original file created by the UI designer through the design software. The original file usually contains multiple layers; the design drawing refers to the image generated based on the design draft, for example: a screenshot. During specific implementation, the image file to be processed may be a design draft or design drawing corresponding to a UI module.

In the above 201, the first control is displayed on the operation interface, and in response to the user's touch operation on the first control, an interface code generation instruction is generated. Of course, a language input button can also be displayed on the operation interface, and the user long presses the language to input language information; according to the user's language information, an interface code generation instruction is generated.

In the technical solution provided by the embodiment of the present application, an operation interface is provided for the user, and the user imports the image file to be processed in the operation interface; in response to the user's interface code generation instruction triggered by the operation interface, the interface is generated based on the image file to be processed Code. It can be seen that, in the technical solution provided by the embodiments of the present application, the interface code is automatically generated by processing the image file to be processed, which not only improves the development efficiency and development cycle, but also reduces the burden on the programmer.

Further, the above method may further include:

203. Display at least one control corresponding to the code language type on the operation interface for the user to select.

Code language types include but are not limited to: HTML (HyperText Markup Language, Hyper Text Markup Language), Vue, Rax. Subsequently, the interface code of the code language type selected by the user can be generated.

Further, the above method may further include:

204. In response to the user's trigger operation on the control corresponding to the first code language type, generate the interface code generation instruction, where the interface code generation instruction carries the first code language type.

Correspondingly, “generating an interface code based on the to-be-processed image data” in 202 above specifically includes:

2021: Determine a code library corresponding to the first code language type.

2022. Combine the code base to generate an interface code based on the image data to be processed.

In the above 2021, corresponding code libraries can be established for various code language types in advance.

For the specific implementation of the above 2021, refer to the corresponding content in the foregoing embodiments, and details are not described herein again.

Further, the image data to be processed is interface element information derived from the image file to be processed. The above 202 "generating an interface code based on the image data to be processed" specifically includes the following steps:

2023. Based on the interface element information, determine layout information of the interface.

2024. Generate interface codes according to the layout information.

For details of the above 2023 and 2024, reference may be made to the corresponding content in the foregoing embodiments, and details are not described herein again.

It should be noted here that in addition to the above steps, the method provided in the embodiments of the present application may also include all or part of the steps in the above embodiments. For details, please refer to the corresponding content in the above embodiments, not here Repeat again.

The method for generating the interface code provided by the embodiment of the present application will be described in the following in conjunction with FIGS. 7, 8 and 9:

51. Paste interface element information.

The interface element information may be pasted at the data pasting box 701 on the operation interface shown in FIG. 9. The design software plug-in can cache the interface element information exported from the image file to be processed in the clipboard.

52. Layout identification to obtain layout information.

53. Semantic naming of interface elements.

The semantically named rules may include rules configured by default and rules manually bound by the user. The user can perform manual binding through the "shortcut binding" operation area 703 on the operation interface shown in FIG. 9.

54. Perform a similarity scan on the interface element information to identify and mark structural loops.

55. Multi-DSL (domain-specific language, domain-specific language) conversion and export.

Among them, DSL can be HTML, Vue, Rax, etc.

56. Interface code output.

Wherein, as shown in FIG. 8, the above step 52 may specifically include the following steps:

501: Perform a sequential scan of interface element information.

502. Border overlapping trimming.

That is, some data may be overflowed and slightly crossed.

503. Determine the initial parent-child node relationship based on the position intersection of the interface elements

504. A first container node is added to accommodate the picture/text type interface element as a parent node and its child nodes

505. The parent-child node relationship confirmation output.

506. Perform a boundary scan on each row and column of the child node under the parent node.

507. Confirm the Flex layout attributes of each row and column of the child node under the parent node.

508. Get the Flex layout attributes of all rows.

509. Determine whether the flex layout attributes of all rows or all columns can converge to the parent node.

510. Add the flex attribute to the parent node.

511. Add a second container node to each row and add a flex attribute.

512. Exclude the extra flex attribute under certain rules.

513. Perform absolute positioning and layout on the parent-child nodes whose parent-child relationship is a cross relationship.

514. Output layout information.

For specific implementation of the above steps, reference may be made to the corresponding content in the foregoing embodiments, and details are not described herein again.

FIG. 3 shows a structural block diagram of an apparatus for generating an interface code provided by another embodiment of the present application. As shown in FIG. 3, the apparatus includes: a first determination module 301, a second determination module 302, and a first generation module 303. among them,

The first determining module 301 is used to determine interface element information of the image file to be processed;

The second determining module 302 is configured to determine layout information of the interface based on the interface element information;

The first generating module 303 is configured to generate an interface code according to the layout information.

Further, the second determination module 302 is specifically used to:

Based on attribute information of each interface element in the interface element information, determining a parent-child node relationship between each interface element and distribution information of at least one child node of each parent node;

The layout information is determined according to the parent-child node relationship and distribution information of at least one child node of each parent node.

Further, the second determination module 302 is specifically used to:

Determine the initial parent-child node relationship between each interface element based on the position information and drawing sequence information of each interface element;

In the initial parent-child node relationship, if there is a designated class interface element as a parent node, a first container node is added to the initial parent node relationship to accommodate the designated class interface element and the designated class The child node of the interface element to obtain the parent-child node relationship.

Further, each parent node includes a first-type parent node; a second determination module 302 is specifically used to:

According to the position information of the at least one first child node, the row and column information of the at least one first child node and the surrounding distance information of each first child node in the at least one first child node are determined.

Further, the above device further includes:

A third determining module, configured to determine the telescopic layout attributes of each row and the telescopic layout attributes of each column according to the information about the surrounding distance of each first sub-node in the at least one first sub-node;

The fourth determining module is used to determine a correction plan for the layout information according to the scalable layout attributes of each row and the scalable layout attributes of each column.

Further, the fourth determination module is specifically used for:

When the scalable layout attributes of the rows are consistent, add the scalable layout attributes of any row to the first type of parent node in the layout information;

When the telescopic layout attributes of the columns are consistent, add the telescopic layout attributes of any column to the first type of parent node in the layout information;

When the scaling layout attributes of each row and the scaling layout attributes of each column are inconsistent, in the layout information, a second container node is added to each row to accommodate the first child node on each row, and new The added second container node adds a telescopic layout attribute corresponding to each row, where the second container node of each row is a child node of the parent node of the first type.

Further, each of the parent nodes includes a second type of parent node; the second determination module 302 is specifically used to:

According to the position information of the second-type parent node and the at least one second child node, perform absolute positioning layout on the at least one second child node to obtain distribution information of the at least one second child node.

Further, the first generation module 303 is specifically used for:

According to the layout information, the corresponding code layout information is matched from the code library;

According to the attribute information of each interface element in the layout information, matching the attribute information of each node in the code layout information at the code level from the code library;

The attribute information of each node at the code level is added to the code layout information to obtain the interface code.

Further, each of the interface elements includes: a first interface element; and, a first generation module 303 is specifically used to:

According to the type attribute of the first interface element, matching the type attribute of the first node at the code level in the code base;

According to the display attributes of the first interface element, the display attributes of the first node at the code level are matched in the code library.

Further, the first generation module 303 is also used to:

Before acquiring the corresponding code layout information according to the layout information, each interface element in the layout information is renamed according to the semantic naming rules.

Further, the interface element information includes: attribute information of each interface element;

The attribute information includes at least one of the following items: name, type, coordinates, width and height, drawing order, filling content, and style of filling content.

Further, the first determining module 301 is specifically used to:

Export layer attribute information of each layer from the image file to be processed, where the layer attribute information includes name, type, coordinates, width and height, drawing order, filling content, and style of filling content;

Perform preset rule processing on the layer attribute information of each layer; use the layer attribute information of each layer after the prediction rule processing as the attribute information of each interface element in the interface element information.

Further, the first determining module 301 is specifically configured to execute at least one of the following:

By performing layer redundancy and sequence check on each layer, the layer attribute information of useless layers is deleted;

Correct the layer type of the layer with the wrong layer type by checking the layer type of each layer;

By performing layer merge verification on each layer, multiple layers with merge relevance are merged into one layer;

Combine the design rules of the design tool of the image file to be processed to modify the layer attribute information of each layer;

Convert the layer attribute information of each layer into data in a preset format.

Further, the image file to be processed is a design drawing; the first determining module 301 is specifically used to:

Performing text and image recognition on the design drawing to separate multiple layers from the design drawing;

Perform layer type identification on the multiple layers to determine the layer types of the multiple layers.

It should be noted here that the generating device provided in the above embodiments can implement the technical solutions described in the above method embodiments. For the specific implementation principles of the above modules or units, please refer to the corresponding contents in the above method embodiments. Repeat again.

FIG. 4 shows a structural block diagram of an apparatus for generating an interface code provided by another embodiment of the present application. The device includes:

The first display module 401 is used to display an operation interface for the user to input image data to be processed;

The second generation module 402 is configured to generate an interface code based on the to-be-processed image data in response to an interface code generation instruction triggered by the user through the operation interface.

In the technical solution provided by the embodiments of the present application, an operation interface is provided for the user, and the user imports the image file to be processed on the operation interface; in response to the user's interface code generation instruction triggered by the operation interface, the interface is generated based on the image file to be processed Code. It can be seen that, in the technical solution provided by the embodiments of the present application, the interface code is automatically generated by processing the image file to be processed, which not only improves the development efficiency and development cycle, but also reduces the burden on the programmer.

Further, the first display module 401 is also used to:

A control corresponding to at least one code language type is displayed on the operation interface for the user to select.

Further, the above device further includes:

A third generation module, configured to generate the interface code generation instruction in response to the user's trigger operation on the control corresponding to the first code language type, and the interface code generation instruction carries the first code language type;

And, the second generation module is specifically used for:

Determine a code base corresponding to the first code language type;

In combination with the code base, an interface code is generated based on the image data to be processed.

Further, the image data to be processed is interface element information derived from the image file to be processed, and the second generation module is specifically used to:

According to the layout information, an interface code is generated.

FIG. 5 is a schematic structural diagram of an electronic device provided by an embodiment of the present application. The electronic device includes: a first memory 1101 and a first processor 1102. The first memory 1101 may be configured to store various other data to support operations on the electronic device. Examples of these data include instructions for any applications or methods that operate on the electronic device. The first memory 1101 may be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read only memory (EEPROM), erasable In addition to programmable read only memory (EPROM), programmable read only memory (PROM), read only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

The first processor 1102 is coupled to the first memory 1101, and is used to execute the program stored in the first memory 1101 for:

Determine the interface element information of the image file to be processed;

According to the layout information, an interface code is generated.

Wherein, when the first processor 1102 executes the program in the first memory 1101, in addition to the above functions, other functions may also be implemented. For details, refer to the description of the foregoing embodiments.

Further, as shown in FIG. 5, the electronic device further includes: a first communication component 1103, a first display 1104, a first power component 1105, a first audio component 1106 and other components. FIG. 5 only schematically shows some components, and does not mean that the electronic device includes only the components shown in FIG. 5.

Correspondingly, the embodiments of the present application also provide a computer-readable storage medium storing a computer program, which when executed by a computer can implement the steps or functions of the object positioning method provided by the foregoing embodiments.

FIG. 6 shows a schematic structural diagram of an electronic device provided by an embodiment of the present application. As shown, the electronic device includes a second display 1204, a second memory 1201, and a second processor 1202. The second memory 1201 may be configured to store various other data to support operations on the electronic device. Examples of these data include instructions for any applications or methods operating on the electronic device. The second memory 1201 may be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read only memory (EEPROM), erasable In addition to programmable read only memory (EPROM), programmable read only memory (PROM), read only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

The second display 1204 is coupled to the second processor 1202, and is used to display an operation interface for the user to input image data to be processed;

The second processor 1202 is coupled to the second memory 1201, and is used to execute the program stored in the second memory 1201 for:

Wherein, when the second processor 1202 executes the program in the second memory 1201, in addition to the above functions, other functions may be implemented. For details, refer to the description of the foregoing embodiments.

Further, as shown in FIG. 6, the electronic device further includes: a second communication component 1203, a second power component 1205, a second audio component 1206, and other components. Only some components are schematically shown in FIG. 6, which does not mean that the electronic device includes only the components shown in FIG.

Correspondingly, the embodiments of the present application also provide a computer-readable storage medium storing a computer program, which when executed by a computer can realize the steps or functions of the object positioning method provided by the foregoing embodiments.

The device embodiments described above are only schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, may be located One place, or can be distributed to multiple network elements. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art can understand and implement without paying creative labor.

Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions can be embodied in the form of software products in essence or part of the contribution to the existing technology, and the computer software products can be stored in computer-readable storage media, such as ROM/RAM, magnetic Discs, optical discs, etc., include several instructions to enable a computer device (which may be a personal computer, server, or network device, etc.) to perform the methods described in the various embodiments or some parts of the embodiments.

Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still The technical solutions described in the foregoing embodiments are modified, or some of the technical features are equivalently replaced; and these modifications or replacements do not deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

A method for generating interface code, characterized in that it includes:

Determine the interface element information of the image file to be processed;

Determine the layout information of the interface based on the interface element information;

According to the layout information, an interface code is generated.
The method according to claim 1, wherein determining the layout information of the interface based on the interface element information includes:

Based on attribute information of each interface element in the interface element information, determining a parent-child node relationship between each interface element and distribution information of at least one child node of each parent node;

The layout information is determined according to the parent-child node relationship and distribution information of at least one child node of each parent node.
The method according to claim 2, wherein determining the parent-child node relationship between each interface element based on the attribute information of each interface element in the interface element information includes:

Determine the initial parent-child node relationship between each interface element based on the position information and drawing sequence information of each interface element;

In the initial parent-child node relationship, if there is a designated class interface element as a parent node, a first container node is added to the initial parent node relationship to accommodate the designated class interface element and the designated class The child node of the interface element to obtain the parent-child node relationship.
The method according to claim 2, wherein each parent node includes a first type of parent node;

And, determining the distribution information of at least one first child node of the first type parent node includes:

According to the position information of the at least one first child node, the row and column information of the at least one first child node and the surrounding distance information of each first child node in the at least one first child node are determined.
The method according to claim 4, further comprising:

Determine the telescopic layout attributes of each row and the telescopic layout attributes of each column according to the surrounding information of each first sub-node among the at least one first sub-node;

The correction scheme for the layout information is determined according to the scalable layout attributes of each row and the scalable layout attributes of each column.
The method according to claim 5, wherein determining the correction scheme for the layout information according to the scalable layout attributes of the rows and the scalable layout attributes of the columns includes:

When the scalable layout attributes of the rows are consistent, add the scalable layout attributes of any row to the first-type parent node in the layout information;

When the telescopic layout attributes of the columns are consistent, add the telescopic layout attributes of any column to the first type of parent node in the layout information;

When the telescopic layout attributes of the rows are inconsistent and the telescopic layout attributes of the columns are also inconsistent, in the layout information, a second container node is added to each row to accommodate the first child node on each row, and is The second container node newly added in each row adds a telescopic layout attribute corresponding to each row, wherein the second container node in each row is a child node of the parent node of the first type.
The method according to claim 2, wherein each parent node includes a second type of parent node;

And, determining the distribution information of at least one second child node of the second type parent node includes:

According to the position information of the second-type parent node and the at least one second child node, perform absolute positioning layout on the at least one second child node to obtain distribution information of the at least one second child node.
The method according to any one of claims 1 to 7, wherein generating the interface code according to the layout information includes:

According to the layout information, the corresponding code layout information is matched from the code library;

According to the attribute information of each interface element in the layout information, matching the attribute information of each node in the code layout information at the code level from the code library;

The attribute information of each node at the code level is added to the code layout information to obtain the interface code.
The method according to claim 8, wherein each of the interface elements comprises: a first interface element;

And, according to the attribute information of the first interface element, matching code attribute information of the first node corresponding to the first interface element in the code layout information from the code library includes:

According to the type attribute of the first interface element, matching the type attribute of the first node at the code level in the code base;

According to the display attributes of the first interface element, the display attributes of the first node at the code level are matched in the code library.
The method according to claim 8, wherein generating the interface code according to the layout information further comprises:

Before acquiring the corresponding code layout information according to the layout information, each interface element in the layout information is renamed according to the semantic naming rules.
The method according to any one of claims 1 to 7, wherein the interface element information includes: attribute information of each interface element;

The attribute information includes at least one of the following items: name, type, coordinates, width and height, drawing order, fill content, and style of the fill content.
The method according to any one of claims 1 to 7, wherein determining the interface element information of the image file to be processed includes:

Export layer attribute information of each layer from the image file to be processed, where the layer attribute information includes name, type, coordinates, width and height, drawing order, filling content, and style of filling content;

Perform preset rule processing on the layer attribute information of each layer;

The layer attribute information of each layer processed by the prediction rule is used as the attribute information of each interface element in the interface element information.
The method according to claim 12, wherein the step of performing preset rule processing on the layer attribute information of each layer includes at least one of the following:

By performing layer redundancy and sequence check on each layer, the layer attribute information of useless layers is deleted;

Correct the layer type of the layer with the wrong layer type by checking the layer type of each layer;

By performing layer merge verification on each layer, multiple layers with merge relevance are merged into one layer;

Combine the design rules of the design tool of the image file to be processed to modify the layer attribute information of each layer;

Convert the layer attribute information of each layer into data in a preset format.
The method according to claim 12, wherein the image file to be processed is a design drawing;

Exporting layer attribute information of each layer from the image file to be processed includes:

Performing text and image recognition on the design drawing to separate multiple layers from the design drawing;

Perform layer type identification on the multiple layers to determine the layer types of the multiple layers.
A method for generating interface code, characterized in that it includes:

Display operation interface for users to input image data to be processed;

In response to an interface code generation instruction triggered by the user through the operation interface, an interface code is generated based on the image data to be processed.
The method of claim 15, further comprising:

A control corresponding to at least one code language type is displayed on the operation interface for the user to select.
The method of claim 16, further comprising:

Generating the interface code generation instruction in response to the user's trigger operation on the control corresponding to the first code language type, the interface code generation instruction carrying the first code language type;

And, generating the interface code based on the image file to be processed includes:

Determine a code base corresponding to the first code language type;

In combination with the code base, an interface code is generated based on the image data to be processed.
The method according to any one of claims 15 to 17, wherein the image data to be processed is interface element information derived from the image file to be processed;

Based on the to-be-processed image data, generating an interface code includes:

Determine the layout information of the interface based on the interface element information;

According to the layout information, an interface code is generated.
An electronic device, comprising: a first memory and a first processor, wherein,

The first memory is used to store programs;

The first processor, coupled to the first memory, is used to execute the program stored in the first memory for:

Determine the interface element information of the image file to be processed;

Determine the layout information of the interface based on the interface element information;

According to the layout information, an interface code is generated.
An electronic device, characterized by comprising: a second memory, a second processor and a second display, wherein,

The second memory is used to store programs;

The second display, coupled with the second processor, is used to display an operation interface for the user to input image data to be processed;

The second processor, coupled to the second memory, is used to execute the program stored in the second memory for:

In response to an interface code generation instruction triggered by the user through the operation interface, an interface code is generated based on the image data to be processed.