CN115202661A - Hybrid generation method with hierarchical structure layout and related equipment - Google Patents
Hybrid generation method with hierarchical structure layout and related equipment Download PDFInfo
- Publication number
- CN115202661A CN115202661A CN202211121484.6A CN202211121484A CN115202661A CN 115202661 A CN115202661 A CN 115202661A CN 202211121484 A CN202211121484 A CN 202211121484A CN 115202661 A CN115202661 A CN 115202661A
- Authority
- CN
- China
- Prior art keywords
- nodes
- node
- branch
- layout
- tree
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F8/00—Arrangements for software engineering
- G06F8/30—Creation or generation of source code
- G06F8/38—Creation or generation of source code for implementing user interfaces
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
- G06F17/10—Complex mathematical operations
- G06F17/11—Complex mathematical operations for solving equations, e.g. nonlinear equations, general mathematical optimization problems
- G06F17/12—Simultaneous equations, e.g. systems of linear equations
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/451—Execution arrangements for user interfaces
Abstract
The invention discloses a hybrid generation method with hierarchical structure layout and related equipment, wherein the method comprises the following steps: receiving a plurality of layout templates input by a user, wherein the layout templates are represented by a multi-branch tree structure, leaf nodes in the multi-branch tree structure represent single elements in the layout templates, and the corresponding relation among the elements is calculated according to a node corresponding rule to obtain corresponding information; establishing a combined tree according to the corresponding information, and dynamically determining intermediate state elements and relations which should be met between the elements according to node types and parameters generated during user interaction on the basis of the combined tree; and taking the positions and the sizes of the nodes as soft constraints, taking the relation which should be met among the nodes as hard constraints, optimizing an objective function, and generating a new diversity layout after the optimization solution is completed. The invention only needs to input a small number of layout templates without a large data set for training, thereby saving computing resources and leading a user to generate brand new diversified layouts through simple interaction.
Description
Technical Field
The present invention relates to the field of computer graphics technologies, and in particular, to a hybrid generation method, system, terminal, and computer-readable storage medium with hierarchical layout.
Background
The two-dimensional layout is widely applied to information carriers such as web pages, posters, magazines and the like. Different information modules can be divided by layout, so that one content presentation layout is obtained. The reasonable layout can accord with the reading habit of the user, and the user can conveniently and quickly obtain the key information. Therefore, it is a meaningful task to explore how to generate and apply a two-dimensional layout on information presentation.
For different forms of layout generation, there are mainly the following implementations: the first method is to solve and optimize the generation of the relationship between elements as a constraint. For example, (1) the alternative relations existing among a series of elements, the size range of the elements and the size of the display interface are required to be input, the various relations among the elements are connected through an 'OR' constraint, and the whole body is solved as a hard constraint to generate the layout adaptive to the sizes of various display devices. (2) A method is provided, which can automatically detect the possible relationship such as alignment, equal size and the like between elements, allow a user to manually edit the relationship between the elements in an interactive mode, and finally, take the position and the size of the elements as the relationship between soft constraint elements as the hard constraint solution optimization to generate a regular layout. (3) A framework capable of performing mixed interpolation on three-dimensional building layouts with similar structures is provided, the work input is a plurality of building layouts, the layout elements have clear correspondence, and the boundaries of the building layouts are used as hard constraints, and the lighting, the height, the courtyard and the like of rooms are used as soft constraints. The hard constraint and the soft constraint are combined to define the standard of 'good layout', and then a new layout with the same structure as the input layout is generated by means of linear interpolation. Another way is to automatically generate the layout based on deep learning. (4) A generating model is trained by utilizing a deep neural network, and then a two-dimensional house type graph layout can be automatically generated by combining with input constraints of a user. (5) A completely new framework is proposed that utilizes a self-attention mechanism to learn the contextual relationships between layout elements and then for the generation of new layouts.
The above methods have certain drawbacks, for example, (1) and (3) when generating the layout, they require explicit correspondence between the input layout elements; (2) requiring a user to manually edit relationships between elements; (4) And (5) these deep learning based layout generation efforts tend to require a large data set for training the model, such approaches require significant computational resources.
Accordingly, there is a need for improvements and developments in the art.
Disclosure of Invention
The invention mainly aims to provide a hybrid generation method, a hybrid generation system, a hybrid generation terminal and a computer readable storage medium with hierarchical structure layout, and aims to solve the problem that generation of a new layout is too complex when information display is carried out in the prior art.
In order to achieve the above object, the present invention provides a hybrid generation method with a hierarchical layout, including the steps of:
receiving a plurality of layout templates input by a user, wherein the layout templates are represented by a multi-branch tree structure, leaf nodes in the multi-branch tree structure represent single elements in the layout templates, and the corresponding relation between the elements is calculated according to a node corresponding rule to obtain corresponding information;
establishing a combined tree according to the corresponding information, and dynamically determining intermediate state elements and relations which should be met between the elements based on the combined tree according to node types and in combination with parameters generated during user interaction;
and taking the positions and the sizes of the nodes as soft constraints, taking the relation which should be met among the nodes as hard constraints, optimizing an objective function, and generating a new diversity layout after the optimization solution is completed.
Optionally, in the hybrid generation method with a hierarchical layout, the node correspondence rule includes a first correspondence rule, a second correspondence rule, a third correspondence rule, and a fourth correspondence rule;
the first correspondence rule includes:
two leaf nodes with similar geometric information tend to correspond together;
two branch nodes with similar substructures tend to correspond together;
two leaf nodes with different semantic information cannot be corresponded together;
the second rule of correspondence includes:
the leaf node and branch node allow correspondences to null;
the third rule of correspondence includes:
if two branch nodesAndcorrespond together to form a subtreeA node in (2) can only correspond to a sub-treeNode in or corresponding null;
the fourth rule of correspondence includes:
two nodes at different levels are allowed to correspond together;
wherein, the father node of the leaf node is a branch node.
Optionally, the hybrid generation method with a hierarchical structure layout, where the calculating a correspondence between elements according to a node correspondence rule to obtain correspondence information further includes:
defining a function for calculating corresponding losses between two nodes and between the nodes and the spaces by using a recursive method;
the corresponding cost of two leaf nodes defines two normal forms of the position difference of the two nodes plus two normal forms of the size difference plusA first penalty term associated with the semantic informationThe introduction of the first penalty term makes two nodes with different semantic information unable to be corresponded together, and is in the following form:
wherein, the first and the second end of the pipe are connected with each other,andrespectively representing two leaf nodes;calculating the corresponding cost of two leaf nodes;andrespectively representing the positions of two leaf nodes;andrespectively representing the sizes of two leaf nodes;andrespectively representing semantic information corresponding to the two leaf nodes;
leaf node corresponding to null costA loss corresponding to a node with a size of 0 for the leaf node, specifically a two-norm of the difference between the size of the node and 0 plus a second penalty termThe method comprises the following steps:
The empty cost corresponding to the branch node is defined as the cumulative sum of the empty costs corresponding to all leaf nodes in the branch, and is as follows:
wherein the content of the first and second substances,is a function of the costs of the branch nodes corresponding to the nulls,is a leaf node below the branch node;
cost of branch node corresponding to leaf node: recursively calling a calculation loss method to calculate the costs of the child nodes and the leaf nodes of the branch nodes, and calculating the costs of the child nodes of the branch nodes and the costs of the leaf nodes to be empty; and (3) constructing a two-dimensional matrix by using the calculated cost values, and solving by using a Hungarian algorithm to obtain a minimum loss and an optimal matching result.
Optionally, the hybrid generation method with a hierarchical layout includes:
neither branch node sinks; recursively invoking a calculate loss method to calculate a cost between child nodes of the first branch node and child nodes of the other branch node; calculating the cost of the child node of the first branch node to the null and the cost of the child node of the second branch node to the null; a two-dimensional matrix is constructed by the calculated cost values and solved by Hungary algorithm to obtain a loss value which is recorded as;
The first branch node sinks by one stage, and the second branch node does not sink; recursively invoking a compute loss method to compute a cost between a child node of a first branch node and another branch node; calculating the cost of the first branch node to be empty and the cost of the child nodes of the second branch node to be empty; constructing a two-dimensional matrix by using the calculated cost values and solving through a Hungarian algorithm to obtain a loss value which is recorded as;
The first branch node does not sink, and the second branch node sinks one level; recursively invoking a calculate loss method to calculate a cost between a child node of the first branch node and another branch node; calculating the cost of the child node of the first branch node to the null and the cost of the second branch node to the null; constructing a two-dimensional matrix by using the calculated cost values and solving through a Hungarian algorithm to obtain a loss value which is recorded as;
The final loss valueThe element correspondence results in the two branch nodes equal to the loss valueA small corresponding match; and transmitting the root nodes of the two layouts as parameters into a defined recursive function to obtain the total corresponding loss value and the corresponding information between the two layout elements.
Optionally, the hybrid generation method with a hierarchical structure layout, where the dynamically determining, based on the combined tree, the intermediate state elements and the relationships that should be satisfied between the elements according to the node types and in combination with parameters generated during user interaction specifically includes:
the combined tree comprises three types of nodes, wherein the first type of node is a node corresponding to both the two trees; the second kind of nodes are corresponding nodes only in the first tree; the third kind of nodes are corresponding nodes only in the second tree;
respectively and uniformly distributing a deletion threshold value for the second class node and the third class node according to the located levels, and dynamically determining the relation which should be met between the tree structure of the intermediate state and the brother nodes according to the input parameters;
the relationships among sibling nodes include equal size, equal spacing and alignment.
Optionally, the hybrid generation method with hierarchical structure layout, where the positions and sizes of the nodes are used as soft constraints, a relationship that should be satisfied between the nodes is used as a hard constraint, and the optimizing the objective function specifically includes:
when the parameter value is inputtedIf the value is less than 0.5, using the relation in the first tree, otherwise, using the relation in the second tree;
during solving, the position and the size are used as soft constraints, the relation which is required to be met between the nodes is used as hard constraints, and the objective function to be optimized is as follows:
wherein, the first and the second end of the pipe are connected with each other,is an energy equation that measures the difference in position,is an energy equation that measures the magnitude difference;
wherein the content of the first and second substances,is a node in the intermediate state tree;representing the amount of position to be solved for by the node;representing a magnitude value to be solved for the node; if it is notNull, indicating a functionOtherwise, otherwise;Representing nodesCorresponding to nodes in the first treeThe position of (a);representing nodesThe size of (d);representing nodesCorresponding to nodes in the second treeThe position of (a);representing nodesThe size of (d); the objective function is minimized under a series of hard constraints.
Optionally, the hybrid generation method with a hierarchical structure layout further includes:
if two nodes are vertically aligned, then there are:
wherein the content of the first and second substances,representing nodesThe abscissa of the corresponding element;representing nodesThe width of the corresponding element;representing nodesThe abscissa of the corresponding element;representing nodesThe width of the corresponding element.
In addition, to achieve the above object, the present invention further provides a hybrid generation system having a hierarchical layout, wherein the hybrid generation system having a hierarchical layout includes:
the hybrid generation system with hierarchical layout comprises:
the information calculation module is used for receiving a plurality of layout templates input by a user, wherein the layout templates are represented by a multi-branch tree structure, leaf nodes in the multi-branch tree structure represent single elements in the layout templates, and the corresponding relation between the elements is calculated according to a node corresponding rule to obtain corresponding information;
the relation determining module is used for establishing a combined tree according to the corresponding information, and dynamically determining the intermediate state elements and the relation which should be met between the elements according to the node types and the parameters generated during the user interaction based on the combined tree;
and the optimization generation module is used for taking the positions and the sizes of the nodes as soft constraints, taking the relation which should be met between the nodes as hard constraints, optimizing an objective function, and generating a new diversity layout after the optimization solution is completed.
In addition, to achieve the above object, the present invention further provides a terminal, wherein the terminal includes: the hybrid generation program with the hierarchical layout is executed by the processor to realize the steps of the hybrid generation method with the hierarchical layout.
Further, to achieve the above object, the present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a hybrid generation program having a hierarchical layout, which when executed by a processor implements the steps of the hybrid generation method having a hierarchical layout as described above.
Receiving a plurality of layout templates input by a user, wherein the layout templates are represented by a multi-branch tree structure, leaf nodes in the multi-branch tree structure represent single elements in the layout templates, and the corresponding relation between the elements is calculated according to a node corresponding rule to obtain corresponding information; establishing a combined tree according to the corresponding information, and dynamically determining intermediate state elements and relations which should be met between the elements according to node types and parameters generated during user interaction on the basis of the combined tree; and taking the positions and the sizes of the nodes as soft constraints, taking the relation which should be met between the nodes as hard constraints, optimizing an objective function, and generating a new diversity layout after the optimization solution is completed. The invention only needs to input a small amount of layout templates without a large data set for training, saves the computing resources and enables users to generate brand-new diversified layouts through simple interaction.
Drawings
FIG. 1 is a flow chart of a preferred embodiment of a hybrid generation method with a hierarchical layout of the present invention;
FIG. 2 is a diagram illustrating a first rule of the hybrid generation method with hierarchical layout according to the present invention;
FIG. 3 is a diagram illustrating a second rule according to the preferred embodiment of the hybrid generation method with hierarchical layout of the present invention;
FIG. 4 is a diagram illustrating a third rule according to a preferred embodiment of the hybrid generation method with hierarchical layout of the present invention;
FIG. 5 is a diagram illustrating a fourth rule according to the preferred embodiment of the present invention;
FIG. 6 is a diagram illustrating a combined tree created according to corresponding information in the preferred embodiment of the hybrid generation method with hierarchical layout of the present invention;
FIG. 7 is a diagram illustrating a tree structure process for dynamically determining intermediate states in a preferred embodiment of the hybrid generation method with hierarchical layout of the present invention;
FIG. 8 is a diagram illustrating a user-generated layout and its corresponding visualization in accordance with a preferred embodiment of the hybrid generation method with hierarchical layout of the present invention;
FIG. 9 is a schematic diagram of a preferred embodiment of the hybrid generation system with hierarchical layout of the present invention;
fig. 10 is a schematic diagram illustrating an operating environment of a terminal according to a preferred embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention clearer and clearer, the present invention is further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
The invention enables a user to generate a brand-new layout through simple interaction, firstly, the correspondence between layout elements needs to be calculated before carrying out mixed interpolation, and the calculated correspondence information is kept consistent with the division structure of the layout; establishing a combined tree according to the corresponding information, then establishing a proper node deletion strategy, and determining intermediate variables and constraint relations which should be met among the variables according to the input parameters; the method of the invention can calculate the optimal correspondence even under the condition that the elements do not have definite correspondence, and the element correspondence result is consistent with the division structure of the layout, so that the method can be used for mixed interpolation; the user only needs to input at least two layout templates and generate a new layout through simple interaction, and the generated result can be used for subsequent designer creation and the like.
As shown in fig. 1, the hybrid generation method with a hierarchical structure layout according to the preferred embodiment of the present invention includes the following steps:
step S10, receiving a plurality of layout templates input by a user, wherein the layout templates are represented by a multi-branch tree structure, leaf nodes in the multi-branch tree structure represent single elements in the layout templates, and the corresponding relation between the elements is calculated according to a node corresponding rule to obtain corresponding information.
Specifically, a layout template input by a user is first represented by a multi-branch tree structure (whether the input layout or the output layout is represented by a multi-branch tree), leaf nodes in the multi-branch tree structure represent single elements in the layout, branch nodes are bounding boxes of child nodes, parent nodes of the leaf nodes are branch nodes, for example: there are two siblings (children) corresponding to the two elements in the layout, respectively, and the parent (branch) of these two siblings) is a box that can just "wrap" completely around those two elements. Before the mixed interpolation is carried out, the correspondence between the elements is calculated, so a series of node correspondence rules are predefined.
The node correspondence rules comprise a first correspondence rule, a second correspondence rule, a third correspondence rule and a fourth correspondence rule.
As shown in fig. 2, the first correspondence rule includes:
(a) Two leaf nodes with similar geometric information tend to correspond together;
(b) Two branch nodes with similar substructures tend to correspond together;
(c) Two leaf nodes with different semantic information cannot be corresponded together.
As shown in fig. 3, the second rule includes:
(a) The leaf node and branch node allow correspondences are null.
As shown in fig. 4, the third rule includes:
(a) If two branch nodesAndcorrespond together to form a sub-treeA node in (2) can only correspond to a sub-treeOr corresponding null.
As shown in fig. 5, the fourth rule includes:
(a) Two nodes at different levels are allowed to correspond together.
Wherein, the father node of the leaf node is a branch node; the circle in fig. 2-5 is a node (branch node or leaf node), and if there is a line below the circle, the circle represents a branch node, and if there is no line below the circle, the circle represents a leaf node; those rectangular boxes of different colors are elements in the layout, represented by nodes.
Under the guidance of the correspondence rule, a function for calculating the correspondence loss between two nodes and between a node and a null is defined by a recursive method.
Firstly, corresponding cost of two leaf nodes defines two normal forms of position difference of two nodes, two normal forms of size difference and a first punishment item related to semantic informationThe introduction of the first penalty term enables two nodes with different semantic information not to be corresponding together, and the two nodes are in the following forms:
wherein the content of the first and second substances,andrespectively representing two leaf nodes;calculating the corresponding cost of the two leaf nodes;andrespectively representing the positions of two leaf nodes;andrespectively representing the sizes of two leaf nodes;andand respectively representing semantic information corresponding to the two leaf nodes.
Leaf node corresponding to null costA loss corresponding to a node with a size of 0 for the leaf node, specifically a two-norm of the difference between the size of the node and 0 plus a second penalty termThe method comprises the following steps:
The cost of the branch node corresponding to the null is defined as the cumulative sum of the costs of all leaf nodes corresponding to the null in the branch, and is:
wherein the content of the first and second substances,is a function of the costs of the branch nodes corresponding to the nulls,is the leaf node below the branch node.
Cost of a branch node corresponding to a leaf node: recursively calling a previously defined calculation loss method to calculate costs of child nodes and leaf nodes of the branch node; similarly, calculating the costs of the child nodes of the branch node to be empty and the costs of the leaf nodes to be empty; and then, a two-dimensional matrix is constructed by using the calculated cost values, and the minimum loss and the optimal matching result can be obtained by solving through the Hungarian algorithm.
The calculation cost of the branch node corresponding to the branch node is divided into three conditions:
(1) Neither branch node sinks; recursively invoking the previously defined computation penalty method to compute a cost between child nodes of the first branch node and child nodes of the other branch node; calculating the cost of the child node of the first branch node to the null and the cost of the child node of the second branch node to the null; using the calculated cost value to construct a two-dimensional matrix and passingObtaining a loss value recorded by solving the Hungary algorithm;
(2) The first branch node sinks by one level, and the second branch node does not sink; recursively invoking a previously defined computation penalty method to compute a cost between a child node of the first branch node and another branch node; calculating the cost of the first branch node being empty and the cost of the child nodes of the second branch node being empty; constructing a two-dimensional matrix by using the calculated cost values and solving through a Hungarian algorithm to obtain a loss value which is recorded as;
(3) The first branch node does not sink, and the second branch node sinks one level; recursively invoking the previously defined computation penalty method to compute a cost between a child node of the first branch node and another branch node; calculating the cost of the child node of the first branch node to the null and the cost of the second branch node to the null; a two-dimensional matrix is constructed by using the calculated cost values, and a loss value is obtained and recorded by solving through Hungary algorithm;
Then, the final loss valueThe element correspondence results within the two branch nodes equal the corresponding match with the smallest penalty value; and transmitting the root nodes of the two layouts as parameters into a defined recursive function to obtain the total corresponding loss value and the corresponding information between the two layout elements.
And S20, establishing a combined tree according to the corresponding information, and dynamically determining the intermediate state elements and the relation which should be met between the elements according to the node types and the parameters generated during the user interaction based on the combined tree.
Specifically, a combined tree is established according to the calculated correspondence between the elements; as shown in FIG. 6, the combined tree includes three types of nodes, and the first type of node is a node having correspondence in both trees, such as node [15, 16 ]](ii) a The second type of node is a node that has a correspondence only in the first tree, e.g. node 10,](ii) a The third type of node is a node that has correspondence only in the second tree, e.g., node,7]。
As shown in fig. 7, a deletion threshold is uniformly assigned to the second class node and the third class node according to the located hierarchy, and then the tree structure of the intermediate state and the relationship to be satisfied between sibling nodes (nodes having the same parent node are sibling nodes, for example, in the tree in the top left corner of fig. 6, node 10 and node 11 are sibling nodes) are dynamically determined according to the input parameters.
The relationships among sibling nodes include equal size, equal spacing and alignment.
And S30, taking the positions and the sizes of the nodes as soft constraints, taking the relation which should be met among the nodes as hard constraints, optimizing an objective function, and generating a new diversity layout after the optimization solution is completed.
In particular, when the parameter value is inputtedLess than 0.5, the relationship in the first tree (top left in fig. 6, i.e. the first layout of the input) is used, otherwise, the relationship in the second tree (top right in fig. 6, i.e. the second layout of the input) is used.
During solving, the position and the size are used as soft constraints, the relation which is required to be met between the nodes is used as hard constraints, and the objective function to be optimized is as follows:
wherein the content of the first and second substances,is an energy equation that measures the difference in position,is an energy equation that measures the difference in magnitude.
Wherein, the first and the second end of the pipe are connected with each other,is a node in the intermediate state tree;representing the amount of position to be solved for by the node;a magnitude value representing the node to be solved for; if it is usedNull, indicating a functionOtherwise;Representing nodesCorresponding to nodes in the first treeThe position of (a);representing nodesThe size of (d);representing nodesCorresponding to nodes in the second treeThe position of (a);representing nodesThe size of (d); the objective function is minimized under a series of hard constraints.
For example, if two nodes are vertically aligned, then there are:
wherein the content of the first and second substances,representing nodesSit across of corresponding elementMarking;representing nodesThe width of the corresponding element;representing nodesThe abscissa of the corresponding element;representing nodesThe width of the corresponding element.
Similarly, other hard constraints can be written in a similar linear equation form, and the position and size of each node can be obtained by solving the quadratic programming problem.
As shown in fig. 8, a plurality of experimenters are invited to participate in the experience of the interactive system, and the experimenters select three layout templates from the candidate layout template set as inputs, and automatically generate the layout by the method of the present invention. The experimenter selects 5 different layouts from the generated results and stores the layouts, and the process is repeated for 5 times. Finally, user research is carried out on experimenters, and the experimenters show that various, reasonable and beautiful layouts can be generated by using the tool (method) provided by the invention. The present invention further uses these results in the production of contents such as posters and the like, and achieves an aesthetic visual effect.
The method provided by the invention only needs to input a small number of layout templates and does not need a large data set for training, so that the computing resources are saved; the user can conveniently generate diversified layouts through simple interaction.
The main content of the invention is a hybrid generation method with hierarchical structure layout, after a user inputs a plurality of layout templates, the invention can automatically calculate the correspondence between elements, then establish a combined tree according to the corresponding information, dynamically determine the relationship between the elements in the intermediate state and the elements which should be satisfied by combining the parameters generated during the user interaction, and finally optimize and solve to generate a new layout. Further, a partial editing function is provided, and a user can fix or individually change a certain part of the generated result for increasing the diversity of the generated layout.
Further, as shown in fig. 9, based on the above hybrid generation method with a hierarchical structure layout, the present invention also provides a hybrid generation system with a hierarchical structure layout, where the hybrid generation system with a hierarchical structure layout includes:
the information calculation module 51 is configured to receive multiple layout templates input by a user, where the layout templates are represented by using a multi-branch tree structure, leaf nodes in the multi-branch tree structure represent a single element in the layout template, and calculate a correspondence between elements according to a node correspondence rule to obtain corresponding information;
the relationship determining module 52 is configured to establish a combined tree according to the corresponding information, and dynamically determine, based on the combined tree, intermediate state elements and relationships that should be satisfied between the elements according to node types and in combination with parameters generated during user interaction;
and the optimization generation module 53 is configured to use the positions and sizes of the nodes as soft constraints, use the relationship that should be satisfied between the nodes as hard constraints, optimize an objective function, and generate a new diversity layout after the optimization solution is completed.
Further, as shown in fig. 10, based on the above hybrid generation method and system with hierarchical layout, the present invention also provides a terminal, which includes a processor 10, a memory 20, and a display 30. Fig. 10 shows only some of the components of the terminal, but it should be understood that not all of the shown components are required to be implemented, and that more or fewer components may be implemented instead.
The memory 20 may in some embodiments be an internal storage unit of the terminal, such as a hard disk or a memory of the terminal. The memory 20 may also be an external storage device of the terminal in other embodiments, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), and the like provided on the terminal. Further, the memory 20 may also include both an internal storage unit and an external storage device of the terminal. The memory 20 is used for storing application software installed in the terminal and various types of data, such as program codes of the installation terminal. The memory 20 may also be used to temporarily store data that has been output or is to be output. In an embodiment, the memory 20 stores a hybrid generation program 40 with a hierarchical layout, and the hybrid generation program 40 with a hierarchical layout can be executed by the processor 10, so as to implement the hybrid generation method with a hierarchical layout in the present application.
The processor 10 may be a Central Processing Unit (CPU), a microprocessor or other data Processing chip in some embodiments, and is used for running program codes stored in the memory 20 or Processing data, such as executing the hybrid generation method with hierarchical layout, and the like.
The display 30 may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, an OLED (Organic Light-Emitting Diode) touch panel, or the like in some embodiments. The display 30 is used for displaying information at the terminal and for displaying a visual user interface. The components 10-30 of the terminal communicate with each other via a system bus.
In an embodiment, the steps of the hybrid generation method with hierarchical layout are implemented when the processor 10 executes the hybrid generation program 40 with hierarchical layout in the memory 20.
The present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a hybrid generation program with a hierarchical layout, which when executed by a processor implements the steps of the hybrid generation method with a hierarchical layout as described above.
In summary, the present invention provides a hybrid generation method with a hierarchical layout and a related device, where the method includes: receiving a plurality of layout templates input by a user, wherein the layout templates are represented by a multi-branch tree structure, leaf nodes in the multi-branch tree structure represent single elements in the layout templates, and the corresponding relation between the elements is calculated according to a node corresponding rule to obtain corresponding information; establishing a combined tree according to the corresponding information, and dynamically determining intermediate state elements and relations which should be met between the elements based on the combined tree according to node types and in combination with parameters generated during user interaction; and taking the positions and the sizes of the nodes as soft constraints, taking the relation which should be met between the nodes as hard constraints, optimizing an objective function, and generating a new diversity layout after the optimization solution is completed. The invention only needs to input a small amount of layout templates without a large data set for training, saves the computing resources and enables users to generate brand-new diversified layouts through simple interaction.
It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or terminal that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or terminal. Without further limitation, an element defined by the phrase "comprising one of 8230, and" comprising 8230does not exclude the presence of additional like elements in a process, method, article, or terminal comprising the element.
Of course, it will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by instructing relevant hardware (such as a processor, a controller, etc.) through a computer program, and the program can be stored in a computer readable storage medium, and when executed, the program can include the processes of the embodiments of the methods described above. The computer readable storage medium may be a memory, a magnetic disk, an optical disk, etc.
It is to be understood that the invention is not limited to the examples described above, but that modifications and variations may be effected thereto by those of ordinary skill in the art in light of the foregoing description, and that all such modifications and variations are intended to be within the scope of the invention as defined by the appended claims.
Claims (10)
1. A hybrid generation method with a hierarchical layout, the hybrid generation method with a hierarchical layout comprising:
receiving a plurality of layout templates input by a user, wherein the layout templates are represented by a multi-branch tree structure, leaf nodes in the multi-branch tree structure represent single elements in the layout templates, and the corresponding relation among the elements is calculated according to a node corresponding rule to obtain corresponding information;
establishing a combined tree according to the corresponding information, and dynamically determining intermediate state elements and relations which should be met between the elements according to node types and parameters generated during user interaction on the basis of the combined tree;
and taking the positions and the sizes of the nodes as soft constraints, taking the relation which should be met among the nodes as hard constraints, optimizing an objective function, and generating a new diversity layout after the optimization solution is completed.
2. The hybrid generation method with hierarchical layout of claim 1, wherein the node correspondence rule includes a first correspondence rule, a second correspondence rule, a third correspondence rule, and a fourth correspondence rule;
the first correspondence rule includes:
two leaf nodes with similar geometric information tend to correspond together;
two branch nodes with similar substructures tend to correspond together;
two leaf nodes with different semantic information cannot be corresponded together;
the second correspondence rule includes:
the leaf node and branch node allow correspondences to null;
the third rule of correspondence includes:
if two branch nodesAndcorrespond together to form a sub-treeA node in (2) can only correspond to a sub-treeNode in or corresponding null;
the fourth rule of correspondence includes:
two nodes at different levels are allowed to correspond together;
wherein, the father node of the leaf node is a branch node.
3. The method of claim 2, wherein the calculating the correspondence between the elements according to the node correspondence rule to obtain the correspondence information further comprises:
defining a function for calculating corresponding loss between two nodes and between the node and the space by using a recursive method;
the corresponding cost of two leaf nodes defines two normal forms of the position difference of two nodes, the two normal forms of the size difference and a first punishment item related to semantic informationThe introduction of the first penalty term makes two nodes with different semantic information unable to be corresponded together, and is in the following form:
wherein the content of the first and second substances,andrespectively representing two leaf nodes;calculating the corresponding cost of two leaf nodes;andrespectively representing the positions of two leaf nodes;andrespectively representing the sizes of two leaf nodes;andrespectively representing semantic information corresponding to the two leaf nodes;
leaf node corresponding to null costCorresponding to a loss of a node of size 0 for the leaf node, in particular the nodeA second penalty term added to the two normal form of the difference between the magnitude and 0The method comprises the following steps:
The empty cost corresponding to the branch node is defined as the cumulative sum of the empty costs corresponding to all leaf nodes in the branch, and is as follows:
wherein the content of the first and second substances,is a function of the costs of the branch nodes corresponding to the nulls,is a leaf node below the branch node;
cost of branch node corresponding to leaf node: recursively calling a calculation loss method to calculate the costs of child nodes and leaf nodes of the branch nodes, and calculating the costs of the child nodes and the leaf nodes of the branch nodes of being empty; and (3) constructing a two-dimensional matrix by using the calculated cost values, and solving by using a Hungarian algorithm to obtain a minimum loss and an optimal matching result.
4. The hybrid generation method with hierarchical layout of claim 3, wherein the computation cost of the branch node corresponding to the branch node comprises:
two branch nodesDo not sink; recursively invoking a compute penalty method to compute a cost between a child node of a first branch node and a child node of another branch node; calculating the cost of the child node of the first branch node to the null and the cost of the child node of the second branch node to the null; a two-dimensional matrix is constructed by the calculated cost values and solved by Hungary algorithm to obtain a loss value which is recorded as;
The first branch node sinks by one level, and the second branch node does not sink; recursively invoking a compute loss method to compute a cost between a child node of a first branch node and another branch node; calculating the cost of the first branch node being empty and the cost of the child nodes of the second branch node being empty; a two-dimensional matrix is constructed by using the calculated cost values, and a loss value is obtained and recorded by solving through Hungary algorithm;
The first branch node does not sink, and the second branch node sinks one level; recursively invoking a compute penalty method to compute a cost between a child node of the first branch node and another branch node; calculating the cost of the child node of the first branch node to the null and the cost of the second branch node to the null; a two-dimensional matrix is constructed by using the calculated cost values, and a loss value is obtained and recorded by solving through Hungary algorithm;
The final loss valueThe element correspondence results in the two branch nodes are equal to the corresponding match with the smallest loss value; the root nodes of the two layouts are taken as parameters to be transmitted into a defined recursive function to obtain the corresponding total loss valueAnd correspondence information between the two layout elements.
5. The hybrid generation method with hierarchical layout according to claim 4, wherein the dynamically determining the intermediate state elements and the relationships that should be satisfied between the elements based on the composition tree according to the node types and in combination with the parameters generated during the user interaction specifically includes:
the combined tree comprises three types of nodes, wherein the first type of node is a node corresponding to both the two trees; the second type of node is a node corresponding to the first tree; the third kind of nodes are corresponding nodes only in the second tree;
respectively and uniformly distributing a deletion threshold value for the second class node and the third class node according to the located levels, and dynamically determining the relation which should be met between the tree structure of the intermediate state and the brother nodes according to the input parameters;
among them, the relationships between sibling nodes include equal size, equal spacing and alignment.
6. The hybrid generation method with hierarchical layout according to claim 5, wherein the optimizing the objective function by using the position and size of the nodes as soft constraints and the relationship that the nodes should satisfy as hard constraints specifically includes:
when the parameter value is inputtedIf the value is less than 0.5, using the relation in the first tree, otherwise, using the relation in the second tree;
during solving, the position and the size are used as soft constraints, the relation which is required to be met between the nodes is used as hard constraints, and the objective function to be optimized is as follows:
wherein, the first and the second end of the pipe are connected with each other,is an energy equation that measures the difference in position,is an energy equation for measuring the size difference;
wherein, the first and the second end of the pipe are connected with each other,is a node in the intermediate state tree;representing the amount of position that the node is to solve for;a magnitude value representing the node to be solved for; if it is notNull, then indicating a functionOtherwise;Representing nodesCorresponding to nodes in the first treeThe position of (a);representing nodesThe size of (d);representing nodesCorresponding to nodes in the second treeThe position of (a);representing nodesThe size of (d); the objective function is minimized under a series of hard constraints.
7. The method of claim 6, further comprising:
if two nodes are vertically aligned, then there are:
wherein, the first and the second end of the pipe are connected with each other,representing nodesThe abscissa of the corresponding element;representing nodesThe width of the corresponding element;representing nodesThe abscissa of the corresponding element;representing nodesThe width of the corresponding element.
8. A hybrid generation system having a hierarchical layout, the hybrid generation system having a hierarchical layout comprising:
the information calculation module is used for receiving a plurality of layout templates input by a user, wherein the layout templates are represented by a multi-branch tree structure, leaf nodes in the multi-branch tree structure represent single elements in the layout templates, and the corresponding relation among the elements is calculated according to a node corresponding rule to obtain corresponding information;
the relation determining module is used for establishing a combined tree according to the corresponding information and dynamically determining the intermediate state elements and the relation which should be met among the elements based on the combined tree according to the node type and in combination with the parameters generated during user interaction;
and the optimization generation module is used for taking the positions and the sizes of the nodes as soft constraints, taking the relation which should be met between the nodes as hard constraints, optimizing an objective function, and generating a new diversity layout after the optimization solution is completed.
9. A terminal, characterized in that the terminal comprises: memory, a processor and a hybrid generating program with a hierarchical layout stored on the memory and executable on the processor, the hybrid generating program with a hierarchical layout implementing the steps of the hybrid generating method with a hierarchical layout of any one of claims 1 to 7 when executed by the processor.
10. A computer-readable storage medium, characterized in that the computer-readable storage medium stores a hybrid generation program having a hierarchical layout, which when executed by a processor implements the steps of the hybrid generation method having a hierarchical layout according to any one of claims 1 to 7.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202211121484.6A CN115202661B (en) | 2022-09-15 | 2022-09-15 | Hybrid generation method with hierarchical structure layout and related equipment |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202211121484.6A CN115202661B (en) | 2022-09-15 | 2022-09-15 | Hybrid generation method with hierarchical structure layout and related equipment |
Publications (2)
Publication Number | Publication Date |
---|---|
CN115202661A true CN115202661A (en) | 2022-10-18 |
CN115202661B CN115202661B (en) | 2022-11-29 |
Family
ID=83571840
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202211121484.6A Active CN115202661B (en) | 2022-09-15 | 2022-09-15 | Hybrid generation method with hierarchical structure layout and related equipment |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN115202661B (en) |
Cited By (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2023185917A1 (en) * | 2022-03-30 | 2023-10-05 | 华为技术有限公司 | Module arrangement method for chip and related device |
CN117058491A (en) * | 2023-10-12 | 2023-11-14 | 深圳大学 | Structured grid layout generation method and device based on recurrent neural network |
CN117828701A (en) * | 2024-03-05 | 2024-04-05 | 中国石油大学(华东) | Engineering drawing layout optimization method, system, equipment and medium |
Citations (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN101339571A (en) * | 2007-11-01 | 2009-01-07 | 复旦大学 | VLSI layout planning centralized constrain implementing method |
US20130107698A1 (en) * | 2011-11-01 | 2013-05-02 | Alcatel-Lucent Usa Inc. | Ip fast reroute scheme offering full protection |
US20190378010A1 (en) * | 2018-06-12 | 2019-12-12 | Bank Of America Corporation | Unsupervised machine learning system to automate functions on a graph structure |
CN111354076A (en) * | 2020-02-29 | 2020-06-30 | 北京航空航天大学 | Single-image three-dimensional part combined modeling method based on embedding space |
CN111625235A (en) * | 2020-04-17 | 2020-09-04 | 北京大学 | Method and system for constructing and deconstructing tree visualization form based on descriptive language |
CN113420804A (en) * | 2021-06-18 | 2021-09-21 | 工业互联网创新中心(上海)有限公司 | Data processing method, device, network equipment and storage medium |
CN113688593A (en) * | 2021-08-11 | 2021-11-23 | 上海交通大学 | Hybrid bonding layout and wiring optimization method among three-dimensional integrated circuit chips |
-
2022
- 2022-09-15 CN CN202211121484.6A patent/CN115202661B/en active Active
Patent Citations (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN101339571A (en) * | 2007-11-01 | 2009-01-07 | 复旦大学 | VLSI layout planning centralized constrain implementing method |
US20130107698A1 (en) * | 2011-11-01 | 2013-05-02 | Alcatel-Lucent Usa Inc. | Ip fast reroute scheme offering full protection |
US20190378010A1 (en) * | 2018-06-12 | 2019-12-12 | Bank Of America Corporation | Unsupervised machine learning system to automate functions on a graph structure |
CN111354076A (en) * | 2020-02-29 | 2020-06-30 | 北京航空航天大学 | Single-image three-dimensional part combined modeling method based on embedding space |
CN111625235A (en) * | 2020-04-17 | 2020-09-04 | 北京大学 | Method and system for constructing and deconstructing tree visualization form based on descriptive language |
CN113420804A (en) * | 2021-06-18 | 2021-09-21 | 工业互联网创新中心(上海)有限公司 | Data processing method, device, network equipment and storage medium |
CN113688593A (en) * | 2021-08-11 | 2021-11-23 | 上海交通大学 | Hybrid bonding layout and wiring optimization method among three-dimensional integrated circuit chips |
Non-Patent Citations (2)
Title |
---|
ALSTRUP, S 等: "Efficient tree layout in a multilevel memory hierar", 《ARXIV PREPRINT CS》 * |
李煜 等: "基于O-TREE树表示的总线约束在VLSI/PCB布局中的应用", 《成都信息工程学院学报》 * |
Cited By (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2023185917A1 (en) * | 2022-03-30 | 2023-10-05 | 华为技术有限公司 | Module arrangement method for chip and related device |
CN117058491A (en) * | 2023-10-12 | 2023-11-14 | 深圳大学 | Structured grid layout generation method and device based on recurrent neural network |
CN117058491B (en) * | 2023-10-12 | 2024-04-02 | 深圳大学 | Structured grid layout generation method and device based on recurrent neural network |
CN117828701A (en) * | 2024-03-05 | 2024-04-05 | 中国石油大学(华东) | Engineering drawing layout optimization method, system, equipment and medium |
Also Published As
Publication number | Publication date |
---|---|
CN115202661B (en) | 2022-11-29 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN115202661B (en) | Hybrid generation method with hierarchical structure layout and related equipment | |
CN101971204B (en) | Arranging graphic objects on a page with relative position based control | |
CN1834903B (en) | Method and system for fitting text to shapes within a graphic | |
US20060048044A1 (en) | Logical spreadsheets | |
WO2015196784A1 (en) | Visual software modeling method based on software meta-view for constructing software view | |
CN109933311A (en) | A kind of information system creation method and relevant apparatus | |
CN110263281B (en) | Device and method for self-adapting page resolution in data visualization development | |
US11557088B2 (en) | Generating space models from map files | |
WO2015196788A1 (en) | Visual interface modelling editor for constructing interface model | |
WO2015196785A1 (en) | Visual software modelling editor for constructing software model | |
CN114117645B (en) | Ship overall performance forecasting integrated application system | |
Martin et al. | Ruleset-rewriting for procedural modeling of buildings | |
Li et al. | Hitailor: Interactive transformation and visualization for hierarchical tabular data | |
CN106096159B (en) | A kind of implementation method of distributed system behavior simulation analysis system under cloud platform | |
KR100807061B1 (en) | Automatic design method with MS Excel | |
CN117009038B (en) | Graph computing platform based on cloud native technology | |
Cruz et al. | Drawing graphs by example efficiently: Trees and planar acyclic digraphs | |
CN109977514B (en) | Method for generating model scheduling sequence of radar synchronous data flow graph | |
CN103593496A (en) | Model for managing variations in a product structure for a product | |
WO2015196786A1 (en) | Interface element model-based universal interface modelling method for constructing interface model | |
Hall | The Lish: A Data Model for Grid Free Spreadsheets | |
WO2015196787A1 (en) | Interface element view-based visual interface modelling method for constructing interface view | |
CN114218745A (en) | Intelligent design method for model-driven autonomous evolution | |
Jesus et al. | Generalized selections for direct control in procedural buildings | |
Bottoni et al. | Enforced generative patterns for the specification of the syntax and semantics of visual languages |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant |