WO2023097990A1

WO2023097990A1 - Element layout method and related device

Info

Publication number: WO2023097990A1
Application number: PCT/CN2022/093727
Authority: WO
Inventors: 赵刚华; 刘坤
Original assignee: 华为云计算技术有限公司
Priority date: 2021-11-30
Filing date: 2022-05-19
Publication date: 2023-06-08
Also published as: CN116206012A

Abstract

The present application provides an element layout method, comprising: receiving a plurality of elements inputted by a user, determining initial sizes and initial locations of the plurality of elements in a canvas, and starting from the initial sizes and initial locations of the plurality of elements in the canvas in combination with the size of the canvas and design constraints, iteratively optimizing the sizes and locations of the plurality of elements in the canvas to obtain a layout result, the layout result comprising recommended sizes and recommended locations of the plurality of elements in the canvas. In the method, an element layout problem is transformed into a linear programming problem, and for a set of inputted elements, recommended locations and recommended sizes of elements are determined by using an iterative optimization approach, such that an intelligent element layout is achieved. The method is also applicable to elements of types such as charts, achieves a good layout effect in the layout of elements such as charts, and therefore is highly available.

Description

Element layout method and related equipment

This application claims the priority of the Chinese patent application with the application number 202111447457.3 and the title of the invention "element layout method and related equipment" filed with the State Intellectual Property Office of China on November 30, 2021, the entire contents of which are incorporated by reference in this application middle.

technical field

The present application relates to the field of computer technology, and in particular to an element layout method, a system, a computer cluster, a computer-readable storage medium, and a computer program product.

Background technique

In daily work or life, it is often necessary to perform content demonstration through a screen, for example, a large-size screen (also referred to as a large screen for short), so as to improve communication efficiency. Among them, the content of the screen presentation is usually provided by the element. An element refers to a component that expresses a certain meaning. For example, an element can include different types of components such as text and graphics.

As users add and modify elements, the layout of elements on the screen needs to change accordingly. That is to say, in the whole design process of the visual screen, it is necessary to constantly adjust the layout of the elements. The diversity of element combinations and the complexity of layout goals, such as mutual alignment between elements and element filling, have brought great challenges to the layout of elements on the screen.

The industry provides methods for automatically laying out elements based on templates and dynamic programming. This method provides a variety of layout templates, and when the user inputs elements, the corresponding layout templates can be automatically matched based on dynamic programming, so as to realize element layout. However, the above method is mainly suitable for text-type elements. The format of elements such as charts is more free, and it is difficult to match the layout template, which leads to poor layout effects based on templates and dynamic programming.

Contents of the invention

This application provides an element layout method, which transforms the element layout problem into a linear programming problem, and uses an iterative optimization method to determine the recommended position and size of elements for a set of input elements, thereby realizing intelligent element layout . Since there is no need to match with a fixed layout template, it solves the problem that it is not suitable for elements such as charts, resulting in poor layout effects. The present application also provides an element layout system, a computer cluster, a computer-readable storage medium, and a computer program product corresponding to the above method.

In a first aspect, the present application provides an element layout method. This method can be performed by the element layout system. In some embodiments, the element layout system may be a software system, and the computer cluster executes the element layout method by running the program code of the software system. In some other embodiments, the element layout system may also be a hardware system for element layout.

Specifically, the element layout system may receive multiple elements input by the user, determine the initial size and initial position of the multiple elements on the canvas, and then use the initial size and initial position of the multiple elements on the canvas as a starting point, combine the The size and design constraints of the canvas, iteratively optimize the size and position of the multiple elements on the canvas, and obtain the recommended size and position of the multiple elements on the canvas, thereby realizing intelligent elements layout.

In this method, the element layout system transforms the element layout problem into a linear programming problem, and uses an iterative optimization method to solve the recommended position and recommended size of each element for a set of input elements (usually multiple elements), so that Each element does not overlap other elements and does not overflow the canvas. Because it does not match with a fixed layout template, it solves the problem in the related art that it is not suitable for elements of types such as charts and the layout effect is not good, and has high usability. Moreover, the method realizes driving based on user intent by setting necessary layout attributes such as the position of the upper, lower, left, and right edges of the element, and performing iterative optimization based on the initial size and initial position of the element input by the user. , which improves interactivity and reduces the workload of designers. In addition, this method uses a fixed-size canvas instead of a pixel-discretized canvas, so that continuous decision variables can be used instead of a large number of discrete decision variables, and iterative optimization can be performed in the continuous space provided by continuous decision variables, which can greatly reduce the amount of calculation. The algorithm complexity shortens the time of intelligent layout and improves the efficiency of intelligent layout.

In some possible implementation manners, the element layout system iteratively optimizes the sizes and positions of the multiple elements on the canvas to obtain multiple candidate layouts. Each candidate layout of the plurality of candidate layouts includes a size and position of the plurality of elements on the canvas. Then the element layout system sorts multiple candidate layouts to obtain layout results. The layout result is a top-ranked candidate layout among the multiple candidate layouts, for example, the layout result may be the first-ranked candidate layout among the multiple candidate layouts.

In this method, the element layout system sorts a plurality of candidate layouts, and determines the layout result from the plurality of candidate results according to the sorting result, which further improves the layout effect and meets business requirements.

In some possible implementations, the element layout system may be based on the balance of the size of each of the multiple elements, the balance of the images presented by the multiple elements on the canvas, and the alignment of the multiple elements At least one of the multiple candidate layouts is sorted. In this way, the layout effect can be sorted from at least one of the dimensions of the balance of the size of the element, the balance of the picture presented by the element on the canvas, and the alignment of the elements, so as to determine the balance of the size of the element, the balance of the element on the canvas The balance of the presented picture and/or the alignment of elements meet the required layout results.

In some possible implementation manners, the balance of the images presented by the multiple elements on the canvas is represented by a sum of importance scores of each element. Wherein, the importance score is determined according to the area of the element and the distance from the element to the center point of the canvas. In this way, the elements with higher importance can be laid out according to the weight, so that the elements with higher importance are in the visual focus and meet business needs.

In some possible implementation manners, the design constraints may include not only the built-in basic constraints of the system, but also user-defined constraints. In addition to iteratively optimizing the size and position of elements based on the built-in basic constraints of the system, the element layout system can also receive user-defined constraints and combine them to iteratively optimize the size and position of elements. This enables smart element layout based on user intent.

In some possible implementation manners, the basic constraints include any one or more of alignment constraints, grid constraints, size constraints, boundary constraints, and filling constraints. The self-defined constraints include any one or more of locking constraints, aspect ratio constraints, and alignment preference constraints.

Alignment constraints are specifically used to ensure the alignment of elements. Grid constraints are used to ensure a reasonable division of the canvas area and prevent elements from overlapping. Size constraints are used to ensure that the size (ie size) of elements is reasonable to meet the readability of elements such as charts by the human eye. The boundary constraint is used to ensure that the input elements fill the canvas, and the filling constraint is used to ensure that the interior of the layout is full, so as to avoid the internal vacancy of the layout and make the overall space lose the sense of continuity, which in turn affects the aesthetics of the layout to a certain extent.

Lock constraints are used to lock the size and position of one or more elements. Aspect ratio constraints are used to constrain the scaling ratio of one or more elements. Alignment preference constraints are used to constrain one or more elements to be top, left, right, or bottom.

By iteratively optimizing the size and position of elements under the above constraints, the final layout result can have a better layout effect and meet business needs.

In some possible implementations, the elements include one or more of graphics, text or decorations. Wherein, the decoration may refer to, for example, a graph, a title of a text, a background, and the like. In this method, the element layout system can implement intelligent element layout for different types of elements such as charts, texts, and decorations, and has high usability.

In some possible implementations, the element layout system can present a design interface to the user, the design interface carries a visual component, and the user can trigger a drag operation on the visual component, and the element layout system responds to the above operation of the user, receiving Multiple elements corresponding to visual components.

Through the above visual components, user operations are simplified and the efficiency of element layout is improved. Moreover, by dragging and dropping visual components to the design interface, the transparency of the element layout process can be guaranteed, and it is convenient for users to manually adjust the layout of elements to meet users' refined layout requirements.

In some possible implementations, the element layout system can set trigger conditions for smart element layout. For example, the trigger condition can be set such that the user triggers the smart layout control, or the user's vision of the last input element is greater than the preset time. Based on this, when the user triggers the smart layout control or the time since the user last entered an element is greater than the preset time, the trigger condition of the smart element layout is met, and the element layout system can use the initial size and initial size of the multiple elements on the canvas The position is used as a starting point, and the size and position of the multiple elements on the canvas are iteratively optimized in combination with the size of the canvas and design constraints.

The method sets the triggering condition of the smart element layout, and automatically triggers the smart element layout when the triggering condition of the smart element layout is satisfied, further reduces user operations, improves the efficiency of the element layout, and improves user experience.

In a second aspect, the present application provides an element layout system. The system includes:

An interactive module for receiving multiple elements input by the user;

A determining module, configured to determine the initial size and initial position of the plurality of elements on the canvas;

An optimization module, configured to iteratively optimize the size and position of the multiple elements on the canvas, taking the initial size and initial position of the multiple elements on the canvas as a starting point, in combination with the size of the canvas and design constraints, A layout result is obtained, where the layout result includes recommended sizes and recommended positions of the plurality of elements on the canvas.

In some possible implementation manners, the optimization module is specifically used to:

Iteratively optimizing the size and position of the multiple elements on the canvas to obtain multiple candidate layouts, each of the multiple candidate layouts includes the size and position of the multiple elements on the canvas ;

Sorting the multiple candidate layouts to obtain a layout result, where the layout result is a top-ranked candidate layout among the multiple candidate layouts.

According to at least one of the degree of balance of the size of each of the plurality of elements, the degree of balance of the images presented by the plurality of elements on the canvas, and the degree of alignment of the plurality of elements, the plurality of candidate layouts Sort.

In some possible implementation manners, the balance degree of the picture presented by the multiple elements on the canvas is characterized by the sum of the importance scores of each element, and the importance score is based on the area of the element and the distance between the element and the Determine the distance from the center point of the canvas.

In some possible implementations, the interaction module is also used to:

Receive custom constraints entered by the user;

The design constraints include basic constraints and the custom constraints.

In some possible implementations, the basic constraints include any one or more of alignment constraints, grid constraints, size constraints, boundary constraints, and fill constraints, and the custom constraints include locking constraints, aspect ratio constraints, Any one or more of the alignment preference constraints.

In some possible implementations, the elements include one or more of graphics, text or decorations.

In some possible implementation manners, the interaction module is specifically used to:

presenting a design interface to a user, the design interface carrying a visual component;

A plurality of elements corresponding to the visualization component are received in response to a user's drag operation on the visualization component.

When the user triggers the smart layout control or the time since the user last entered an element is greater than the preset time, starting from the initial size and initial position of the multiple elements on the canvas, combined with the size of the canvas and design constraints, the Iteratively optimizing the size and position of the plurality of elements on the canvas.

In a third aspect, the present application provides a computer cluster. The computer cluster includes at least one computer, and the at least one computer includes at least one processor and at least one memory. The at least one processor and the at least one memory communicate with each other. The at least one processor is configured to execute instructions stored in the at least one memory, so that the computer cluster executes the element layout method in the first aspect or any implementation manner of the first aspect.

In a fourth aspect, the present application provides a computer-readable storage medium. Instructions are stored in the computer-readable storage medium, and the instructions instruct a cluster of computers to execute the element layout method described in the first aspect or any implementation manner of the first aspect.

In the fifth aspect, the present application provides a computer program product containing instructions. When it is run on a computer cluster, the computer cluster executes the element layout method described in the first aspect or any implementation manner of the first aspect. .

On the basis of the implementation manners provided in the foregoing aspects, the present application may further be combined to provide more implementation manners.

Description of drawings

In order to more clearly illustrate the technical methods of the embodiments of the present application, the following will briefly introduce the drawings required in the embodiments.

FIG. 1A to FIG. 1D are application scene diagrams of the element layout method provided by the embodiment of the present application;

FIG. 2 is a system architecture diagram of an element layout method provided by an embodiment of the present application;

FIG. 3 is a flow chart of an element layout method provided by an embodiment of the present application;

4A to 4H are schematic diagrams of the design constraints provided by the embodiments of the present application;

FIG. 5 is a schematic diagram of an element layout method provided by an embodiment of the present application;

6A to 6C are schematic diagrams of scoring different layouts provided by the embodiment of the present application;

FIG. 7 is a schematic structural diagram of a computer cluster provided by an embodiment of the present application.

Detailed ways

The terms "first" and "second" in the embodiments of the present application are used for description purposes only, and cannot be interpreted as indicating or implying relative importance or implicitly indicating the quantity of indicated technical features. Thus, a feature defined as "first" and "second" may explicitly or implicitly include one or more of these features.

First, some technical terms involved in the embodiments of the present application are introduced.

Element layout refers to setting the size and position of elements on the screen (such as the canvas of the screen application). Among them, the size of the screen or canvas is usually fixed, and the size and position of elements are adjustable. Based on this, the process of element layout is usually a process of continuously adjusting the size and position of elements on the screen or canvas to optimize the presentation effect.

An element is a component that expresses a certain meaning. Wherein, the elements may be of various types, for example, the elements may include different types such as text, graphics, and decorations. Among them, the text can include text or title (the title of the text or the title of the chart), etc., the chart can include bar charts, line charts, pie charts or patterns, and the decoration can include shapes, symbols or backgrounds used to highlight or beautify text and charts .

Automated element layout is possible based on templates and dynamic programming. Specifically, the element layout application can provide multiple layout templates. When the user inputs multiple elements, the element layout application can automatically match the layout templates that match the multiple elements based on dynamic programming, and provide the user with the layout templates that match the multiple elements. Layout templates for the user to select to implement element layout. However, the above method is mainly suitable for text-type elements. The format of elements such as charts is more free, and it is difficult to match the layout template, which leads to poor layout effects based on templates and dynamic programming.

In view of this, the embodiment of the present application provides an element layout method. This method can be performed by the element layout system. In some embodiments, the element layout system may be a software system, and the computer cluster executes the element layout method by running the program code of the software system. In some other embodiments, the element layout system may also be a hardware system for element layout. The embodiment of the present application uses an element layout system as a software system for illustration.

Among them, the element layout problem can be defined as: for a set of elements (such as rectangular elements) that need to be placed on a fixed-size canvas with a specified width and height, find a feasible solution so that all elements do not overlap and do not overflow the canvas. This method can transform the element layout problem into a linear programming problem. Specifically, the linear programming problem is: for a set of input elements (usually multiple elements), solve the recommended position and recommended size of each element, so that each element does not overlap with other elements and does not overflow the canvas.

During specific implementation, the element layout system can receive multiple elements input by the user, determine the initial size and initial position of the multiple elements on the canvas, and then use the initial size and initial position of the multiple elements on the canvas as a starting point, combine The size and design constraints of the canvas, iteratively optimize the size and position of the multiple elements on the canvas, and obtain the recommended size and recommended position of the multiple elements on the canvas, thereby realizing intelligent Element layout.

Since the method uses an iterative optimization method to determine the recommended size and recommended position instead of matching with a fixed layout template, it solves the problem in the related art that the elements of the chart and other types are not applicable and the layout effect is not good. high availability. Moreover, the method realizes driving based on user intent by setting necessary layout attributes such as the position of the upper, lower, left, and right edges of the element, and performing iterative optimization based on the initial size and initial position of the element input by the user. , which improves interactivity and reduces the workload of designers. In addition, this method uses a fixed-size canvas instead of a pixel-discretized canvas, so that continuous decision variables can be used instead of a large number of discrete decision variables, and iterative optimization can be performed in the continuous space provided by continuous decision variables, which can greatly reduce the amount of calculation. The algorithm complexity shortens the time of intelligent layout and improves the efficiency of intelligent layout.

The element layout method in the embodiment of the present application can be adapted to various scenarios. For example, this element layout method can be applied to security situational awareness scenarios. As shown in Figure 1A, the element layout system can rank histograms of access areas, access area maps, line graphs of access trends, histograms of attack area rankings, and line graphs of attack trends. , attack type ranking list and other elements are intelligently arranged to realize comprehensive security situation awareness from micro to macro, and provide decision-making basis for security incident handling. For another example, this element layout method can be applied to trade analysis scenarios. As shown in Figure 1B, the element layout system can analyze the sales statistics, the global quantity distribution of goods exports, the global sales ranking table, the sales situation table, and the global port distribution. Smart layout of map and other elements to realize global economic and trade analysis.

In some possible implementations, the element layout method can also be used in business management scenarios. By accessing the business data of the enterprise, it is possible to present the production situation, product quality and/or product sales of the enterprise, and provide a visual basis for decision-making for enterprise managers. For example, the element layout method can be used to analyze the cockpit sales situation, see Figure 1C, the element layout system can rank column charts for premium regions, popular insurance policies, annual premium column charts and line charts, and compare new and old customers Graphs, real-time transaction tables, annual cumulative total premiums and other elements are intelligently laid out to analyze cockpit sales. For another example, the element layout method can be used to analyze the sales of real estate projects. As shown in Figure 1D, the element layout system can analyze the annual plan chart, sales amount statistical column chart, sales return trend chart, and operating income column chart. Graphs, investment and profitability column charts, inventory line charts, project value indicator charts, inventory indicator column charts, and conventional project indicator charts are intelligently laid out to realize the analysis of real estate project sales.

The above element layout method can be specifically provided to users in the form of data visualization service (data lake visualization, DLV). Wherein, the DLV may be a cloud service. DLV can provide a one-stop data visualization platform, such as the above-mentioned element layout system, which can adapt to various data sources on or off the cloud, and provide a variety of two-dimensional (2dimension, 2D) or three-dimensional (3dimension, 3D) visualization components, users can drag and drop the above visualization components to freely layout, so as to realize efficient customization of screen display content.

In order to make the technical solution of the present application clearer and easier to understand, the system architecture of the embodiment of the present application will be introduced below with reference to the accompanying drawings.

Referring to the system architecture diagram of the element layout method shown in FIG. 2 , the terminal 10 and the terminal 30 are respectively connected to the cloud environment 20 . Wherein, the terminal 10 is a user terminal used to design the presentation content of the terminal 30, such as a desktop computer, a notebook computer, a tablet computer, etc., and the terminal 30 is a device used for presentation content, and is usually equipped with a large screen. The cloud environment 20 indicates a central computing cluster owned by a cloud service provider for providing computing, storage, and communication resources. The central computing cluster is a computer cluster formed by one or more central computing devices (such as a central server).

The element layout system 200 can be deployed on one or more computers in the cloud environment 20 . As shown in Figure 2, the element layout system 200 includes multiple modules such as an interaction module 202, a determination module 204, and an optimization module 206. Different computers on environment 20.

The terminal 10 is deployed with a client (not shown in FIG. 2 ). The client can be a general client with element layout functions, such as a browser, or a dedicated client dedicated to element layout. The client can be connected to the element layout system 200 to load the interaction logic of the interaction module 202 in the element layout system 200, so that the client can present the design interface to the user. Wherein, the design interface may be a graphical user interface (graphical user interface, GUI). Users can interact through the above-mentioned design interface to realize element layout.

Specifically, the design interface carries visual components, which can include different types such as column charts, trend graphs, maps, and statistical tables. Users can trigger drag and drop operations on the visual components through the design interface, and the client responds to the above operations , and report the corresponding event to the element layout system 200 .

The interaction module 202 of the element layout system 200 can receive the above events, so as to obtain a plurality of elements corresponding to the visual components. Among them, multiple elements are determined based on the components dragged by the user and the connected data sources. Then the determination module 204 of the element layout system 200 determines the initial size and initial position of the multiple elements on the canvas, and the optimization module 206 of the element layout system 200 takes the initial size and initial position of the multiple elements on the canvas as a starting point, combined with the The size and design constraints of the canvas, performing iterative optimization on the size and position of the plurality of elements on the canvas, to obtain the recommended size and position of the plurality of elements on the canvas.

The interaction module 202 of the element layout system 200 can also return the recommended size and recommended position of multiple elements on the canvas to the client. When the user confirms to place the multiple elements according to the recommended size and recommended position, the element layout system 200 can send the elements and their recommended size and recommended position to the terminal 30, and the terminal 30 places multiple elements, thereby achieving intelligent element layout.

It should be noted that FIG. 2 is a schematic deployment manner of the element layout system 200 in the embodiment of the present application. In other possible implementation manners of the embodiment of the present application, the element layout system 200 may also be deployed in an edge environment. The edge environment indicates an edge computing cluster that is geographically close to the terminal (that is, the end-side device) and is used to provide computing, storage, and communication resources. An edge computing cluster is a computer cluster formed by one or more edge computing devices (such as edge servers). In some embodiments, the element layout system 200 can also be deployed in a terminal, such as the terminal 10 . The user can directly control the terminal 30 to demonstrate content through the terminal 10 .

The element layout system 200 can be connected to a plurality of terminals 10 and a plurality of terminals 30, and each user can select a terminal 30 for displaying content from a plurality of terminals 30 through a respective terminal 10, and the element layout system 200 can, according to the selection of the user, Send the layout result to the corresponding terminal 30, so that different users can be provided with intelligent layout services.

Next, from the perspective of the element layout system 200, the element layout method provided by the embodiment of the present application will be described in detail with reference to the accompanying drawings.

Referring to the flow chart of the element layout method shown in Figure 3, the method includes:

S302: The element layout system 200 receives a plurality of elements input by the user.

The element layout system 200 can provide a wealth of visual components, such as visual components of chart types such as column charts, line charts, maps, and data tables, or visual components of text types such as title boxes, and visual components of decorative types such as backgrounds. The user can input multiple elements based on the above-mentioned visual components provided by the element layout system 200 .

Specifically, the element layout system 200 can present a design interface to the user, the design interface carries at least one visual component, and then the element layout system 200 receives multiple elements corresponding to the visual component in response to the user's drag operation on the visual component .

Wherein, when the user performs a drag operation on the visual component of the chart type, the element layout system 200 can obtain corresponding elements according to the visual component of the chart type and the data source connected thereto. When the user performs a drag operation on a text-type visual component, the element layout system 200 can obtain corresponding elements according to the text-type visual component and the text content input by the user. For example, when the user performs a drag operation on the title box, the element layout system 200 can obtain title-type elements according to the title box dragged by the user and the text content entered by the user. When the user performs a drag operation on the decoration-type visual component, the element layout system 200 can obtain corresponding elements according to the decoration-type visual component.

In some possible implementation manners, the element layout system 200 may also receive multiple elements input by the user through the element adding control. Wherein, the above-mentioned multiple elements may be added one by one by the user, or may be added by the user in batches. The user can set the size and position of the element to be added through the element adding control. When the element is a rectangular element, the size and position of the element can be determined by the positions of the top, bottom, left, and right edges of the element. In the case of a fixed canvas size, the positions of the top, bottom, left, and right edges can be determined by the top margin (the distance from the top edge of the element to the top edge of the canvas), the bottom margin (the distance from the bottom edge of the element to the bottom edge of the canvas) ), left margin (the distance from the left edge of the element to the left edge of the canvas), and right margin (the distance from the right edge of the element to the right edge of the canvas).

When the element layout system 200 provides multiple types of elements, the user can also set the type of element to be added through the element adding control. For example, it can be a chart, text or decoration. Among them, charts can be further divided into column charts, line charts, maps and other types. Text can be further divided into types such as heading or body text. When the user sets the type of the added element to chart or text, the user can also set the data source accessed by the chart or the text content corresponding to the text through the element adding control.

S304: The element layout system 200 receives user-defined constraints input by the user.

In this embodiment, the element layout system 200 converts the element layout problem into a linear programming problem. Therefore, the element layout system 200 can formulate design constraints, so as to perform linear programming based on the design constraints. Among them, design constraints are usually mathematical constraints. The design constraints may include basic constraints (also referred to as model internal constraints) formulated according to design criteria such as design criteria related to large screens, and may also include user-defined constraints input by the user. Among them, custom constraints can be configured according to user needs. The element layout system 200 can provide various custom constraints, and then receive one or more custom constraints selected by the user.

Wherein, the basic constraints may include any one or more of alignment constraints, grid constraints, size constraints, boundary constraints, and filling constraints. The custom constraints may include any one or more of locking constraints, aspect ratio constraints, and alignment preference constraints. The above-mentioned basic constraints and custom constraints are described in detail below.

(1) Alignment constraints

Alignment constraints are used to ensure alignment of elements. Alignment constraints divide multiple elements input by the user into several alignment groups, such as upper, lower, left, and right alignment groups. Among them, elements aligned in a certain direction belong to an alignment group, and a line in which elements in an alignment group are aligned is called an alignment line.

Refer to formulas (1) to (4) for details. Alignment constraints are used to divide all elements input by the user into several alignment groups such as top, bottom, left, and right, and each edge of an element (such as top edge, bottom edge, left edge or the right edge) respectively correspond to one and only one alignment group.

Among them, e is used to represent the element,

Used to represent collections of elements,

Represents any element in a collection of elements. α, β, γ, and δ denote the left edge, right edge, upper edge, and lower edge, respectively,

Indicates the alignment group corresponding to the left edge of element e,

Indicates the alignment group corresponding to the right edge of element e,

Indicates the alignment group corresponding to the top edge of element e,

Indicates the alignment group corresponding to the bottom edge of element e.

For ease of understanding, this application also provides a specific example. See Figure 4A, the second element of the first row is aligned with the first element of the first row, the third element of the first row, and the second element of the first row is aligned with the third element of the first row Bottom alignment. In addition, the second element in the first row is also left-aligned and right-aligned with the first element in the second row. The four dotted lines shown in Figure 4A are alignment lines corresponding to the above-mentioned alignment group.

(2) Grid constraints

Grid constraints are used to ensure a reasonable division of the canvas area and prevent elements from overlapping. Common design guidelines include visual balance. A classic example of visual balance is symmetrical balance. Symmetrical balance is a common way to organize elements in a stable and regular layout. Grid constraints place elements in place based on symmetrical balance, preventing elements from overlapping.

Specifically, according to the number of elements, the canvas area can be divided into grids including several rows and several columns, and the number of alignment lines can be calculated as a benchmark amount of alignment constraints. Then divide the design by vertically or horizontally dividing the area containing the elements, formulate a segmentation strategy based on pixel coordinates, and adopt a simple layered design style to make the elements visually balanced.

Among them, the calculation of the number of alignment lines can refer to formulas (5) to (8):

align ^count ＝4*grid ^count +2*(grid ^addCol +grid ^addRest ) (5)

grid ^addRest ＝N ^el -grid ^count *grid ^count -grid ^addCol *grid ^count (8)

Among them, N ^el represents the number of elements, grid ^count represents the number of grid line rows, grid ^addCol represents the number of column extensions, grid ^addRest represents the number of remaining elements after column expansion, and align ^count represents the number of alignment lines. When calculating the number of alignment lines, such as formulas (6) to (8), the number of elements can be rooted first, and then rounded down to determine the number of rows of grid lines, and the remaining elements are calculated according to the number of rows Column expansion, and then determine the number of remaining elements after column expansion according to the column expansion number. Then, referring to the above formula (5), the number of alignment lines is determined according to the number of grid lines, the number of column extensions, and the number of remaining elements after column expansion.

For ease of understanding, this application also provides some specific examples. Referring to Figure 4B, when the number of elements is 9, they can be arranged in three rows and three columns, and each element can have four alignment lines of up, down, left, and right. Different elements can include repeated alignment lines, and exclude repeated alignment lines. There are 4*3=12 alignment lines in total. When the number of elements is 13, the root operation is performed first, and then the result of the root operation can be rounded down to determine that the above elements can be arranged in 3 rows, and then expand based on the remaining 5 elements (as shown in the dotted line box in the figure). 1 column, and then according to the number of expanded columns, determine the number of remaining elements after the expanded column is 13-3*3-3*1=1, and calculate the number of alignment lines as 3*4+2*(1+1)= 16.

There can be multiple layouts for a collection of elements that includes the same number of elements. The number of alignment lines can be determined for each layout. The larger the number of alignment lines, the more balanced the element layout.

(3) Size constraints

Size constraints are used to ensure that the size (ie dimensions) of an element is reasonable. Considering that the human eye has certain requirements for the readability of the chart, elements should not be too small during the layout process. In addition, in terms of the overall aesthetics of the content displayed on the screen, enlarged charts can affect the overall harmony of the screen and create a sense of clutter. Therefore, elements should also be avoided from being overly enlarged during the layout process. For this reason, the zoom range of the element can be constrained to ensure that the element is at a reasonable size.

See equations (9) to (12):

in,

identifies the initial width of the element,

Represents the initial height of the element, width ^ca represents the width of the canvas, and height ^ca represents the height of the canvas, that is, the zoom range of the element can be between 2/3 of the initial width (ie

) and the width of the canvas (ie width ^ca ), and the height is between 2/3 of the initial height (ie

) and the height of the canvas (height ^ca ).

For ease of understanding, this application also provides a specific example. As shown in Figure 4C, the initial size of the element is equal to the size of the dark gray area in Figure 4C, and the minimum width and height of the element can be reduced to 2/3 of the width and height of the dark gray area, that is, the smallest element can be as shown in the figure As shown in the light gray area, the width and height of the element can be scaled to the maximum width and height of the canvas, that is, the maximum size of the element can be shown in the black area in the figure.

(4) Boundary constraints

Bound constraints are used to ensure that the layout fills the canvas. Jagged outlines, unbalanced outlines, or any non-convex arrangement can affect aesthetics. Therefore, you can set the overall outline caused by element layout to be close to the outer outline of a rectangle, so that the layout result occupies the entire screen and ensures that the elements fill the canvas.

For the above-mentioned requirements of the rectangular outline, constraints can be made through the upper and lower limits of the alignment group. For example, the boundary constraint makes the element with the smallest coordinate value in the left and upper alignment groups of all elements be at the left and upper boundaries of the canvas, as shown in formulas (13) and (14), the element with the largest coordinate value in the right and lower alignment groups is at The right and bottom boundaries of the canvas, as shown in equations (15) (16). details as follows:

Among them, padding represents the margin. Taking the upper left corner of the canvas as the origin of the coordinate system, the upper edge as the X axis, and the left edge as the Y axis, padding X can represent the set margin along the X axis, and padding Y can represent the set margin along the Y axis. Set margins.

Characterizes the distance from the left edge of the element with the smallest coordinate value (such as the Y-axis coordinate value) in the left-aligned group to the sitting edge of the canvas,

Indicates the distance from the upper edge of the element with the smallest coordinate value (such as the X-axis coordinate value) in the upper alignment group to the upper edge of the canvas. Similarly,

Characterizes the distance from the element with the largest coordinate value (such as the Y-axis coordinate value) in the right-aligned group to the right edge and the left edge of the canvas,

Indicates the distance from the element with the largest coordinate value (such as the X-axis coordinate value) in the lower alignment group to the upper edge of the canvas.

For ease of understanding, this application also provides a specific example. Referring to Figure 4D, among the four elements, two elements in the first row form an upper alignment group, two elements in the second row form a lower alignment group, two elements in the first column form a left alignment group, and two elements in the second column form a right alignment group Group. When arranging the above elements, the element with the smallest coordinate value in the top alignment group and the left alignment group is, for example, the first element in the first row, which can be arranged at a distance of padding X and padding Y from the left and top edges of the canvas, similarly , the element with the largest coordinate value in the bottom alignment group and the right alignment group is, for example, the second element in the second row, which can be arranged at a distance of width ^canvas -paddingX and height ^canvas -paddingY from the left and top edges of the canvas. Thus, it is possible to make all elements fill the canvas.

(5) Filling constraints

Padding constraints are used to ensure that the interior of the layout is filled. In addition to the outline shape of the layout, the internal vacancy of the layout can make the overall space lose the sense of continuity, which in turn affects the aesthetics of the layout to a certain extent. For this reason, you can set the spacing between each element to not exceed the set spacing, so that the interior of the layout result is filled. Wherein, the closer the spacing between elements is, the closer the total area of the elements is to the area of the canvas. Therefore, it is also possible to scale the elements so that the total area of the elements approaches the area of the canvas.

Specifically, when scaling an element, the element may be scaled according to the ratio of the area of the element to the area of the canvas. Considering that the scaling ratio of the area is equal to the scaling ratio of the width multiplied by the scaling ratio of the height, the root operation can be performed on the ratio of the area of the element to the area of the canvas, and then the width and height of the element can be scaled according to the result of the root operation, such as the formula As shown in (17) and (18):

Among them, width ^canvas represents the width of the canvas, height ^canvas represents the height of the canvas, area ^canvas represents the area of the canvas, area ^e represents the area of the element,

Indicates the maximum width of the element,

Indicates the maximum value of the element's height.

Referring to FIG. 4E , when the total area of the scaled elements is close to the area of the canvas, it can be determined that the filling constraint is satisfied. Wherein, whether the total area of the scaled elements is close to the area of the canvas can be measured by the area ratio. For example, the ratio of the total area of the scaled elements to the area of the canvas is greater than or equal to a preset ratio, then it may be determined that the total area of the scaled elements is close to the area of the canvas. The preset ratio can be set according to experience values, for example, it can be set to 99%.

(6) Aspect ratio constraints

Aspect ratio constraints are used to constrain the scaling ratio of one or more elements. That is, when scaling elements, scale one or more elements while maintaining the specified aspect ratio (ratio). Wherein, the aspect ratio may be determined according to the initial height and initial width of the element, for example, may be a ratio of the initial height to the initial width. Referring to Figure 4F, when scaling an element, the height of the scaled element can be determined according to the scaled width and aspect ratio:

height ^′ = width ^′ *ratio (19)

Wherein, width ^' represents the width after scaling, and height ^' represents the height after scaling.

(7) Alignment preference constraints

Alignment preference constraints are used to constrain one or more elements to be top, left, right, or bottom. For example, the element that needs to be on top can be placed on top of other elements, as shown in FIG. 4G . In order to achieve different alignment preferences, the target element can be set above other elements, such as formula (20), or the target element is on the right of other elements, such as formula (21), or the target element is on the left of other elements, such as formula ( 22), or the target element is under other elements, such as formula (23):

Wherein, e1 represents the target element, and e2 represents other elements other than the target element. A target element is an element that the user intends to be top, right, left, or bottom.

is a boolean logic variable. When e1 is above e2, then

The value is 1, otherwise,

The value is 0. Similarly,

Also a boolean logic variable. When e1 is to the right of e2, then

The value is 1, otherwise,

The value is 0. When e1 is to the left of e2,

The value is 1, otherwise

The value is 0. When e1 is below e2, then

The value is 1, otherwise the value is 0.

(8) Lock constraints

Lock constraints are used to lock the size and position of one or more elements. That is to say, it is stipulated that the size and position of one or more elements are consistent with the size and position of the above-mentioned elements. Usually, when inputting an element, a lock field can be added, which is a Boolean variable that can take a value of true or false. When the value is true, the size and position of the corresponding element can be locked.

Referring to FIG. 4H , when an element is locked, the position and size of the element remain unchanged during the smart layout process. That is, the element satisfies the following formula:

L ^lock = X ^lock (24)

B ^lock = Y ^lock (25)

Among them, L ^lock indicates the position of the left boundary of the locked element, which can be expressed specifically by the distance from the left boundary to the Y axis, and B ^lock indicates the position of the lower boundary of the locked element, which can be expressed specifically by the distance from the lower boundary to the X axis, X ^lock and Y ^lock respectively represent the coordinates of the lower left vertex of the locked element,

Width ^lock respectively indicates the minimum, maximum and initial value of the width of the locked element,

The height ^lock indicates the minimum, maximum, and initial values of the height of the locked element, respectively.

The above constraints (1) to (5) are basic constraints, and constraints (6) to (8) are self-defined constraints. In some embodiments, the element layout system 200 may not perform S304, and use basic constraints to perform linear programming, so as to implement element layout.

S306: The element layout system 200 determines the initial size and initial position of the multiple elements on the canvas.

When the user inputs multiple elements by dragging and dropping visual components, the element layout system 200 can determine the initial size of each element by determining the distance from the reference point or side of each of the multiple elements to the edge of the canvas and initial position.

For example, the element layout system 200 can determine the distance from the top and bottom edges of each element to the top edge of the canvas (denoted as the X axis), and the distance from the left and right edges of each element to the left edge of the canvas (denoted as the Y axis). axis) distance. Then, the element layout system 200 can determine the height of the element according to the difference between the upper and lower edges of the element and the upper edge of the canvas, and determine the width of the element according to the difference between the left and right edges of the element and the left edge of the canvas. , so you can get the initial size of multiple elements in the canvas. The element layout system 200 can determine the ordinate of the center point of the element according to the average distance between the upper and lower edges of the element and the upper edge of the canvas, and determine the element according to the average distance between the left and right edges of the element and the left edge of the canvas. The abscissa of the center point of the element, the initial position of the element can be determined based on the abscissa and ordinate of the center point of the element.

When the user inputs multiple elements by using the element adding control, the element layout system 200 can directly determine multiple The element's initial size and position on the canvas.

S308: The element layout system 200 takes the initial size and position of multiple elements on the canvas as a starting point, combines the size of the canvas and design constraints, iteratively optimizes the size and position of the multiple elements on the canvas, and obtains multiple Candidate layout.

Specifically, the layout of an element can be characterized by the size and position of the element, and the element layout system 200 determines an appropriate size and position through an iterative optimization algorithm to realize the element layout. Among them, the element layout system 200 can start from the initial size and initial position of multiple elements on the canvas, combine the size of the canvas and design constraints, such as basic constraints and user-defined constraints (if any), and use an optimizer to Iteratively optimizing the size and position of the multiple elements on the canvas, so as to obtain multiple candidate layouts.

Considering that the Gurobi optimizer has the leading performance in quadratic programming problems such as mixed integer linear programming, convex or non-convex, and the CBC optimizer also has a good optimization effect, the element layout system 200 can be implemented using the above-mentioned Gurobi optimizer or CBC optimizer Iterative optimization to meet different business needs.

In some possible implementation manners, the element layout system 200 may also obtain the type of each element, such as chart, text, decoration and other types. Correspondingly, when the element layout system 200 performs iterative optimization on the size and position of multiple elements on the canvas, iterative optimization may also be performed in combination with element types. For example, the element layout system 200 can group elements based on element types, bind elements belonging to the same group, and then perform iterative optimization in units of bound element groups. For example, the element layout system 200 can bind a chart type element with a text type element (for example, the title of a chart), and then treat the bound element groups as one element for iterative optimization.

Referring to the schematic diagram of the intelligent layout shown in FIG. 5 , the element layout system 200 uses a generative model, such as the core mixed-integer linear programming (MILP) generative model, to generate candidate layouts during the layout generation phase. Among them, the size of the generative model usually depends on the number of elements. The number of decision variables and constraints in the generative model is independent of the canvas and other factors.

The core MILP formulation (not shown in Figure 5) and objective function are defined in the generative model. The core MILP formulas are used to generate an appropriate layout grid skeleton to ensure that elements are within allowed position and size constraints, do not overlap each other, and do not overflow the canvas. Among them, the generation model defines the origin of the coordinate system as the upper left corner of the canvas, the positive direction of the X-axis is horizontally to the right, and the positive direction of the Y-axis is vertically downward. Continuous decision variables Le, Re, Te, and Be are defined in the generative model to indicate the position of the left, right, upper, and lower edges of a single element e, through which the appropriate element size can be ensured and element overflow can be prevented. Design constraints such as alignment constraints and padding constraints can be used as the objective function of the generative model (also called additional constraints) to optimize the layout grid skeleton and generate candidate layouts with aesthetic effects such as alignment.

S310: The element layout system 200 sorts the multiple candidate layouts to obtain a layout result.

The element layout system 200 can sort multiple candidate layouts from different dimensions, so as to select a candidate layout with better performance as the layout result. For example, the element layout system 200 may, according to at least one of the size balance of each of the multiple elements, the balance of the picture presented by the multiple elements on the canvas, and the alignment of the multiple elements, Ranking the plurality of candidate layouts.

In some possible implementation manners, the element layout system 200 may score the balance of the size of each of the multiple elements, the balance of the picture presented by the multiple elements on the canvas, and the alignment of the multiple elements, The plurality of candidate layouts are ranked according to the corresponding scores. Referring to FIG. 5 , the element layout system 200 adopts a sorting model to evaluate the size quality Q _s (the balance degree of the size), the balance quality Q _b (the balance degree of the picture presented by multiple elements on the canvas), and the alignment quality Q _a (the balance degree of the multiple elements Alignment degree) is scored to obtain the size balance score Score _size , balance score (importance score Score _importance ) and alignment score Score _align . The element layout system 200 can calculate the total score for each candidate layout as follows:

Score＝Score _size +Score _align +Score _importance (30)

The element layout system 200 can sort multiple candidate layouts based on the above total score to obtain a layout result. Wherein, the layout result may be the top candidate layout among the plurality of candidate layouts, for example, it may be the candidate layout whose total score is ranked top3. In the example in FIG. 5 , the layout result may be a candidate layout sorted as top1, such as candidate layout 1. The layout result includes recommended sizes and recommended positions of elements on the canvas. Wherein, the recommended size and recommended position may be the size and position of elements in the above-mentioned top-ranked candidate layouts.

Next, how to determine the size balance score Score _size , the alignment score Score _align , and the importance score Score _importance will be described.

The Score _size can be determined based on the sum of the normalized width and normalized height of each element. Please refer to the formula (31) for the specific calculation method:

Among them, width ^e and height ^e are used to represent the width and height of element e, and width ^ca and height ^ca are used to represent the width and height of the canvas. Correspondingly,

Used to characterize the normalized width and normalized height of element e. N ^el is used to characterize the number of elements.

Referring to the schematic diagram of the size balance scores of different layouts shown in Figure 6A, the element size difference in the left layout is greater than the element size difference in the right layout, and the size balance score corresponding to the left layout is lower than the size balance score corresponding to the right layout. Among them, the higher the size balance score, the more balanced the size of each element in the layout.

The alignment score Score _alig can be determined according to the top, bottom, left, and right alignments between every two elements. If two elements are aligned, then

Take 1, similarly, if the two elements are bottom-aligned, left-aligned, and right-aligned, then

Take 1. The alignment score Score _ali of a candidate layout can be referred to the following formula:

Referring to the schematic diagram of alignment scores of different layouts shown in FIG. 6B , the number of element alignments in the left layout is smaller than that of the right layout, and correspondingly, the alignment score of the left layout is smaller than that of the right layout.

The importance score Score _importance may be determined according to the area of the element and the distance from the element to the center point of the canvas. For example, the importance score for a candidate layout may be determined based on the area of each element and the distance of each element from the center point of the canvas. Please refer to the formula (33) for the specific calculation method:

Among them, important_weight ^e represents the importance coefficient of the element, area ^e represents the area of the element, x ^e represents the x coordinate of the element, y ^e represents the y coordinate of the element, width ^e and height ^e represent the width and height of the element respectively, width ^canvas , height ^canvas indicates the width and height of the canvas respectively.

Referring to Figure 6C, the larger the area ratio of an element and the closer it is to the center of the canvas, the higher the importance of the element, and vice versa. Based on this, the importance score of the element in the third column and the first row in the layout on the left in FIG. 6C is smaller than the importance score of the element in the second column in the layout on the right.

The above example is illustrated by using the element layout system 200 to sort based on the sum of the three scores, and then determine the layout result according to the sorting result. In some possible implementation manners, sorting may also be performed based on the weighted average of the three scores or one or two of the three scores, and the layout result is determined according to the sorting result.

It should be noted that, the above S308 to S310 are the element layout system 200 in the embodiment of the present application, starting from the initial size and initial position of the multiple elements on the canvas, combining the size of the canvas and design constraints, Iteratively optimizes the size and position of each element on the canvas to obtain a layout result. In some possible implementations, the element layout system 200 may not perform operations such as sorting. For example, the element layout system 200 can generate a candidate layout and use the candidate layout as a layout result.

Moreover, the element layout system 200 can set various trigger conditions to trigger the process of iterative optimization and layout generation. For example, the element layout system 200 may provide an intelligent layout control, and the trigger condition may include that the user triggers the above-mentioned intelligent layout control. For another example, the element layout system 200 is set with a pre-review time, and the trigger condition may include that the time since the user last entered an element is greater than the preset time. That is to say, the element layout system 200 may start from the initial size and initial position of the multiple elements on the canvas when the user triggers the smart layout control, or when the time since the user last entered an element is greater than the preset time, in combination with the The size and design constraints of the canvas, iteratively optimize the size and position of the multiple elements on the canvas.

Based on the above content description, the embodiment of the present application provides an element layout method. The method adopts an iterative optimization method to determine the recommended size and recommended position, rather than matching with a fixed layout template, thus solving the problems in related technologies that are not suitable for elements such as charts and poor layout effects, and has high usability . Furthermore, during iterative optimization, multiple candidate layouts can be generated and sorted based on the size balance score, alignment score and/or importance score to obtain the layout result, which can realize the intelligent layout driven by user intent .

In the scenario where the presentation data chart is the main one, one chart (also called the main chart) can be used as the visual focus through the above method, and the rest of the visual charts can be used to assist the main chart to complete the narration of the data fact. In this way, it can be obtained Better presentation effect.

The embodiments of the present application also design relevant verification experiments to verify the demonstration effect. Taking into account the age, gender, culture, professional background and other factors of different personnel, 20 people were selected for the test in this verification experiment. Among them, the 20 people who conducted the test were specifically 15 visualization professionals and 5 technical experts.

The verification experiment uses the T-test test method to analyze the test data. T-test is a statistical test method widely used to compare the means of two groups of samples. It uses the T distribution theory to infer the probability of the difference, so as to determine whether the difference between the two means is significant. It is mainly used for normal distribution with small sample size (eg n<30) and unknown population standard deviation σ.

Specifically, given the same element, use the element layout method of the embodiment of the present application and other element layout methods (such as element layout methods based on templates and dynamic programming) to generate different layouts, assuming layout 1 and layout 2. Then, 20 testers can score the above layout 1 and layout 2 respectively. For Layout 1 and Layout 2, T-test is used to determine whether the difference between the mean scores of 20 testers is significant.

T-test results show that the demonstration effect of the element layout method adopted in the embodiment of the present application is significantly better than that of the element layout method based on templates and dynamic programming (confidence degree >99%). The overall effect of the layout generated by the element layout method in the embodiment of the present application is better, the size of the elements is reasonable, and the canvas is better filled.

The element layout method provided by the embodiment of the present application has been described in detail above with reference to FIG. 1A to FIG. 6C , and the element layout system 200 provided by the embodiment of the present application will be introduced below with reference to the accompanying drawings.

Referring to the structural diagram of the element layout system shown in FIG. 2, the system 200 includes:

An interaction module 202, configured to receive multiple elements input by the user;

A determining module 204, configured to determine the initial size and initial position of the plurality of elements on the canvas;

An optimization module 206, configured to iteratively optimize the size and position of the multiple elements on the canvas, taking the initial size and initial position of the multiple elements on the canvas as a starting point, in combination with the size of the canvas and design constraints , obtaining a layout result, where the layout result includes recommended sizes and recommended positions of the plurality of elements on the canvas.

In some possible implementation manners, the optimization module 206 is specifically configured to:

In some possible implementations, the interaction module 202 is also configured to:

Receive custom constraints entered by the user;

The design constraints include basic constraints and the custom constraints.

In some possible implementation manners, the interaction module 202 is specifically configured to:

The element layout system 200 according to the embodiment of the present application may correspond to the implementation of the method described in the embodiment of the present application, and the above-mentioned and other operations and/or functions of the various modules/units of the element layout system 200 are respectively in order to realize the implementation shown in FIG. 2 For the sake of brevity, the corresponding flow of each method in the example is not repeated here.

The embodiment of the present application also provides a computer cluster. The computer cluster includes at least one computer, and any computer in the at least one computer may be from a cloud environment or an edge environment, or may be a terminal device. The computer cluster is specifically used to implement the functions of the element layout system 200 in the embodiment shown in FIG. 2 .

FIG. 7 provides a schematic structural diagram of a computer cluster. As shown in FIG. 7 , the computer cluster 70 includes multiple computers 700 , and the computers 700 include a bus 701 , a processor 702 , a communication interface 703 and a memory 704 . The processor 702 , the memory 704 and the communication interface 703 communicate through the bus 701 .

The bus 701 may be a peripheral component interconnect standard (peripheral component interconnect, PCI) bus or an extended industry standard architecture (extended industry standard architecture, EISA) bus, etc. The bus can be divided into address bus, data bus, control bus and so on. For ease of representation, only one thick line is used in FIG. 7 , but it does not mean that there is only one bus or one type of bus.

The processor 702 may be a central processing unit (central processing unit, CPU), a graphics processing unit (graphics processing unit, GPU), a microprocessor (micro processor, MP) or a digital signal processor (digital signal processor, DSP) etc. Any one or more of them.

The communication interface 703 is used for communicating with the outside. For example, the communication interface 703 is used to receive multiple elements input by the user, output layout results, and so on.

The memory 704 may include a volatile memory (volatile memory), such as a random access memory (random access memory, RAM). Memory 704 can also include non-volatile memory (non-volatile memory), such as read-only memory (read-only memory, ROM), flash memory, hard disk drive (hard disk drive, HDD) or solid state drive (solid state drive) , SSD).

Computer-readable instructions are stored in the memory 704 , and the processor 702 executes the computer-readable instructions, so that the computer cluster 70 executes the aforementioned element layout method (or implements the aforementioned element layout system 200 functions).

Specifically, in the case of realizing the embodiment of the system shown in FIG. 2 , and the functions of the various modules of the element layout system 200 described in FIG. 2 such as the interaction module 202, the determination module 204 and the optimization module 206 are realized by software In some cases, software or program codes required to execute the functions of the modules in FIG. 2 may be stored in at least one memory 704 in the computer cluster 70 . At least one processor 702 executes the program code stored in the memory 704, so that the computer cluster 70 executes the aforementioned element layout method.

The embodiment of the present application also provides a computer-readable storage medium. The computer-readable storage medium may be any available medium that can be stored by a computer cluster or a data storage device such as a data center including one or more available media. The available media may be magnetic media (eg, floppy disk, hard disk, magnetic tape), optical media (eg, DVD), or semiconductor media (eg, solid state hard disk), etc. The computer-readable storage medium includes instructions, and the instructions instruct a cluster of computers to execute the above element layout method.

The embodiment of the present application also provides a computer program product. The computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on the computer cluster, the processes or functions according to the embodiments of the present application are generated in whole or in part. The computer instructions may be stored in or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, from a website, computing device, or data center via Wired (eg, coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (eg, infrared, wireless, microwave, etc.) transmission to another website site, computing device, or data center. The computer program product may be a software installation package, which may be downloaded and executed on a computer cluster if any of the aforementioned element layout methods needs to be used.

The description of the process or structure corresponding to each of the above drawings has its own emphasis. For the part that is not described in detail in a certain process or structure, you can refer to the relevant description of other processes or structures.

Claims

An element layout method, characterized in that the method comprises:

Multiple elements that receive user input;

determining the initial size and initial position of the plurality of elements on the canvas;

Taking the initial size and position of the plurality of elements on the canvas as a starting point, combined with the size and design constraints of the canvas, iteratively optimizes the size and position of the plurality of elements on the canvas to obtain a layout result. The layout result includes recommended sizes and recommended positions of the plurality of elements on the canvas.
The method according to claim 1, wherein the iterative optimization of the size and position of the plurality of elements on the canvas to obtain a layout result includes:

Iteratively optimizing the size and position of the multiple elements on the canvas to obtain multiple candidate layouts, each of the multiple candidate layouts includes the size and position of the multiple elements on the canvas ;

Sorting the multiple candidate layouts to obtain a layout result, where the layout result is a top-ranked candidate layout among the multiple candidate layouts.
The method according to claim 2, wherein said sorting said plurality of candidate layouts comprises:

According to at least one of the degree of balance of the size of each of the plurality of elements, the degree of balance of the images presented by the plurality of elements on the canvas, and the degree of alignment of the plurality of elements, the plurality of candidate layouts Sort.
The method according to claim 3, wherein the balance degree of the picture presented by the plurality of elements on the canvas is characterized by the sum of the importance scores of each element, and the importance score is based on the area of the elements and The distance from the element to the center point of the canvas is determined.
The method according to any one of claims 1 to 4, wherein the method further comprises:

Receive custom constraints entered by the user;

The design constraints include basic constraints and the custom constraints.
The method according to claim 5, wherein the basic constraints include any one or more of alignment constraints, grid constraints, size constraints, boundary constraints, and filling constraints, and the custom constraints include locking constraints Any one or more of , aspect ratio constraints, and alignment preference constraints.
The method according to any one of claims 1 to 6, wherein the elements include one or more of diagrams, texts or decorations.
The method according to any one of claims 1 to 7, wherein the multiple elements input by the user include:

presenting a design interface to a user, the design interface carrying a visual component;

A plurality of elements corresponding to the visualization component are received in response to a user's drag operation on the visualization component.
The method according to any one of claims 1 to 8, characterized in that, starting from the initial size and initial position of the plurality of elements on the canvas, combined with the size of the canvas and design constraints, the Multiple elements are iteratively optimized in size and position on the canvas, including:

When the user triggers the smart layout control or the time since the user last entered an element is greater than the preset time, starting from the initial size and initial position of the multiple elements on the canvas, combined with the size of the canvas and design constraints, the Iteratively optimizing the size and position of the plurality of elements on the canvas.
An element layout system, characterized in that the system includes:

An interactive module for receiving multiple elements input by the user;

A determining module, configured to determine the initial size and initial position of the plurality of elements on the canvas;

An optimization module, configured to iteratively optimize the size and position of the multiple elements on the canvas, taking the initial size and initial position of the multiple elements on the canvas as a starting point, in combination with the size of the canvas and design constraints, A layout result is obtained, where the layout result includes recommended sizes and recommended positions of the plurality of elements on the canvas.
The system according to claim 10, wherein the optimization module is specifically used for:

Iteratively optimizing the size and position of the multiple elements on the canvas to obtain multiple candidate layouts, each of the multiple candidate layouts includes the size and position of the multiple elements on the canvas ;

Sorting the multiple candidate layouts to obtain a layout result, where the layout result is a top-ranked candidate layout among the multiple candidate layouts.
The system according to claim 11, wherein the optimization module is specifically used for:

According to at least one of the degree of balance of the size of each of the plurality of elements, the degree of balance of the images presented by the plurality of elements on the canvas, and the degree of alignment of the plurality of elements, the plurality of candidate layouts Sort.
The system according to claim 12, wherein the balance degree of the picture presented by the plurality of elements on the canvas is represented by the sum of the importance scores of each element, and the importance score is based on the area of the elements and The distance from the element to the center point of the canvas is determined.
The system according to any one of claims 10 to 13, wherein the interaction module is also used for:

Receive custom constraints entered by the user;

The design constraints include basic constraints and the custom constraints.
The system according to claim 14, wherein the basic constraints include any one or more of alignment constraints, grid constraints, size constraints, boundary constraints, and filling constraints, and the custom constraints include locking constraints Any one or more of , aspect ratio constraints, and alignment preference constraints.
The system according to any one of claims 10 to 15, wherein the element comprises one or more of graphics, text or decoration.
The system according to any one of claims 10 to 16, wherein the interaction module is specifically used for:

presenting a design interface to a user, the design interface carrying a visual component;

A plurality of elements corresponding to the visualization component are received in response to a user's drag operation on the visualization component.
The system according to any one of claims 10 to 17, wherein the optimization module is specifically used for:

When the user triggers the smart layout control or the time since the user last entered an element is greater than the preset time, starting from the initial size and initial position of the multiple elements on the canvas, combined with the size of the canvas and design constraints, the Iteratively optimizing the size and position of the plurality of elements on the canvas.
A computer cluster, characterized in that the computer cluster includes at least one computer, the at least one computer includes at least one processor and at least one memory, and computer-readable instructions are stored in the at least one memory, the At least one processor executes the computer-readable instructions, causing the computer cluster to perform the method according to any one of claims 1-9.
A computer-readable storage medium, characterized in that it includes computer-readable instructions, and when the computer-readable instructions are run on a computer cluster, the computer cluster executes the method described in any one of claims 1 to 9. method.
A computer program product, characterized by comprising computer-readable instructions, which, when the computer-readable instructions are run on a computer cluster, cause the computer cluster to execute the method according to any one of claims 1 to 9.