WO2023093390A1

WO2023093390A1 - Layout method, readable medium and electronic device

Info

Publication number: WO2023093390A1
Application number: PCT/CN2022/126443
Authority: WO
Inventors: 刘梦迪; 张警吁; 乔韩
Original assignee: 华为技术有限公司
Priority date: 2021-11-24
Filing date: 2022-10-20
Publication date: 2023-06-01
Also published as: CN116166155A

Abstract

The present application relates to the technical field of electronics. Disclosed are a layout method, a readable medium and an electronic device. The method comprises: acquiring layout features of a plurality of elements to be laid out; on the basis of the layout features of said elements, determining a layout feature distance between every two of said elements, wherein the higher the similarity between the layout features of said elements, the smaller the layout feature distance between said elements; determining a layout scheme on the basis of the similarity between the layout feature distance between every two of said elements and a spatial distance between every two of said elements; and adjusting the positions of said elements according to the determined layout scheme. The consistency of the principle of similarity and the principle of proximity in Gestalt theory embodied by an interface that has been laid out by means of a layout scheme determined by the method provided in the present application is relatively high, such that the visual complexity of a display interface of an electronic device laid out by means of the layout scheme can be reduced, thereby improving the interface simplicity of the display interface, and improving the user experience.

Description

Layout method, readable medium and electronic device

This application claims priority to a Chinese patent application with application number 202111405667.6 and titled "Layout Method, Readable Medium, and Electronic Device" filed with the China Patent Office on November 24, 2021, the entire contents of which are incorporated herein by reference Applying.

technical field

The present application relates to the field of electronic technology, in particular to a layout method, a readable medium and electronic equipment.

Background technique

Electronic devices usually display multiple display elements on the same interface, for example, displaying icons of multiple application programs on the desktop or menu bar of the electronic device. In order to improve the aesthetics of the interface, reduce the visual complexity of the interface, etc., the user usually manually arranges multiple display elements in a certain order, for example, according to the similarity of colors and the category of each display element. However, in the case of a large number of display elements, it is cumbersome for the user to manually arrange the multiple display elements, which affects user experience. For example, referring to Figure 1, if the user wants to move multiple application program icons in the interface displayed by the electronic device to be arranged according to the color similarity of each icon, it is necessary to move the application program icons one by one, the operation is cumbersome, and it is not easy to determine the color similarity The arrangement order of taller application icons affects user experience.

Contents of the invention

In view of this, the embodiments of the present application provide a layout method, a readable medium and an electronic device. By obtaining the layout features of the elements to be laid out, determine the distance between the layout features of the elements to be laid out, and according to the similarity between the layout feature distance and the space distance between the two elements to be laid out layout. In this way, the electronic device can automatically layout the layout elements, and the interface after layout has low visual complexity and high simplicity, which improves user experience.

In the first aspect, the embodiment of the present application provides a layout method, which is applied to electronic devices, and the method includes: obtaining the layout features of multiple elements to be laid out; The layout feature distance between the elements to be laid out, wherein the higher the similarity of the layout features between the elements to be laid out, the smaller the layout feature distance between the elements to be laid out; According to the similarity between the two spatial distances, the layout scheme is determined; according to the determined layout scheme, the position of each element to be laid out is adjusted.

In the embodiment of the present application, the electronic device may automatically layout the elements to be laid out according to at least one layout feature among the elements to be laid out, which improves user experience. In addition, due to the high similarity between the spatial distance between the elements to be laid out in the determined layout scheme and the layout feature distance between the elements to be laid out, the interface after layout according to the layout scheme reflects the above-mentioned Gestalt theory. The consistency of similarity principle and proximity principle is high, which makes the visual complexity of the interface after the layout low and the interface simple, which improves the user experience.

In a possible implementation of the foregoing first aspect, the foregoing layout features include at least one of the following features: color features and semantic features.

In a possible implementation of the above first aspect, when the layout feature is a color feature, the above-mentioned determination of the layout feature distance between each element to be laid out includes: according to the layout feature of each element to be laid out, determine the The characteristic color of the layout element; the color distance between the characteristic colors of each element to be laid out is determined as the layout characteristic distance between the two elements to be laid out; wherein, when the main color of the element to be laid out is a single In this case, the characteristic color of the element to be laid out is the main color of the element to be laid out; when there are multiple main colors of the element to be laid out, the characteristic color of the element to be laid out is the average value of the main colors of the elements to be laid out.

That is, in the embodiment of the present application, the layout characteristic distance is the color distance between two characteristic colors of the elements to be laid out. Wherein, the color distance may be calculated according to the color distance calculation method mentioned below, or other methods may be used for calculation.

In a possible implementation of the above-mentioned first aspect, the color distance between the feature colors of the above-mentioned elements to be laid out includes any one of the following distances: Euclidean distance, Manhattan distance, Chebyshev distance, Minkowski distance, normalized Euclidean distance, Mahalanobis distance, angle cosine, Hamming distance.

In a possible implementation of the above first aspect, when the layout feature is a color feature, the above-mentioned determination of the layout feature distance between each element to be laid out includes: according to the layout feature of each element to be laid out, determine the The characteristic color of the layout element; determine the layout characteristic distance between each pair of layout elements whose characteristic color belongs to the same hue as 0, and determine the layout characteristic distance between each pair of layout elements whose characteristic color belongs to different hues as 1; Among them, when the main color of the element to be laid out is single, the feature color of the element to be laid out is the main color of the element to be laid out; in the case of multiple main colors of the element to be laid out, the feature color of the element to be laid out is The average of multiple dominant colors for the element to be laid out. .

In this embodiment of the application, the layout feature distance between two elements to be laid out can be determined according to the hue of the characteristic color of each element to be laid out, that is, the distance between two elements to be laid out whose feature color belongs to the same hue The layout feature distance is determined to be 0, and the layout feature distance between two to-be-layout elements whose feature colors belong to different hues is determined to be 1. In this way, the calculation amount of the electronic device in the process of determining the layout scheme based on the similarity between the layout feature distance between the two elements to be laid out and the space distance between the two elements to be laid out can be reduced, and the layout speed can be improved.

In a possible implementation of the first aspect above, when the layout feature is a semantic feature, the above-mentioned determination of the layout feature distance between each element to be laid out includes: according to the semantics of each element to be laid out Classify; determine the layout feature distance between two layout elements classified into the same category as 0, and determine the layout feature distance between two layout elements of different categories as 1.

In a possible implementation of the above-mentioned first aspect, the above-mentioned determination of the layout scheme based on the similarity between the layout feature distance between each pair of elements to be laid out and the spatial distance between each pair of elements to be laid out includes: The layout scheme with the largest similarity between the layout feature distance between two elements to be laid out and the space distance between two elements to be laid out.

In a possible implementation of the above-mentioned first aspect, the above-mentioned determination of the layout scheme based on the similarity between the layout feature distance between each pair of elements to be laid out and the spatial distance between each pair of elements to be laid out includes: A layout scheme in which the similarity between the layout feature distance between the elements to be laid out and the space distance between the elements to be laid out is greater than the similarity threshold.

In a possible implementation of the above-mentioned first aspect, the layout between two elements to be laid out is determined by the correlation coefficient between the layout characteristic distance between two elements to be laid out and the space distance between two elements to be laid out The similarity between the feature distance and the spatial distance between each element to be laid out.

That is, in the embodiment of the present application, the similarity between the layout feature distance between two elements to be laid out and the space distance between each element to be laid out is based on the layout feature distance between two elements to be laid out and each The correlation coefficient between the spatial distances between two layout elements is determined, and the larger the correlation coefficient, the higher the similarity.

In a possible implementation of the first aspect above, the correlation coefficient includes any one of the following correlation coefficients: Pearson correlation coefficient, Spearman correlation coefficient, and Kendall correlation coefficient.

In a possible implementation of the foregoing first aspect, the foregoing elements to be laid out include any one of the following elements: icons, pictures, and service cards.

That is, in this embodiment of the application, the elements to be laid out may include icons, pictures, service cards, and the like. For example, the application icon on the desktop of the electronic device, the icon in the pull-down menu of the electronic device, the service card of the application program on the desktop of the electronic device, etc., are not limited here.

In a possible implementation of the foregoing first aspect, the electronic device acquires layout features of multiple elements to be laid out when detecting a layout instruction input by a user on the electronic device.

For example, when the electronic device detects that the user is turning on the automatic layout function of the electronic device or performs an automatic layout gesture or operation in the electronic device, the above first aspect and various possible implementations of the above first aspect provide The layout method automatically layouts display elements in an electronic device.

In the second aspect, the embodiment of the present application provides a readable medium, on which instructions are stored, and when the instructions are executed on the electronic device, the electronic device realizes the above-mentioned first aspect and various possible implementations of the above-mentioned first aspect provide Any one of the layout methods.

In a third aspect, an embodiment of the present application provides an electronic device, which includes: a memory for storing instructions executed by one or more processors of the electronic device; and a processor, which is one of the processors of the electronic device , for executing the instructions stored in the memory to implement the above first aspect and any one of the layout methods provided by various possible implementations of the above first aspect.

Description of drawings

Fig. 1 shows a schematic diagram of a layout of application program icons according to some embodiments of the present application;

2A to 2F are schematic diagrams showing different basic principles of Gestalt theory according to some embodiments of the present application;

3A and 3B are schematic diagrams showing icons arranged in different orders according to some embodiments of the present application;

Fig. 3C is a schematic diagram showing a visual complexity comparison of the icons arranged in Fig. 3A and Fig. 3B according to some embodiments of the present application;

4A to 4D show schematic diagrams of four different color spaces according to some embodiments of the present application;

Fig. 5 shows a schematic flowchart of a layout method according to some embodiments of the present application;

6A to 6C show a schematic diagram of a display interface of the mobile phone 100 during a process in which the user triggers the layout command of the mobile phone 100 according to some embodiments of the present application;

Fig. 7 shows a schematic diagram of the color distance of each icon in the display interface shown in Fig. 1 (A) according to some embodiments of the present application;

FIG. 8A and FIG. 8B are schematic diagrams showing spatial distances between different icons according to some embodiments of the present application;

Fig. 9 shows a schematic diagram of the spatial distance of each icon in the display interface shown in Fig. 1 (A) according to some embodiments of the present application;

Fig. 10A shows a schematic diagram of a layout scheme expressed in matrix form according to some embodiments of the present application;

Fig. 10B shows a schematic diagram of layout positions in the display interface shown in Fig. 1 (A) according to some embodiments of the present application;

Fig. 11 shows a schematic diagram of a drop-down menu interface of a mobile phone 100 according to some embodiments of the present application;

Fig. 12 shows a schematic flowchart of a layout method according to some embodiments of the present application;

Fig. 13 shows a schematic diagram of the display interface of the smart icon layout function of the pull-down menu interface of the account opening mobile phone 100 according to some embodiments of the present application;

Fig. 14 shows a schematic diagram of a semantic clustering result of icons in the drop-down menu interface shown in Fig. 11 according to some embodiments of the present application;

Fig. 15 shows a schematic diagram of the effect of rearranging the icons in the pull-down menu interface shown in Fig. 11 according to some embodiments of the present application;

Fig. 16 shows a schematic flowchart of a layout evaluation method according to some embodiments of the present application;

Fig. 17 shows a schematic diagram of a layout evaluation scene according to some embodiments of the present application;

Fig. 18 shows a schematic diagram of evaluation results of a variety of different layouts according to some embodiments of the present application;

Fig. 19 shows a schematic structural diagram of a mobile phone 100 according to some embodiments of the present application;

Fig. 20 shows a schematic structural diagram of a mobile phone 100 according to some embodiments of the present application.

Detailed ways

Illustrative embodiments of the present application include, but are not limited to, layout methods, readable media, and electronic devices.

In order to facilitate the understanding of the technical solutions of the embodiments of the present application, firstly, the theories and terms involved in the embodiments of the present application are introduced.

(1) Gestalt theory

Gestalt is a psychological theory that emphasizes the integrity of experience and behavior, emphasizes the overall role of structure and the connection between the components that produce perception, and believes that the whole is not equal to the sum of the parts. When a person observes multiple objects, he does not distinguish the individual components of an image at the beginning, but combines the various parts to make it a more understandable unity, that is, for multiple Goals are grouped, and the following principles are mainly followed for grouping goals based on Gestalt theory with reference to the accompanying drawings.

Proximity principle: Among the multiple targets to be grouped, the targets that are close to each other are regarded as a group. For example, FIG. 2A shows a schematic diagram of a proximity principle. With reference to FIG. 2A, multiple circles in the rectangular area 10 are close to each other. Based on the principle of proximity, the white circles in the rectangular area 10 can be considered as a group, square The white circles in area 11 form a group, and the white circles in square area 13 form a group.

Similarity principle: treat multiple targets to be grouped, and treat targets with similarities as a group, such as shape similarity, color similarity, shadow similarity, semantic similarity, etc. For example, FIG. 2B shows a schematic diagram of a similarity principle. Referring to FIG. 2B , based on the similarity principle, it can be considered that a row with all white circles is a group, and a row with all black circles is a group.

Closure principle: Among the objects to be grouped, the object is regarded as a complete object, and the gaps that may be included between the objects are not paid attention to. For example, Fig. 2C shows a schematic diagram of a principle of closure. With reference to Fig. 2C, the illustrated multi-segment arc is a part of the circle 14, and it can be considered that the illustrated three-segment arc is a group, similarly, a plurality of line segments All of the polylines/polylines are part of the rectangle 15, so the five line segments/polylines in the illustration can be considered as a group.

The principle of symmetry: in the objects to be grouped, if there are two non-adjacent objects that are symmetrical to each other, the two objects can be considered as a group. For example, FIG. 2D shows a schematic diagram of a symmetry principle. Referring to FIG. 2D, "[" and "]" in the figure are a group, "{" and "}" are a group, and "【" and "] " as a group.

The principle of common fate: Among the targets to be grouped, multiple targets with the same movement tendency are considered as a group. For example, FIG. 2E shows a schematic diagram of a common fate principle. With reference to FIG. 2E , the objects in the illustrated area 16 have the same movement tendency, such as moving to the left, and the objects in other parts have different movement patterns, then The objects in the area 16 can be considered as a group.

Continuity principle: In the target to be grouped, if multiple targets can be regarded as connected together, then the object composed of multiple targets is regarded as a complete whole. For example, Fig. 2F shows a schematic diagram of a continuity principle, with reference to Fig. 2F, although the middle part of the key pattern 18 in the figure is blocked by the key pattern 17, it is still considered that the key pattern 18 is a complete key pattern, rather than In the middle, there is a lack of patterns covered by the key pattern 17 .

(2) Interface (page) simplicity

Interface simplicity means that the interface can be perceived by users as visual complexity. The higher the visual complexity, the lower the interface simplicity. Visual complexity is affected by the amount of information, the quality of information, and the organization of information (eg, the way elements are laid out in an interface). The amount of information and the quality of information are usually fixed in an interface, and the visual complexity can be optimized by adjusting the interface organization, such as adjusting the position of each layout element in the interface, so as to improve the simplicity of the interface.

It can be understood that the layout of the interface often embodies multiple principles of the above-mentioned Gestalt theory. When the consistency among the principles is high, the visual complexity of the interface will be low; conversely, when the consistency among multiple principles embodied in the interface is low, the visual complexity of the interface will be high.

For example, referring to Figure 3A, Figure 3B and Figure 3C, the icons in Figure 3A are randomly arranged, the color distribution is messy, the consistency of the proximity principle and the similarity principle is low, the visual complexity is high, and the interface simplicity is low; The icons in Figure 3B are laid out according to the proximity of their colors. Icons with similar distances have similar colors. The consistency between the proximity principle and the similarity principle is high, the visual complexity is low, and the interface is simple.

(3) Color space

A color space (color model) is an abstract mathematical model that describes the way a color is represented using a set of values (usually using three or four values or color components). Common color spaces include but are not limited to RGB color space, HSV color space, CIE (International Commission on Illumination, CIE for short)-XYZ color space, CIE-Lab color space, etc. Exemplarily, FIG. 4A to FIG. 4D show schematic diagrams of four common color spaces.

Referring to Figure 4A, the RGB color space uses three values R, G, and B to represent a color, where R represents the intensity of red (Red) in a color, G (Green) represents the intensity of green in a color, and B (Blue) Represents the intensity of blue in a color. For example, in some embodiments, 0 to 255 can be used to represent the intensity of R, G, and B in a color. Referring to FIG. 4A, (0, 0, 255) represents blue, and (255, 255, 255) represents white. , (0, 255, 0) means green, (255, 0, 0) means red.

Referring to Figure 4B, the HSV color space uses three values H, S, and V to represent a color, where H (Hue, hue) is measured by angle, and the value range is 0° to 360°, starting from red in a counterclockwise direction For calculation, red is 0°, green is 120°, and blue is 240°; S (Saturation, saturation) indicates the degree to which the color is close to the spectral color. A color that can be seen as the result of mixing a certain spectral color with white. Among them, the larger the proportion of spectral color, the higher the degree of color close to spectral color, and the higher the saturation of color. The saturation is high, and the color is deep and vivid. The white light component of the spectral color is 0, and the saturation reaches the highest. Usually the value ranges from 0 to 1, the larger the value, the more saturated the color; V (Value, lightness) indicates the degree of brightness of the color, for the light source color, the lightness value is related to the brightness of the illuminant; for the object color, this value and The transmittance or reflectance of the object is related. Typically values range from 0 (black) to 1 (white). For example, (-, 0, 1) represents white, (-, 0, 0) represents black, and (0°, 1, 1) represents red.

Figure 4C shows the projection of the chromaticity diagram of the CIE-XYZ color space on the XY plane. Similar to the RGB color space, CIE-XYZ also uses three values X, Y, and Z to represent a color, which represents a color in red The proportion of primary colors, Y indicates the proportion of green primary color in a color, and Z indicates the proportion of blue primary color in a color.

Figure 4D shows a schematic diagram of the CIE-Lab color space. CIE-Lab uses three values L, a, and b to represent a color. Refer to Figure 4D, where L represents brightness; the positive extreme value of a represents red, and the negative extreme value represents Green; the positive extreme value of b represents yellow, and the negative extreme value represents blue. In some scenarios, the value range of L is 0~100, the value range of a is -128~127, and the value range of b is -128~127. For example, (100, -, -) means white, ( 0, -, -) means black.

(4) Color distance

The color distance (that is, the color difference) is used to characterize the degree of difference between two colors, and the color difference can be calculated by the Euclidean distance in the color space. For example, formulas (1) to formulas (4) show Figure 4A The color distance (ΔE) calculation formulas to the four color spaces shown in FIG. 4D.

The color distance ΔE between two colors (R ₁ , G ₁ , B ₁ ) and (R ₂ , G ₂ , B ₂ ) in the RGB color space can be determined by the following formula (1):

The color distance ΔE between two colors (H ₁ , S ₁ , V ₁ ) and (H ₂ , S ₂ , V ₂ ) in the HSV color space can be determined by the following formula (2):

Among them, r is the radius of the base of the cone in the HSV color space, and h is the height of the cone in the HSV color space.

The color distance ΔE between two colors (X ₁ , Y ₁ , Z ₁ ) and (X ₂ , Y ₂ , Z ₂ ) in the CIE-XYZ color space can be determined by the following formula (3):

The color distance ΔE between two colors (L ₁ , a ₁ , b ₁ ) and (L ₂ , a ₂ , b ₂ ) in the CIE-Lab color space can be determined by the following formula (4):

It can be understood that in some embodiments, the color distance between two colors (L ₁ , a ₁ , b ₁ ) and (L ₂ , a ₂ , b ₂ ) in the CIE-Lab color space can also be calculated in other ways. For example, the color distance ΔE ₉₄ of the two colors can be determined based on the CIE 94 color distance calculation formula (5) proposed by CIE.

In formula (5): ΔL=L ₁ −L ₂ ;

S _L =1,

k _C =1, k _H =1, and k _L , k ₁ , k ₂ take different values according to different application scenarios.

For another example, the color distance ΔE CMC (I:c) between the two colors can also be calculated based on the CMC (I:c) color distance calculation formula (6) proposed by the Society's Color Measurement Committee ₍ CMC):

In the formula (6), ΔL, ΔC _ab , ΔH _ab are defined the same as in the aforementioned formula (5), and will not be repeated here, and:

S _H =S _C (FT+1-F);

l and c take different values according to different application scenarios.

It can be understood that the color distance can also be calculated based on other distances in the two color color spaces, such as Manhattan distance, Chebyshev distance, Minkowski distance, standardized Euclidean distance, Mahalanobis distance, angle cosine, Hamming The distance and the like are not limited here.

It can be understood that when calculating the distance between two colors in different color spaces, the two colors can be converted into the same color space before calculation.

The technical solutions of the embodiments of the present application will be described below with reference to the accompanying drawings.

As mentioned above, in order to improve the aesthetics of the interface displayed by the electronic device, reduce the visual complexity of the interface, etc., the user usually manually arranges multiple display elements in a certain order, such as according to the similarity of colors, the category of each display element, etc. etc., but it is cumbersome for users to manually arrange multiple display elements, which affects user experience.

In view of this, the embodiment of the present application provides a layout method, the electronic device can obtain the layout features between the elements to be laid out in the interface, such as color features, semantic features, etc., and determine the layout feature distance between the elements to be laid out, such as color distance, semantic distance, etc., and then determine the layout scheme that makes the spatial distance between the elements to be laid out and the layout feature distance between the elements to be laid out have the highest similarity based on the layout feature distance between the elements to be laid out, and according to the determined The layout scheme of is used to automatically layout the elements to be laid out. In this way, the electronic device can automatically layout the elements to be laid out according to the layout characteristics among the elements to be laid out, which improves the user experience. In addition, because the spatial distance between the elements to be laid out in the determined layout scheme has the highest correlation with the layout feature distance between the elements to be laid out, that is, the layout interface embodied according to the layout scheme is reflected in the aforementioned Gestalt theory. The principle of similarity and proximity is highly consistent, which makes the visual complexity of the interface after the layout low and the interface simple, which further improves the user experience.

For example, for the scene shown in FIG. 1, the electronic device 100 can obtain the main colors of each icon, and determine the color distance between the main colors of each icon, and then use an optimization algorithm, such as a genetic algorithm, to determine the color distance between the main colors of each icon. The layout scheme with the highest similarity between the color distance between the main colors and the space distance between each icon, and automatically adjust the position of each icon according to the determined layout scheme. In this way, the electronic device 100 can automatically arrange the icons in the electronic device 100 according to the main color of each icon, which improves the user's experience of adjusting the interface layout of the electronic device 100 . In addition, since the color distance between icons and the spatial distance between icons in the determined layout scheme have the highest similarity, that is, icons with similar colors are adjacent in position, that is, the icons after layout according to the layout scheme The above-mentioned similarity principle and proximity principle reflected in the interface are highly consistent, making the interface after the layout relatively simple and further improving user experience.

It can be understood that the main color of the icon is the main color of the icon, for example, the main color of the icon may be the most used color of the icon, or the average value of the most used color of a category. In some embodiments, the electronic device 100 can obtain the color values (such as RGB values) of each pixel of the icon, use a clustering method, such as a spectral clustering algorithm, to group pixels with similar color values into the same class, and use The color corresponding to the average value of the color values of the pixels of the category with the largest number of pixels is used as the main color of the icon.

It can be understood that, in some embodiments, if an icon has multiple main colors, the average value of the multiple main colors can be calculated as the color distance from the main colors of other icons as the color distance between the icon and other icons.

It can be understood that the aforementioned elements to be laid out include but are not limited to icons, service cards, and pictures.

It can be understood that the electronic device 100 to which the embodiment of the present application is applicable includes, but is not limited to, a mobile phone, a laptop computer, a smart TV, a tablet computer, a server, a head-mounted display, a mobile email device, a portable game console, a portable music player, The reader device and the like are not limited in this embodiment of the application.

In the following, the electronic device 100 is the mobile phone 100, combined with the scene shown in FIG. 1, taking the icon in the display interface of the mobile phone 100 as the element to be laid out, and the layout feature as the color feature as an example, the technical solution of the embodiment of the present application is introduced in detail.

Specifically, FIG. 5 shows a schematic flowchart of a layout method according to some embodiments of the present application. The subject of execution of the method is the mobile phone 100, as shown in Figure 5, the process includes the following steps:

S501: A layout instruction is detected. That is, when the mobile phone 100 detects the user's layout instruction, it automatically layouts the icons in the display interface of the mobile phone 100 through the layout method provided by the embodiment of the present application.

In some embodiments, referring to FIG. 6A, the user can enter the desktop layout adjustment interface as shown in FIG. 6B after performing the pinching operation on the touch screen of the mobile phone 100 as shown in FIG. 61 Enter the desktop setting interface shown in FIG. 6C and open the "Smart Arrange Icons" control 62. After the mobile phone 100 detects that the user has opened the "Smart Arrange Icons" control 62, the mobile phone 100 can be arranged according to the layout method provided by the embodiment of the present application. The icons in the display interface are automatically laid out.

In some other embodiments, the user can also shake the mobile phone 100 in the layout adjustment interface shown in FIG. 6B . After the mobile phone 100 detects the user's shaking operation, it can display the interface on the mobile phone 100 according to the layout method provided by the embodiment of the present application. The icons in the .

It can be understood that, in some other embodiments, the user's layout instruction detected by the mobile phone 100 may also be other instructions, such as voice instructions, etc., which is not limited here.

S502: Obtain color data of icons to be laid out, and determine a color distance between icons to be laid out based on the color data. That is, the mobile phone 100 obtains the color data of the icons to be laid out in the display interface, such as the main color of the icons to be laid out, and determines the color distance between two icons to be laid out based on the color data of the icons to be laid out.

For example, in some embodiments, the mobile phone 100 can extract the color data of each icon to be laid out, and determine the color distance between the icons to be laid out according to any color distance calculation method shown in the aforementioned formula (1) to formula (6) .

For example, there are 17 icons to be laid out in the display interface of the mobile phone 100 shown in (A) in Figure 1, the distance between the main colors of each icon can be calculated by the aforementioned formula (1), and the size shown in Figure 7 is The 17×17 color distance matrix represents the color distance between icons. For example, the color distance between the "Gallery Icon" and the "Sports Health" icon is the value corresponding to the 4th row, 11th column or the 11th row, 4th column in the matrix shown in Figure 7, which is 31; the "Music" icon and the " The color distance between the "Smart Life" icons is the value corresponding to row 8, column 14 or row 14, column 8 in the matrix shown in Figure 7, which is 4.

It can be understood that, in some embodiments, the color distance between the icon and itself is 0. Referring to FIG. 7 , the elements located on the diagonal of the matrix, that is, the values corresponding to the elements with the same row and column values are the color distance between the icon and itself. For example, the value of the second row and the second column of the matrix in FIG. 7 corresponds to the color distance between the "clock" icon and itself, which is 0.

It can be understood that, in some embodiments, the mobile phone 100 can determine the color distance between the icons to be laid out based on the method shown in any one of the foregoing formulas (1) to formulas (6); in other embodiments, it can also use Computing in other ways, such as dividing the hue into multiple regions, the color distance between colors belonging to the same hue is 0, and the color distance between colors belonging to different hues is 1, which is not limited here.

It can be understood that, in some embodiments, if an icon has multiple main colors, the average value of the multiple main colors can be calculated, and the color between the icon and other icons can be determined according to the hue of the color corresponding to the average value distance.

S503: Determine a layout scheme that maximizes the similarity between the spatial distance and the color distance between the icons to be laid out. That is, the mobile phone 100 simulates and adjusts the positions of the icons to be laid out, and determines a layout scheme that maximizes the similarity of the spatial distance and the color distance between the icons to be laid out.

It can be understood that the determined layout scheme that maximizes the similarity of the spatial distance and color distance between the icons to be laid out makes the interface after the layout more consistent with the principle of similarity and proximity in the aforementioned Gestalt theory, As a result, the visual complexity of the interface after the layout is lower, and the interface is more concise.

In some embodiments, referring to FIG. 8A , the spatial distance between the icons to be laid out can be determined in the following manner: the interface to be laid out is divided into a plurality of grids, each grid is a step size D, and the grid spaced between two icons The product of the grid number and the step size D is used as the spatial distance between the two icons. For example, the space distance between the "Smooth Call" icon and the "Clock" icon and "File Management" icon is D, and the space distance between the "Smooth Call" icon and the "Calendar" icon and "Huawei Video" icon is 2D. . In some other embodiments, the spatial distance between the icons to be laid out may also be the actual distance between the centers of the icons to be laid out, such as the actual pixel distance of the icons to be laid out on the display screen 194 of the mobile phone 100, etc., for example, refer to FIG. 8B , the space distance between the "Smooth Call" icon and the "Clock" icon is 416 pixels, and the space distance between the "Smooth Call" icon and the "Browser" icon is 1096 pixels. It can be understood that, in some other embodiments, the spatial distance between the icons to be laid out may also be determined in other ways, which is not limited here.

In some embodiments, the spatial distance between the icons to be laid out may be represented by a matrix, so as to determine the similarity between the space distance and the color distance between the icons to be laid out. For example, referring to FIG. 9 , a spatial distance matrix with a size of 17×17 may be used to represent the spatial distance between icons in the scene shown in (A) of FIG. 1 . For example, the spatial distance between the icon of "Smooth Call" and the icon of "Calculator" is the value corresponding to row 1, column 9 or row 9, column 1 of the matrix shown in Figure 9, which is 826 pixels; another example, " The spatial distance between the “Memo” icon and the “Recorder” icon is the value corresponding to row 5, column 16 or row 16, column 5 in the matrix shown in FIG. 9 , that is, 1109 pixels.

It can be understood that, in some embodiments, the spatial distance between the icon and itself is 0. Referring to FIG. 9 , the elements located on the diagonal of the matrix, that is, the values corresponding to the elements with the same row and column values are the spatial distance between the icon and itself. For example, the value in the twelfth row and the twelfth column in the matrix shown in FIG. 9 corresponds to the distance between the "weather" icon and itself, which is 0.

In some embodiments, the similarity between the spatial distance and the color distance between the icons to be laid out may be determined by the correlation coefficient of the space distance matrix and the color distance matrix between the icons to be laid out. For example, the Pearson correlation coefficient of the spatial distance matrix and the color distance matrix between the icons to be laid out can be calculated by formula (7) to determine the similarity of the space distance and the color distance between the icons to be laid out:

In formula (7), A is the space distance matrix between the icons to be laid out, B is the color distance matrix between the icons to be laid out, Cov(A, B) is the covariance of A and B,

is the average value of A,

is the average value of B. It can be understood that the correlation coefficient R(A,B) represents the similarity of the spatial distance and color distance between the icons to be laid out, and the larger the absolute value of R(A,B), the greater the spatial distance and the color distance between the icons to be laid out. The higher the similarity. For example, in the interface shown in (A) in Figure 1, the color distance matrix between the icons to be laid out is shown in Figure 7, and the space distance matrix is shown in Figure 9, then it can be determined from the formula (7) in Figure 1 The correlation coefficient between the icons to be laid out in the interface shown in (A) is 0.2078. For another example, based on the above formula (7), it can be obtained that the correlation coefficient between the spatial distance matrix and the color distance matrix between the icons to be laid out in the layout interface shown in (B) of FIG. 1 is 0.6039.

It can be understood that the above-mentioned determination of the similarity of the spatial distance and color distance between the icons to be laid out by the Pearson correlation coefficient is just an example. In other embodiments, it can also be determined by other methods, for example, the Spearman correlation coefficient , Kendall correlation coefficient, etc., are not limited here.

It can be understood that the determination of the similarity of the spatial distance and the color distance between the icons to be laid out by using the correlation coefficient is just an example, and it can also be determined by other methods in other embodiments, which is not limited here.

In some embodiments, the mobile phone 100 determines that the layout scheme that maximizes the similarity between the spatial distance and the color distance between the icons to be laid out can be transformed into an optimization problem, which is solved by linear programming, genetic algorithm, simulated annealing algorithm, etc. An optimization problem, and the solution of the optimization problem is used as the layout scheme that maximizes the similarity between the spatial distance and the color distance between the icons to be laid out, which will be described in detail below, and will not be repeated here. For example, according to some embodiments of the present application, FIG. 10A shows a layout scheme in which the icons in the display interface shown in (A) of FIG. The distribution can refer to Fig. 10B. In FIG. 10A , the value of row i and column j is 1, which means that the i-th icon is laid out at the j-th position in FIG. 10B . For example, if the value corresponding to the 5th row and 15th column is 1, it means that the 5th icon ("memo" icon) is laid out at the 15th position shown in Figure B.

It can be understood that in some embodiments, the determined layout scheme may not be the layout scheme that maximizes the similarity between the spatial distance and the color distance between the layout icons, but the one that maximizes the similarity between the spatial distance and the color distance between the layout icons. Layout schemes greater than the similarity threshold, for example, layout schemes with a similarity threshold greater than 0.5, are not limited in this embodiment of the present application.

S504: Adjust the positions of the icons to be laid out according to the determined layout scheme. That is, the mobile phone 100 adjusts the positions of the icons to be laid out according to the layout scheme determined in step S503 that maximizes the similarity between the spatial distance and the color distance between the icons to be laid out. For example, the mobile phone 100 can rearrange the icons in the display interface of the mobile phone 100 shown in (A) in FIG. 1 according to the layout scheme shown in FIG. 10A to obtain the display interface shown in (B) in FIG. 1 . It can be seen from the display interface shown in (B) in Figure 1 that the icons with similar colors are arranged in similar positions, which makes the similarity principle and proximity principle in the Gestalt theory reflected in this interface relatively consistent. High, reduces the visual complexity of the display interface of the mobile phone 100, and improves the interface simplicity of the display interface of the mobile phone 100.

Next, in the foregoing step S503, determine the similarity between the spatial distance and the color distance between the icons to be laid out by converting the layout scheme that determines the maximum similarity between the space distance and the color distance between the icons to be laid out into an optimization problem. The layout scheme with the highest degree of precision is introduced.

The mobile phone 100 can number the 17 icons to be laid out in the display interface shown in (A) in FIG. 1 (refer to the matrix shown in FIG. The central coordinates of the icons to be laid out) are marked as the 17 layout positions shown in Figure 10B, so that the layout scheme that determines the maximum similarity between the space distance and the color distance between the icons to be laid out is converted into the maximum position shown in formula (8). Optimization problem, that is, under the constraints shown in formula (8), the x _ij (i, j=1, 2, ... _{, 17) value that makes the sum of Sim and Σs im} _the smallest is to make the layout icon space The layout scheme with the highest similarity between the spatial distance and the color distance.

In formula (8):

The values of x _ij , d _ijmn and y _ijmn are 0 or 1;

x _ij indicates that the i-th icon is laid out at the j-th layout position, that is, when the i-th icon is laid out at the j-th layout position, x _ij =1, otherwise x _ij =0;

Constraint ① indicates that only one icon can be laid out in one layout position;

The constraint condition ② indicates that an icon can only be placed in one layout position;

y _ijmn means that the i-th icon layout is at the j-th layout position and the m-th icon is at the n-th layout position, that is, the i-th icon is at the j-th layout position and the m-th icon is at the n-th layout In the case of layout position, y _ijmn = 1, otherwise y _ijmn = 0;

Constraint condition ③ is used to determine the value of y _ijmn according to x _ij and x _mn , that is, when x _ij = and x _mn =1 (the i-th icon layout is at the j-th layout position and the m-th icon is at the n-th In the case of layout position), yijmn=1;

d _ijmn indicates the distance between the i-th icon and the m-th icon when the i-th icon is at the j-th layout position and the m-th icon is at the n-th layout position;

z _im represents the spatial distance between the i-th icon and the m-th icon, that is, the constraint condition ④ is used to calculate the spatial distance between the j-th icon and the n-th icon;

c _im represents the color distance between the i-th icon and the m-th icon,

s _im is the absolute value of the difference between the spatial distance and color distance between the i-th icon and the m-th icon (constraint condition ⑤), which is used to represent the spatial distance and color between icons to be laid out in a layout scheme through Σs _im For the similarity of distance, the smaller the value of Σs _im is, the greater the similarity is, that is, the solution of the optimization problem shown in formula (8) is the x _ij that satisfies constraints ① to ⑤ and makes Σs _im .

It can be understood that, in some embodiments, the layout scheme can be represented by a layout matrix, where the row indicates the layout position, and the column indicates the icon to be laid out, and the element in the i-th row and the j-th column in the layout matrix is 1, which means that the i-th icon Layout at the jth layout position. In some embodiments, the value of each x _ij determined by the foregoing formula (8) is the value of row i and column j in the layout matrix.

Through the method provided in the embodiment of the present application, the mobile phone 100 can automatically arrange the icons to be arranged according to the color characteristics between the icons to be arranged, which improves the user experience. In addition, since the spatial distance between the icons to be laid out in the determined layout scheme and the color distance between the icons to be laid out have the highest similarity, the similarity in the aforementioned Gestalt theory embodied in the interface after layout according to the layout scheme is the highest. The consistency of the principle and the principle of proximity is high, so that the visual complexity of the interface after the layout is low, and the interface is simple, which further improves the user experience.

As mentioned above, in some embodiments, the elements to be laid out can also be semantic features. For example, referring to FIG. Semantic features of each icon, such as the type of application program/function corresponding to each icon, layout the icons. Combining with the scenario shown in FIG. 11 , the technical solutions of the embodiments of the present application are introduced below by taking the elements to be laid out as icons in the drop-down menu interface of the mobile phone 100 and the layout features as semantic features as examples.

Specifically, FIG. 12 shows a schematic flowchart of a layout method according to some embodiments of the present application. The subject of execution of the process is the mobile phone 100, as shown in FIG. 12, the method includes the following steps.

S1201: A layout instruction is detected. That is, when the mobile phone 100 detects the user's layout instruction, it automatically layouts the icons in the drop-down menu interface of the mobile phone 100 through the layout method provided by the embodiment of the present application.

In some embodiments, referring to FIG. 13 , the user can trigger the “edit menu” 132 of the drop-down menu interface by clicking the “edit control” 131 in the drop-down menu interface of the mobile phone 100, and then trigger the layout by turning on the “smart icon layout” switch 1321 instruction. After the mobile phone 100 detects that the user has turned on the "smart icon layout" switch 1321, it can automatically layout the icons in the drop-down menu interface of the mobile phone 100 according to the layout method provided by the embodiment of the present application.

It can be understood that, in other embodiments, the mobile phone 100 triggers the layout method provided in the embodiment of the present application after detecting the automatic layout gesture, operation, etc., which is not limited here.

S1202: Acquire semantic data of icons to be laid out, and determine a semantic distance between icons to be laid out based on the semantic data. That is, the mobile phone 100 obtains the semantic data of the icons to be laid out in the drop-down menu interface, such as the type of application/function corresponding to each icon, and determines the semantic distance between two icons to be laid out based on the semantic data.

Specifically, in some embodiments, the mobile phone 100 can obtain the type of application/function corresponding to the icon to be laid out from the application market program, and the semantic distance between icons corresponding to the same type of application/function is recorded as 0, and the semantic distance between icons of different types The semantic distance between the icons corresponding to the application program/function is recorded as 1.

In some other embodiments, the mobile phone 100 can also obtain data on icons classified by different users, use neural networks, unsupervised learning and other methods to classify (cluster) the semantics of different icons, and classify icons classified into the same category The semantic distance between icons is recorded as 0, and the semantic distance between different types of icons is recorded as 1. For example, FIG. 14 shows a result of clustering icons in the pull-down menu interface shown in FIG. 11. As shown in FIG. 14, the horizontal axis represents different icons, and the vertical axis represents the clustering threshold. The clustering results are cropped according to the preset clustering threshold, and the icons are classified based on the number of cropped classification lines. For example, referring to FIG. 14 , the mobile phone 100 can cut the clustering results with a preset clustering threshold of 15, classify icons into five categories, namely categories C1, C2, C3, C4 and C5, and classify icons belonging to the same category The semantic distance between icons is recorded as 0, and the semantic distance between different types of icons is recorded as 1, such as the "WLAN" icon (that is, mywifi and the icon on the left in the figure), the "Bluetooth" icon, the "airplane mode" icon, the "mobile data " icon, "Hotspot" icon, "Huawei Sharing" icon and "5G" icon all belong to category C1, and the semantic distance between the above seven icons is recorded as 0.

It can be understood that, in some other embodiments, the semantic distance between icons may also be determined in other ways, which is not limited here.

It can be understood that, in some embodiments, the semantic distance between the icons to be laid out can also be represented by a matrix. For details, reference can be made to the representation of the color distance in the aforementioned step S502 , which will not be repeated here.

S1203: Determine a layout scheme that maximizes the similarity between the spatial distance and the semantic distance between the icons to be laid out. That is, the mobile phone 100 simulates and adjusts the positions of the icons to be laid out, and generates a layout scheme that maximizes the similarity between the spatial distance and the semantic distance between the icons to be laid out. For example, the mobile phone 100 may determine the similarity between the spatial distance and the semantic distance between the icons to be laid out based on the correlation coefficient of the space distance and the semantic distance between the icons to be laid out. For details, refer to the above step S503, which will not be repeated here.

It can be understood that in some other embodiments, the determined layout scheme may not be the layout scheme that maximizes the similarity between the spatial distance and the semantic distance between the icons to be laid out, but the space distance and the semantic distance between the icons to be laid out The layout schemes whose similarity is greater than the similarity threshold, for example, the layout schemes whose similarity is greater than 0.5, are not limited here.

S1204: Adjust the positions of the icons to be laid out according to the determined layout scheme. That is, the mobile phone 100 adjusts the positions of the icons to be laid out in the drop-down menu interface according to the layout scheme determined in the aforementioned step S1203. For example, FIG. 15 shows a schematic diagram of the interface after the mobile phone 100 adjusts the icons in the drop-down menu interface of the mobile phone 100 shown in FIG. As shown in Figure 14, the icons classified into the same category in Figure 14 are arranged in adjacent positions, so that the consistency of the similarity principle and proximity principle in the aforementioned Gestalt theory embodied in this interface is relatively high, reducing the need for mobile phones. The visual complexity of the display drop-down menu interface of 100 improves the interface simplicity of the drop-down menu interface of the mobile phone 100 .

Through the method provided in the embodiment of the present application, the mobile phone 100 can automatically layout the icons to be laid out according to the semantic features between the icons to be laid out, which improves the user experience. In addition, due to the highest similarity between the spatial distance between the icons to be laid out and the semantic distance between the icons to be laid out in the determined layout plan, the similarity in the aforementioned Gestalt theory reflected in the interface after layout according to the layout plan The consistency of the principle and the principle of proximity is high, so that the visual complexity of the interface after the layout is low, and the interface is simple, which further improves the user experience.

It can be understood that, in some embodiments, the mobile phone 100 can also perform automatic layout based on multiple layout features at the same time. For example, the elements to be laid out are first laid out according to the semantic features, and then the layout elements that belong to the same category semantically are laid out according to the color features; The vector represents the layout feature of an element to be laid out, and the distance between multidimensional vectors corresponding to each element to be laid out, such as Euclidean distance, Manhattan distance, Chebyshev distance, Minkowski distance, standardized Euclidean distance, Mahalanobis distance , included angle cosine, Hamming distance, etc. are the layout feature distances between the elements to be laid out, and then determine the layout scheme that maximizes the similarity between the layout feature distance and the space distance between the elements to be laid out, and treat the layout according to the layout scheme Elements are automatically laid out.

The embodiment of the present application also provides a layout evaluation method, which determines the layout feature distance between each layout element, such as color distance, semantic feature, etc. The evaluation value of the layout to be evaluated is determined based on the similarity between the layout feature distance between each layout element and the spatial distance between each layout element. Among them, the higher the similarity between the layout feature distance between each layout element and the spatial distance between each layout element, the higher the evaluation value, the lower the visual complexity and the higher the interface simplicity of the interface corresponding to the layout to be evaluated. In this way, the electronic device can adjust the layout based on the evaluation value of the layout to be evaluated to obtain a layout scheme with a higher evaluation value, that is, a layout scheme with low visual complexity and high simplicity of the interface displayed by the electronic device.

In the following, continue to take the mobile phone 100 as an example of the electronic device to introduce the technical solution of the embodiment of the present application.

Specifically, Fig. 16 shows a schematic flowchart of a layout evaluation method according to some embodiments of the present application. The subject of execution of the process is the mobile phone 100, as shown in FIG. 16, the method includes the following steps.

S1601: Obtain layout features and position data between layout elements in the layout to be evaluated. That is, the mobile phone 100 obtains the layout features of each layout element in the layout to be evaluated, such as the color data, semantic data, etc. of each layout element, and the position data of each layout element, such as the center of each layout element on the display screen Pixel coordinates on 194.

For example, when the layout to be evaluated is the layout shown in (A) in Figure 1, the layout elements can be icons in the display interface shown in (A) in Figure 1, and the layout feature can be the main feature of each icon. The position data of each icon may be the grid position shown in FIG. 8A, or the pixel coordinates shown in FIG. 8B, which is not limited here.

For another example, when the layout to be evaluated is the layout shown in Figure 11, the layout elements can be the icons in the drop-down menu interface shown in Figure 11, and the layout feature can be the semantics of the application/function corresponding to each icon, each The location data of the icons may be the pixel coordinates of the center of each icon on the display screen of the mobile phone 100 .

For another example, in the scene shown in Figure 17, the layout to be evaluated is layout A in which grayscale graphics 1 to grayscale graphics 8 are arranged in the order of 1, 2, ..., 7, 8, and the layout elements are graphics 1 to 8 , the layout feature is the gray value of graph 1 to graph 8.

S1602: Determine the layout feature distance and spatial distance of each layout element. That is, the mobile phone 100 determines the layout feature distance of each layout element according to the layout feature of each layout element, and determines the spatial distance of each layout element according to the position data of each layout element.

For example, when the layout to be evaluated is the layout shown in Figure 1 (A), the layout feature distance between icons can be the color distance of the main color of each icon. For details, please refer to the relevant description in Figure 7 above , which will not be described in detail here; the spatial distance of each icon may be the distance between the pixel coordinates of each icon, for details, refer to the relevant description in FIG. 9 above, and will not be described in detail here.

For another example, when the layout to be evaluated is the layout shown in Figure 11, the characteristic distance between the icons may be the semantic distance of each icon, for details, refer to the relevant description in Figure 14 above, which will not be repeated here; each icon The spatial distance between them may be the distance between the pixel coordinates of the center of each icon on the display screen 194 of the mobile phone 100, for details, please refer to the relevant description of the aforementioned step S1203, which will not be repeated here.

For another example, in the scene shown in Figure 17, the layout feature distance of each layout element in layout A can be the absolute value of the gray value difference between the graphics, for example, the gray value of graphic 1 is 0, and the gray value of graphic 2 If the grayscale value is 180, then the layout feature distance between graphic 1 and graphic 2 is 180; the spatial distance of each layout element can be the distance between the graphic sorting numbers, for example, the sorting number of graphic 1 is 1, and the sorting number of graphic 4 number is 4, then the spatial distance between graphic 1 and graphic 4 is 3. It can be understood that, referring to FIG. 17 , in some embodiments, the layout feature distances and spatial distances between two figures 1 to 8 can be represented by a matrix.

S1603: Determine the evaluation value of the layout to be evaluated according to the similarity between the layout feature distance and the spatial distance of each layout element. That is, the mobile phone 100 determines the evaluation value of the layout to be evaluated according to the similarity between the layout feature distance and the spatial distance of each layout element in the layout to be evaluated, and the higher the similarity between the layout feature distance and the spatial distance of each layout element, the higher the evaluation value , the lower the visual complexity of the display interface corresponding to the layout to be evaluated, the higher the simplicity of the interface.

In some embodiments, the correlation coefficient between the layout feature distance and the spatial distance of each layout element can be calculated based on the method in the aforementioned step S503, and the calculated correlation coefficient can be used as the evaluation value of the layout to be evaluated.

For example, if the layout to be evaluated is the layout shown in (A) in FIG. 1 , the evaluation value of the layout may be the correlation coefficient 0.2078 calculated in the foregoing example.

For another example, when the layout to be evaluated is the layout shown in (B) in FIG. 1 , the evaluation value of the layout may be the correlation coefficient 0.6039 calculated in the foregoing example.

For another example, in the scene shown in FIG. 17 , based on the foregoing formula (7) and the layout feature distance matrix and the space distance matrix in FIG. 17 , it can be determined that the evaluation value of layout A is 0.0658. Further, FIG. 18 shows layout A, layout B, layout C, layout D, layout E, and layout F arranged in different order from graph 1 to graph 8 shown in FIG. 17 and the evaluation values corresponding to each layout. Referring to Figure 18, the evaluation values of layout A, layout B, layout C, layout D, layout E, and layout F are 0.0658, 0.2632, 0.329, 0.329, 0.5264, and 0.7237, respectively. The similarity between the gray value distance and the spatial distance increases, that is, the higher the evaluation value, the lower the visual complexity and the higher the simplicity of the layout.

Through the method provided in the embodiment of the present application, the mobile phone 100 can evaluate the visual complexity and simplicity of the interface corresponding to the layout to be evaluated, and generate an evaluation value of the layout, so that the mobile phone 100 can adjust the layout based on the evaluation value, A layout scheme with a higher evaluation value is obtained, that is, a layout scheme with low visual complexity and high simplicity of the interface displayed on the mobile phone 100 .

Fig. 19 shows a schematic structural diagram of a mobile phone 100 according to some embodiments of the present application.

As shown in Figure 19, the mobile phone 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (universal serial bus, USB) interface 130, a charging management module 140, a power management module 141, a battery 142, Antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, audio module 170, speaker 170A, receiver 170B, microphone 170C, earphone interface 170D, sensor module 180, button 190, motor 191, indicator 192, camera 193, A display screen 194, and a subscriber identification module (subscriber identification module, SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, an acceleration sensor 180E, a fingerprint sensor 180H, a touch sensor 180K, an ambient light sensor 180L and the like.

It can be understood that the structure shown in the embodiment of the present application does not constitute a specific limitation on the mobile phone 100 . In some other embodiments of the present application, the mobile phone 100 may include more or fewer components than shown in the figure, or combine certain components, or separate certain components, or arrange different components. The illustrated components can be realized in hardware, software or a combination of software and hardware.

The processor 110 may include one or more processing units, for example: the processor 110 may include an application processor (application processor, AP), a modem processor, a graphics processing unit (graphics processing unit, GPU), an image signal processor (image signal processor, ISP), controller, video codec, digital signal processor (digital signal processor, DSP), baseband processor, and/or neural network processor (neural-network processing unit, NPU), etc. Wherein, different processing units may be independent devices, or may be integrated in one or more processors.

The controller can generate an operation control signal according to the instruction opcode and timing signal, and complete the control of fetching and executing the instruction.

A memory may also be provided in the processor 110 for storing instructions and data. In some embodiments, the memory in processor 110 is a cache memory. The memory may hold instructions or data that the processor 110 has just used or recycled. If the processor 110 needs to use the instruction or data again, it can be called directly from the memory. Repeated access is avoided, and the waiting time of the processor 110 is reduced, thereby improving the efficiency of the system. In some embodiments, the processor 110 may call and execute execution instructions of the layout method/layout evaluation method provided by the embodiments of the present application stored in the memory, so as to implement the layout method/layout evaluation method provided by the embodiments of the present application.

In some embodiments, processor 110 may include one or more interfaces. The interface may include an integrated circuit (inter-integrated circuit, I2C) interface, an integrated circuit built-in audio (inter-integrated circuit sound, I2S) interface, a pulse code modulation (pulse code modulation, PCM) interface, a universal asynchronous transmitter (universal asynchronous receiver/transmitter, UART) interface, mobile industry processor interface (mobile industry processor interface, MIPI), general-purpose input and output (general-purpose input/output, GPIO) interface, subscriber identity module (subscriber identity module, SIM) interface, and /or universal serial bus (universal serial bus, USB) interface, etc.

The I2C interface is a bidirectional synchronous serial bus, including a serial data line (serial data line, SDA) and a serial clock line (derail clock line, SCL). In some embodiments, processor 110 may include multiple sets of I2C buses. The processor 110 can be respectively coupled to the touch sensor 180K, the charger, the flashlight, the camera 193 and the like through different I2C bus interfaces. For example, the processor 110 may be coupled to the touch sensor 180K through the I2C interface, so that the processor 110 and the touch sensor 180K communicate through the I2C bus interface to realize the touch function of the mobile phone 100 .

The I2S interface can be used for audio communication. In some embodiments, processor 110 may include multiple sets of I2S buses. The processor 110 may be coupled to the audio module 170 through an I2S bus to implement communication between the processor 110 and the audio module 170 . In some embodiments, the audio module 170 can transmit audio signals to the wireless communication module 160 through the I2S interface, so as to realize the function of answering calls through the Bluetooth headset.

The PCM interface can also be used for audio communication, sampling, quantizing and encoding the analog signal. In some embodiments, the audio module 170 and the wireless communication module 160 may be coupled through a PCM bus interface. In some embodiments, the audio module 170 can also transmit audio signals to the wireless communication module 160 through the PCM interface, so as to realize the function of answering calls through the Bluetooth headset. Both the I2S interface and the PCM interface can be used for audio communication.

The UART interface is a universal serial data bus used for asynchronous communication. The bus can be a bidirectional communication bus. It converts the data to be transmitted between serial communication and parallel communication. In some embodiments, a UART interface is generally used to connect the processor 110 and the wireless communication module 160 . For example: the processor 110 communicates with the Bluetooth module in the wireless communication module 160 through the UART interface to realize the Bluetooth function. In some embodiments, the audio module 170 can transmit audio signals to the wireless communication module 160 through the UART interface, so as to realize the function of playing music through the Bluetooth headset.

The MIPI interface can be used to connect the processor 110 with peripheral devices such as the display screen 194 and the camera 193 . MIPI interface includes camera serial interface (camera serial interface, CSI), display serial interface (display serial interface, DSI), etc. In some embodiments, the processor 110 communicates with the camera 193 through the CSI interface to realize the shooting function of the mobile phone 100 . The processor 110 communicates with the display screen 194 through the DSI interface to realize the display function of the mobile phone 100 .

The GPIO interface can be configured by software. The GPIO interface can be configured as a control signal or as a data signal. In some embodiments, the GPIO interface can be used to connect the processor 110 with the camera 193 , the display screen 194 , the wireless communication module 160 , the audio module 170 , the sensor module 180 and so on. The GPIO interface can also be configured as an I2C interface, I2S interface, UART interface, MIPI interface, etc.

The USB interface 130 is an interface conforming to the USB standard specification, specifically, it can be a Mini USB interface, a Micro USB interface, a USB Type C interface, and the like. The USB interface 130 can be used to connect a charger to charge the mobile phone 100, and can also be used to transmit data between the mobile phone 100 and peripheral devices. It can also be used to connect headphones and play audio through them. This interface can also be used to connect other electronic devices, such as AR devices.

The charging management module 140 is configured to receive a charging input from a charger. While the charging management module 140 is charging the battery 142 , it can also supply power to the electronic device through the power management module 141 .

The power management module 141 is used for connecting the battery 142 , the charging management module 140 and the processor 110 . The power management module 141 receives the input from the battery 142 and/or the charging management module 140 to provide power for the processor 110 , the internal memory 121 , the display screen 194 , the camera 193 , and the wireless communication module 160 .

The wireless communication function of the mobile phone 100 can be realized by the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.

Antenna 1 and Antenna 2 are used to transmit and receive electromagnetic wave signals.

The mobile communication module 150 can provide wireless communication solutions including 2G/3G/4G/5G applied on the mobile phone 100 . The mobile communication module 150 may include at least one filter, switch, power amplifier, low noise amplifier (low noise amplifier, LNA) and the like. The mobile communication module 150 can receive electromagnetic waves through the antenna 1, filter and amplify the received electromagnetic waves, and send them to the modem processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modem processor, convert it into electromagnetic wave and radiate it through the antenna 1 . In some embodiments, at least part of the functional modules of the mobile communication module 150 may be set in the processor 110 . In some embodiments, at least part of the functional modules of the mobile communication module 150 and at least part of the modules of the processor 110 may be set in the same device.

The wireless communication module 160 can provide wireless local area networks (wireless local area networks, WLAN) (such as wireless fidelity (Wireless Fidelity, Wi-Fi) network), bluetooth (bluetooth, BT), global navigation satellite system, etc. applied on the mobile phone 100 (global navigation satellite system, GNSS), frequency modulation (frequency modulation, FM), near field communication technology (near field communication, NFC), infrared technology (infrared, IR) and other wireless communication solutions. The wireless communication module 160 may be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2 , frequency-modulates and filters the electromagnetic wave signals, and sends the processed signals to the processor 110 . The wireless communication module 160 can also receive the signal to be sent from the processor 110 , frequency-modulate it, amplify it, and convert it into electromagnetic waves through the antenna 2 for radiation.

The mobile phone 100 realizes the display function through the GPU, the display screen 194, and the application processor. The GPU is a microprocessor for image processing, and is connected to the display screen 194 and the application processor. GPUs are used to perform mathematical and geometric calculations for graphics rendering. Processor 110 may include one or more GPUs that execute program instructions to generate or change display information.

The display screen 194 is used to display images, videos and the like. The display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active matrix organic light emitting diode or an active matrix organic light emitting diode (active-matrix organic light emitting diode, AMOLED), flexible light-emitting diode (flex light-emitting diode, FLED), Mini-LED, Micro-LED, Micro-OLED, quantum dot light emitting diodes (quantum dot light emitting diodes, QLED), etc. In some embodiments, the mobile phone 100 may include 1 or N display screens 194, where N is a positive integer greater than 1.

Camera 193 is used to capture still images or video. The object generates an optical image through the lens and projects it to the photosensitive element. The photosensitive element may be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, and then transmits the electrical signal to the ISP to convert it into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. DSP converts digital image signals into standard RGB, YUV and other image signals. In some embodiments, the mobile phone 100 may include 1 or N cameras 193, where N is a positive integer greater than 1.

The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the mobile phone 100. The external memory card communicates with the processor 110 through the external memory interface 120 to implement a data storage function. Such as saving music, video and other files in the external memory card.

The internal memory 121 may be used to store computer-executable program codes including instructions. The internal memory 121 may include an area for storing programs and an area for storing data. Wherein, the stored program area can store an operating system, at least one application program required by a function (such as a sound playing function, an image playing function, etc.) and the like. The storage data area can store data (such as audio data, phone book, etc.) created during the use of the mobile phone 100 . In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one magnetic disk storage device, flash memory device, universal flash storage (universal flash storage, UFS) and the like. The processor 110 executes various functional applications of the mobile phone 100 by executing instructions stored in the internal memory 121 and/or instructions stored in a memory provided in the processor 110 .

The mobile phone 100 can realize the audio function through the audio module 170 , the speaker 170A, the receiver 170B, the microphone 170C, the earphone interface 170D, and the application processor. Such as music playback, recording, etc.

The audio module 170 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signal. The audio module 170 may also be used to encode and decode audio signals.

Speaker 170A, also referred to as a "horn", is used to convert audio electrical signals into sound signals.

Receiver 170B, also called "earpiece", is used to convert audio electrical signals into sound signals.

The microphone 170C, also called "microphone" or "microphone", is used to convert sound signals into electrical signals.

The earphone interface 170D is used for connecting wired earphones.

The pressure sensor 180A is used to sense the pressure signal and convert the pressure signal into an electrical signal. In some embodiments, pressure sensor 180A may be disposed on display screen 194 . There are many types of pressure sensors 180A, such as resistive pressure sensors, inductive pressure sensors, and capacitive pressure sensors. A capacitive pressure sensor may be comprised of at least two parallel plates with conductive material. When a force is applied to the pressure sensor 180A, the capacitance between the electrodes changes. The mobile phone 100 determines the strength of the pressure based on the change in capacitance. When a touch operation acts on the display screen 194, the mobile phone 100 detects the intensity of the touch operation according to the pressure sensor 180A. The mobile phone 100 may also calculate the touched position according to the detection signal of the pressure sensor 180A. In some embodiments, touch operations acting on the same touch position but with different touch operation intensities may correspond to different operation instructions. For example: when a touch operation whose intensity is less than the first pressure threshold is applied to the “smart icon layout” switch 1321 shown in FIG. When the touch operation of the pressure threshold is applied to the “smart icon layout” switch 1321 shown in FIG. 13 , the instructions arranged according to the color characteristics are executed.

The acceleration sensor 180E can detect the acceleration of the mobile phone 100 in various directions (generally three axes). When the mobile phone 100 is stationary, the magnitude and direction of gravity can be detected. It can also be used to identify the posture of electronic devices, and can be used in applications such as horizontal and vertical screen switching, pedometers, etc.

The ambient light sensor 180L is used for sensing ambient light brightness. The mobile phone 100 can adaptively adjust the brightness of the display screen 194 according to the perceived ambient light brightness. The ambient light sensor 180L can also be used to automatically adjust the white balance when taking pictures. The ambient light sensor 180L can also cooperate with the proximity light sensor 180G to detect whether the mobile phone 100 is in the pocket, so as to prevent accidental touch.

The fingerprint sensor 180H is used to collect fingerprints. The mobile phone 100 can use the collected fingerprint features to realize fingerprint unlocking, access to the application lock, take pictures with the fingerprint, answer calls with the fingerprint, and the like. In this embodiment of the application, the main space and the private space may use different unlocking fingerprints for verification and login, so as to ensure the security of private data in the private space. The user can verify different fingerprints on the mobile phone 100 to enter the main space and the private space respectively.

The touch sensor 180K is also called "touch device". The touch sensor 180K can be disposed on the display screen 194, and the touch sensor 180K and the display screen 194 form a touch screen, also called a “touch screen”. The touch sensor 180K is used to detect a touch operation on or near it. The touch sensor can pass the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through the display screen 194 . In some other embodiments, the touch sensor 180K may also be disposed on the surface of the mobile phone 100 , which is different from the position of the display screen 194 .

The keys 190 include a power key, a volume key and the like. The key 190 may be a mechanical key. It can also be a touch button. The mobile phone 100 can receive key input and generate key signal input related to user settings and function control of the mobile phone 100 .

The motor 191 can generate a vibrating reminder.

The indicator 192 can be an indicator light, and can be used to indicate charging status, power change, and can also be used to indicate messages, missed calls, notifications, and the like.

The SIM card interface 195 is used for connecting a SIM card.

It can be understood that the operating system used by the above-mentioned mobile phone 100 may be one of Android ^™ , IOS ^™ (Input Output System) and Harmony OS (Harmony OS), which is not limited here. Taking the Hongmeng operating system as an example, the software architecture diagram of the mobile phone 100 is introduced below.

Fig. 20 shows a schematic diagram of a software architecture of a mobile phone 100 according to some embodiments of the present application. As shown in FIG. 20 , the software architecture of the mobile phone 100 mainly includes:

Application layer 20: may include system applications 21 and extension applications 22 (or third-party applications). Wherein, the system application 21 may include a desktop 211, a setting 212, and the like. The layout method/layout evaluation method provided by each embodiment of the present application can be executed by the desktop application, that is, the desktop 211 can obtain the layout characteristics of the icons in the display interface of the mobile phone 100 or in the drop-down menu, such as color characteristics and semantics. Features, etc., and determine the layout feature distance between icons, such as color distance, semantic distance, etc., and determine the layout scheme that makes the layout feature distance and space distance between icons the highest similarity, and then follow the determined layout scheme Adjust the display interface of the mobile phone 100 or the icons in the pull-down menu.

The framework layer 30 provides a multilingual framework for the application layer, including an interface (User Interface, UI) framework 31, a user program framework 32, and a capability framework 33. Wherein, the UI framework 31 includes a window manager, a content provider, a view system, a phone manager, a resource manager, a notification manager, etc., which will not be repeated here. The user program framework 32 and the capability framework 33 can provide application programs with the capabilities of various capability components required by the application, such as computing power (may include CPU computing power, graphics processing unit (Graphics Processing Unit, GPU) computing power, image signal processor (Image Signal Processor, ISP) computing power, etc.), sound pickup capabilities (including microphone pickup capabilities, voice recognition capabilities, etc.), security capabilities in equipment security protection (including trusted operating environment security levels, etc.), display capabilities (can include screen resolution, screen size, etc.), playback capability (including sound amplification capability, stereo effect capability, etc., and storage capability (may include device memory capability, random access memory (RAM) capability, etc.) etc., no limitation here.

The system service layer 40 is the core of the software system of the mobile phone 100 and can provide services to the application programs of the application layer 20 through the framework layer 30 . The system service layer 40 includes a distributed soft bus 41, a distributed data management module 42, a distributed task scheduling module 43, and the like. Among them: the distributed soft bus 41 is used to couple the mobile phone 10 with other electronic devices to form a distributed system; the distributed data management module 42 is based on the distributed soft bus 41 to realize distributed management of application program data and user data. The distributed task scheduling module 43 builds a unified distributed service management (discovery, synchronization, registration, call) mechanism based on technical characteristics such as distributed soft bus and distributed data management, and supports remote startup and remote call of cross-device applications , remote connection, migration and other operations, can select the appropriate device to run tasks according to the capabilities, locations, business operation status, resource usage, and user habits and intentions of different devices.

The kernel layer 50 includes a kernel subsystem 51 and a driver subsystem 52 . The kernel subsystem 51 provides basic kernel capabilities to the upper layer by shielding differences between multiple kernels, including process/thread management, memory management, file system, network management, and peripheral management. The driver subsystem 52 includes a hardware driver framework, which can provide the mobile phone 100 with a unified peripheral device access capability and a management framework.

It can be understood that the software architecture of the mobile phone 100 shown in FIG. 20 is just an example. In other embodiments, the mobile phone 10 may also adopt other software architectures different from the above-mentioned FIG. 20 , which is not limited in this embodiment of the present application.

Various embodiments of the mechanisms disclosed in this application may be implemented in hardware, software, firmware, or a combination of these implementation methods. Embodiments of the present application may be implemented as a computer program or program code executed on a programmable system comprising at least one processor, a storage system (including volatile and non-volatile memory and/or storage elements) , at least one input device, and at least one output device.

Program code can be applied to input instructions to perform the functions described herein and to generate output information. The output information may be applied to one or more output devices in known manner. For the purposes of this application, a processing system includes any system having a processor such as, for example, a digital signal processor (DSP), microcontroller, application specific integrated circuit (ASIC), or microprocessor.

The program code can be implemented in a high-level procedural language or an object-oriented programming language to communicate with the processing system. Program code can also be implemented in assembly or machine language, if desired. In fact, the mechanisms described in this application are not limited in scope to any particular programming language. In either case, the language may be a compiled or interpreted language.

In some cases, the disclosed embodiments may be implemented in hardware, firmware, software, or any combination thereof. The disclosed embodiments can also be implemented as instructions carried by or stored on one or more transitory or non-transitory machine-readable (e.g., computer-readable) storage media, which can be executed by one or more processors read and execute. For example, instructions may be distributed over a network or via other computer-readable media. Thus, a machine-readable medium may include any mechanism for storing or transmitting information in a form readable by a machine (e.g., a computer), including, but not limited to, floppy disks, optical disks, optical disks, read-only memories (CD-ROMs), magnetic Optical discs, read-only memory (ROM), random-access memory (RAM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), magnetic or optical cards, flash memory, or A tangible, machine-readable memory used to transmit information (eg, carrier waves, infrared signals, digital signals, etc.) by electrical, optical, acoustic, or other forms of propagating signals using the Internet. Thus, a machine-readable medium includes any type of machine-readable medium suitable for storing or transmitting electronic instructions or information in a form readable by a machine (eg, a computer).

In the drawings, some structural or methodological features may be shown in a particular arrangement and/or order. However, it should be understood that such specific arrangements and/or orderings may not be required. Rather, in some embodiments, these features may be arranged in a different manner and/or order than shown in the illustrative figures. Additionally, the inclusion of structural or methodological features in a particular figure does not imply that such features are required in all embodiments, and in some embodiments these features may not be included or may be combined with other features.

It should be noted that each unit/module mentioned in each device embodiment of this application is a logical unit/module. Physically, a logical unit/module can be a physical unit/module, or a physical unit/module. A part of the module can also be realized with a combination of multiple physical units/modules, the physical implementation of these logical units/modules is not the most important, the combination of functions realized by these logical units/modules is the solution The key to the technical issues raised. In addition, in order to highlight the innovative part of this application, the above-mentioned device embodiments of this application do not introduce units/modules that are not closely related to solving the technical problems proposed by this application, which does not mean that the above-mentioned device embodiments do not exist other units/modules.

It should be noted that in the examples and descriptions of this patent, relative terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply There is no such actual relationship or order between these entities or operations. Furthermore, the term "comprises", "comprises" or any other variation thereof is intended to cover a non-exclusive inclusion such that a process, method, article, or apparatus comprising a set of elements includes not only those elements, but also includes elements not expressly listed. other elements of or also include elements inherent in such a process, method, article, or device. Without further limitations, an element defined by the statement "comprising a" does not exclude the presence of additional identical elements in the process, method, article or apparatus comprising said element.

Although this application has been shown and described with reference to certain preferred embodiments thereof, those skilled in the art will understand that various changes in form and details may be made therein without departing from this disclosure. The spirit and scope of the application.

Claims

A layout method applied to electronic equipment, characterized in that it comprises:

Obtain the layout characteristics of multiple elements to be laid out;

Based on the layout features of each of the elements to be laid out, determine the layout feature distance between each of the elements to be laid out, wherein the higher the similarity of the layout features between the elements to be laid out, the higher the The smaller the layout feature distance is;

Determine the layout scheme based on the similarity between the layout feature distance between the two elements to be laid out and the spatial distance between the two elements to be laid out;

The position of each element to be laid out is adjusted according to the determined layout scheme.
The method according to claim 1, wherein the layout features include at least one of the following features: color features and semantic features.
The method according to claim 2, wherein when the layout feature is a color feature, the determining the layout feature distance between each of the elements to be laid out includes:

Determine the characteristic color of each element to be laid out according to the layout feature of each element to be laid out;

The color distance between the characteristic colors between the two elements to be laid out is determined to determine the layout feature distance between the two elements to be laid out;

Wherein, when the main color of the element to be laid out is single, the characteristic color of the element to be laid out is the main color of the element to be laid out;

When there are multiple main colors of the elements to be laid out, the characteristic color of the elements to be laid out is an average value of the main colors of the elements to be laid out.
The method according to claim 3, wherein the color distance between the characteristic colors of each of the elements to be laid out includes any one of the following distances: Euclidean distance, Manhattan distance, cut Bishev distance, Minkowski distance, normalized Euclidean distance, Mahalanobis distance, angle cosine, Hamming distance.
The method according to claim 2, wherein when the layout feature is a color feature, the determining the layout feature distance between each of the elements to be laid out includes:

Determine the characteristic color of each element to be laid out according to the layout feature of each element to be laid out;

Determining the layout feature distance between two elements to be laid out whose feature colors belong to the same hue as 0, and determining the layout feature distance between two elements to be laid out whose feature colors belong to different hues to be 1;

Wherein, when the main color of the element to be laid out is single, the characteristic color of the element to be laid out is the main color of the element to be laid out;

When there are multiple main colors of the elements to be laid out, the characteristic color of the elements to be laid out is an average value of the main colors of the elements to be laid out.
The method according to claim 2, wherein when the layout feature is a semantic feature, the determining the layout feature distance between each of the elements to be laid out includes:

Classifying each of the elements to be laid out according to the semantics of the elements to be laid out;

The layout feature distance between two layout elements classified into the same category is determined as 0, and the layout feature distance between two layout elements of different categories is determined as 1.
The method according to any one of claims 1 to 6, characterized in that, the distance between the layout features based on each pair of elements to be laid out and the space distance between each pair of elements to be laid out The similarity determines the layout scheme, including:

Determine the layout scheme that maximizes the similarity between the layout feature distances between the elements to be laid out and the spatial distances between the elements to be laid out.
The method according to any one of claims 1 to 6, characterized in that, the distance between the layout features based on each pair of elements to be laid out and the space distance between each pair of elements to be laid out The similarity determines the layout scheme, including:

A layout scheme is determined that makes the similarity between the layout feature distance between two elements to be laid out and the space distance between each element to be laid out greater than a similarity threshold.
The method according to any one of claims 1 to 8, characterized in that, through the correlation between the layout feature distance between two elements to be laid out and the spatial distance between two elements to be laid out The coefficient determines the similarity between the layout feature distance between the elements to be laid out and the space distance between the elements to be laid out.
The method according to claim 9, wherein the correlation coefficient comprises any one of the following correlation coefficients: Pearson correlation coefficient, Spearman correlation coefficient, Kendall correlation coefficient.
The method according to any one of claims 1 to 10, wherein the elements to be laid out include any one of the following elements: icons, pictures, and service cards.
The method according to claim 11, wherein the electronic device acquires layout features of a plurality of elements to be laid out when detecting a layout instruction input by a user on the electronic device.
A readable medium, characterized in that instructions are stored on the readable medium, and when the instructions are executed on the electronic equipment, the electronic equipment implements the method described in any one of claims 1 to 12.
An electronic device, characterized in that it comprises:

memory for storing instructions to be executed by one or more processors of the electronic device;

And the processor is one of the processors of the electronic device, configured to execute the instructions stored in the memory to implement the method according to any one of claims 1 to 12.