WO2023093390A1 - Procédé de disposition, support lisible et dispositif électronique - Google Patents

Procédé de disposition, support lisible et dispositif électronique Download PDF

Info

Publication number
WO2023093390A1
WO2023093390A1 PCT/CN2022/126443 CN2022126443W WO2023093390A1 WO 2023093390 A1 WO2023093390 A1 WO 2023093390A1 CN 2022126443 W CN2022126443 W CN 2022126443W WO 2023093390 A1 WO2023093390 A1 WO 2023093390A1
Authority
WO
WIPO (PCT)
Prior art keywords
layout
laid out
elements
distance
color
Prior art date
Application number
PCT/CN2022/126443
Other languages
English (en)
Chinese (zh)
Inventor
刘梦迪
张警吁
乔韩
Original Assignee
华为技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 华为技术有限公司 filed Critical 华为技术有限公司
Publication of WO2023093390A1 publication Critical patent/WO2023093390A1/fr

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0481—Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
    • G06F3/04817—Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance using icons
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0481—Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
    • G06F3/0482—Interaction with lists of selectable items, e.g. menus
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T11/00—2D [Two Dimensional] image generation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T11/00—2D [Two Dimensional] image generation
    • G06T11/001—Texturing; Colouring; Generation of texture or colour
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T11/00—2D [Two Dimensional] image generation
    • G06T11/60—Editing figures and text; Combining figures or text
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
    • Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
    • Y02P90/30—Computing systems specially adapted for manufacturing

Definitions

  • the present application relates to the field of electronic technology, in particular to a layout method, a readable medium and electronic equipment.
  • Electronic devices usually display multiple display elements on the same interface, for example, displaying icons of multiple application programs on the desktop or menu bar of the electronic device.
  • the user In order to improve the aesthetics of the interface, reduce the visual complexity of the interface, etc., the user usually manually arranges multiple display elements in a certain order, for example, according to the similarity of colors and the category of each display element.
  • the embodiments of the present application provide a layout method, a readable medium and an electronic device.
  • obtaining the layout features of the elements to be laid out determine the distance between the layout features of the elements to be laid out, and according to the similarity between the layout feature distance and the space distance between the two elements to be laid out layout.
  • the electronic device can automatically layout the layout elements, and the interface after layout has low visual complexity and high simplicity, which improves user experience.
  • the embodiment of the present application provides a layout method, which is applied to electronic devices, and the method includes: obtaining the layout features of multiple elements to be laid out; The layout feature distance between the elements to be laid out, wherein the higher the similarity of the layout features between the elements to be laid out, the smaller the layout feature distance between the elements to be laid out; According to the similarity between the two spatial distances, the layout scheme is determined; according to the determined layout scheme, the position of each element to be laid out is adjusted.
  • the electronic device may automatically layout the elements to be laid out according to at least one layout feature among the elements to be laid out, which improves user experience.
  • the interface after layout according to the layout scheme reflects the above-mentioned Gestalt theory. The consistency of similarity principle and proximity principle is high, which makes the visual complexity of the interface after the layout low and the interface simple, which improves the user experience.
  • the foregoing layout features include at least one of the following features: color features and semantic features.
  • the above-mentioned determination of the layout feature distance between each element to be laid out includes: according to the layout feature of each element to be laid out, determine the The characteristic color of the layout element; the color distance between the characteristic colors of each element to be laid out is determined as the layout characteristic distance between the two elements to be laid out; wherein, when the main color of the element to be laid out is a single In this case, the characteristic color of the element to be laid out is the main color of the element to be laid out; when there are multiple main colors of the element to be laid out, the characteristic color of the element to be laid out is the average value of the main colors of the elements to be laid out.
  • the layout characteristic distance is the color distance between two characteristic colors of the elements to be laid out.
  • the color distance may be calculated according to the color distance calculation method mentioned below, or other methods may be used for calculation.
  • the color distance between the feature colors of the above-mentioned elements to be laid out includes any one of the following distances: Euclidean distance, Manhattan distance, Chebyshev distance, Minkowski distance, normalized Euclidean distance, Mahalanobis distance, angle cosine, Hamming distance.
  • the above-mentioned determination of the layout feature distance between each element to be laid out includes: according to the layout feature of each element to be laid out, determine the The characteristic color of the layout element; determine the layout characteristic distance between each pair of layout elements whose characteristic color belongs to the same hue as 0, and determine the layout characteristic distance between each pair of layout elements whose characteristic color belongs to different hues as 1;
  • the feature color of the element to be laid out is the main color of the element to be laid out; in the case of multiple main colors of the element to be laid out; in the case of multiple main colors of the element to be laid out, the feature color of the element to be laid out is The average of multiple dominant colors for the element to be laid out. .
  • the layout feature distance between two elements to be laid out can be determined according to the hue of the characteristic color of each element to be laid out, that is, the distance between two elements to be laid out whose feature color belongs to the same hue
  • the layout feature distance is determined to be 0, and the layout feature distance between two to-be-layout elements whose feature colors belong to different hues is determined to be 1.
  • the calculation amount of the electronic device in the process of determining the layout scheme based on the similarity between the layout feature distance between the two elements to be laid out and the space distance between the two elements to be laid out can be reduced, and the layout speed can be improved.
  • the above-mentioned determination of the layout scheme based on the similarity between the layout feature distance between each pair of elements to be laid out and the spatial distance between each pair of elements to be laid out includes: The layout scheme with the largest similarity between the layout feature distance between two elements to be laid out and the space distance between two elements to be laid out.
  • the above-mentioned determination of the layout scheme based on the similarity between the layout feature distance between each pair of elements to be laid out and the spatial distance between each pair of elements to be laid out includes: A layout scheme in which the similarity between the layout feature distance between the elements to be laid out and the space distance between the elements to be laid out is greater than the similarity threshold.
  • the layout between two elements to be laid out is determined by the correlation coefficient between the layout characteristic distance between two elements to be laid out and the space distance between two elements to be laid out The similarity between the feature distance and the spatial distance between each element to be laid out.
  • the similarity between the layout feature distance between two elements to be laid out and the space distance between each element to be laid out is based on the layout feature distance between two elements to be laid out and each The correlation coefficient between the spatial distances between two layout elements is determined, and the larger the correlation coefficient, the higher the similarity.
  • the correlation coefficient includes any one of the following correlation coefficients: Pearson correlation coefficient, Spearman correlation coefficient, and Kendall correlation coefficient.
  • the foregoing elements to be laid out include any one of the following elements: icons, pictures, and service cards.
  • the elements to be laid out may include icons, pictures, service cards, and the like.
  • the application icon on the desktop of the electronic device, the icon in the pull-down menu of the electronic device, the service card of the application program on the desktop of the electronic device, etc. are not limited here.
  • the electronic device acquires layout features of multiple elements to be laid out when detecting a layout instruction input by a user on the electronic device.
  • the above first aspect and various possible implementations of the above first aspect provide The layout method automatically layouts display elements in an electronic device.
  • the embodiment of the present application provides a readable medium, on which instructions are stored, and when the instructions are executed on the electronic device, the electronic device realizes the above-mentioned first aspect and various possible implementations of the above-mentioned first aspect provide Any one of the layout methods.
  • an embodiment of the present application provides an electronic device, which includes: a memory for storing instructions executed by one or more processors of the electronic device; and a processor, which is one of the processors of the electronic device , for executing the instructions stored in the memory to implement the above first aspect and any one of the layout methods provided by various possible implementations of the above first aspect.
  • Fig. 1 shows a schematic diagram of a layout of application program icons according to some embodiments of the present application
  • FIGS. 2A to 2F are schematic diagrams showing different basic principles of Gestalt theory according to some embodiments of the present application.
  • 3A and 3B are schematic diagrams showing icons arranged in different orders according to some embodiments of the present application.
  • Fig. 3C is a schematic diagram showing a visual complexity comparison of the icons arranged in Fig. 3A and Fig. 3B according to some embodiments of the present application;
  • FIGS. 4A to 4D show schematic diagrams of four different color spaces according to some embodiments of the present application.
  • Fig. 5 shows a schematic flowchart of a layout method according to some embodiments of the present application
  • 6A to 6C show a schematic diagram of a display interface of the mobile phone 100 during a process in which the user triggers the layout command of the mobile phone 100 according to some embodiments of the present application;
  • Fig. 7 shows a schematic diagram of the color distance of each icon in the display interface shown in Fig. 1 (A) according to some embodiments of the present application;
  • FIG. 8A and FIG. 8B are schematic diagrams showing spatial distances between different icons according to some embodiments of the present application.
  • Fig. 9 shows a schematic diagram of the spatial distance of each icon in the display interface shown in Fig. 1 (A) according to some embodiments of the present application;
  • Fig. 10A shows a schematic diagram of a layout scheme expressed in matrix form according to some embodiments of the present application
  • Fig. 10B shows a schematic diagram of layout positions in the display interface shown in Fig. 1 (A) according to some embodiments of the present application;
  • Fig. 11 shows a schematic diagram of a drop-down menu interface of a mobile phone 100 according to some embodiments of the present application
  • Fig. 12 shows a schematic flowchart of a layout method according to some embodiments of the present application.
  • Fig. 13 shows a schematic diagram of the display interface of the smart icon layout function of the pull-down menu interface of the account opening mobile phone 100 according to some embodiments of the present application;
  • Fig. 14 shows a schematic diagram of a semantic clustering result of icons in the drop-down menu interface shown in Fig. 11 according to some embodiments of the present application;
  • Fig. 15 shows a schematic diagram of the effect of rearranging the icons in the pull-down menu interface shown in Fig. 11 according to some embodiments of the present application;
  • Fig. 16 shows a schematic flowchart of a layout evaluation method according to some embodiments of the present application
  • Fig. 17 shows a schematic diagram of a layout evaluation scene according to some embodiments of the present application.
  • Fig. 18 shows a schematic diagram of evaluation results of a variety of different layouts according to some embodiments of the present application.
  • Fig. 19 shows a schematic structural diagram of a mobile phone 100 according to some embodiments of the present application.
  • Fig. 20 shows a schematic structural diagram of a mobile phone 100 according to some embodiments of the present application.
  • Illustrative embodiments of the present application include, but are not limited to, layout methods, readable media, and electronic devices.
  • FIG. 2A shows a schematic diagram of a proximity principle.
  • multiple circles in the rectangular area 10 are close to each other.
  • the white circles in the rectangular area 10 can be considered as a group, square
  • the white circles in area 11 form a group, and the white circles in square area 13 form a group.
  • FIG. 2B shows a schematic diagram of a similarity principle. Referring to FIG. 2B , based on the similarity principle, it can be considered that a row with all white circles is a group, and a row with all black circles is a group.
  • Fig. 2C shows a schematic diagram of a principle of closure.
  • the illustrated multi-segment arc is a part of the circle 14, and it can be considered that the illustrated three-segment arc is a group, similarly, a plurality of line segments All of the polylines/polylines are part of the rectangle 15, so the five line segments/polylines in the illustration can be considered as a group.
  • FIG. 2D shows a schematic diagram of a symmetry principle.
  • "[" and “]” in the figure are a group
  • " ⁇ " and “ ⁇ ” are a group
  • FIG. 2E shows a schematic diagram of a common fate principle.
  • the objects in the illustrated area 16 have the same movement tendency, such as moving to the left, and the objects in other parts have different movement patterns, then The objects in the area 16 can be considered as a group.
  • Fig. 2F shows a schematic diagram of a continuity principle, with reference to Fig. 2F, although the middle part of the key pattern 18 in the figure is blocked by the key pattern 17, it is still considered that the key pattern 18 is a complete key pattern, rather than In the middle, there is a lack of patterns covered by the key pattern 17 .
  • Interface simplicity means that the interface can be perceived by users as visual complexity. The higher the visual complexity, the lower the interface simplicity. Visual complexity is affected by the amount of information, the quality of information, and the organization of information (eg, the way elements are laid out in an interface). The amount of information and the quality of information are usually fixed in an interface, and the visual complexity can be optimized by adjusting the interface organization, such as adjusting the position of each layout element in the interface, so as to improve the simplicity of the interface.
  • the layout of the interface often embodies multiple principles of the above-mentioned Gestalt theory.
  • the visual complexity of the interface will be low; conversely, when the consistency among multiple principles embodied in the interface is low, the visual complexity of the interface will be high.
  • the icons in Figure 3A are randomly arranged, the color distribution is messy, the consistency of the proximity principle and the similarity principle is low, the visual complexity is high, and the interface simplicity is low;
  • the icons in Figure 3B are laid out according to the proximity of their colors. Icons with similar distances have similar colors. The consistency between the proximity principle and the similarity principle is high, the visual complexity is low, and the interface is simple.
  • a color space is an abstract mathematical model that describes the way a color is represented using a set of values (usually using three or four values or color components).
  • Common color spaces include but are not limited to RGB color space, HSV color space, CIE (International Commission on Illumination, CIE for short)-XYZ color space, CIE-Lab color space, etc.
  • FIG. 4A to FIG. 4D show schematic diagrams of four common color spaces.
  • the RGB color space uses three values R, G, and B to represent a color, where R represents the intensity of red (Red) in a color, G (Green) represents the intensity of green in a color, and B (Blue) Represents the intensity of blue in a color.
  • R represents the intensity of red (Red) in a color
  • G Green
  • B represents the intensity of green in a color
  • B Blue
  • 0 to 255 can be used to represent the intensity of R, G, and B in a color.
  • (0, 0, 255) represents blue
  • (255, 255, 255) represents white.
  • (0, 255, 0) means green
  • (255, 0, 0) means red.
  • the HSV color space uses three values H, S, and V to represent a color, where H (Hue, hue) is measured by angle, and the value range is 0° to 360°, starting from red in a counterclockwise direction
  • H Hue, hue
  • S saturated, saturation
  • the larger the proportion of spectral color the higher the degree of color close to spectral color, and the higher the saturation of color.
  • the saturation is high, and the color is deep and vivid.
  • the white light component of the spectral color is 0, and the saturation reaches the highest.
  • the value ranges from 0 to 1, the larger the value, the more saturated the color;
  • V (Value, lightness) indicates the degree of brightness of the color, for the light source color, the lightness value is related to the brightness of the illuminant; for the object color, this value and
  • the transmittance or reflectance of the object is related.
  • values range from 0 (black) to 1 (white). For example, (-, 0, 1) represents white, (-, 0, 0) represents black, and (0°, 1, 1) represents red.
  • Figure 4C shows the projection of the chromaticity diagram of the CIE-XYZ color space on the XY plane. Similar to the RGB color space, CIE-XYZ also uses three values X, Y, and Z to represent a color, which represents a color in red The proportion of primary colors, Y indicates the proportion of green primary color in a color, and Z indicates the proportion of blue primary color in a color.
  • Figure 4D shows a schematic diagram of the CIE-Lab color space.
  • CIE-Lab uses three values L, a, and b to represent a color.
  • L represents brightness
  • the positive extreme value of a represents red
  • the negative extreme value represents Green
  • the positive extreme value of b represents yellow
  • the negative extreme value represents blue.
  • the value range of L is 0 ⁇ 100
  • the value range of a is -128 ⁇ 127
  • the value range of b is -128 ⁇ 127.
  • (100, -, -) means white
  • ( 0, -, -) means black.
  • the color distance (that is, the color difference) is used to characterize the degree of difference between two colors, and the color difference can be calculated by the Euclidean distance in the color space.
  • formulas (1) to formulas (4) show Figure 4A
  • the color distance ⁇ E between two colors (R 1 , G 1 , B 1 ) and (R 2 , G 2 , B 2 ) in the RGB color space can be determined by the following formula (1):
  • the color distance ⁇ E between two colors (H 1 , S 1 , V 1 ) and (H 2 , S 2 , V 2 ) in the HSV color space can be determined by the following formula (2):
  • r is the radius of the base of the cone in the HSV color space
  • h is the height of the cone in the HSV color space.
  • the color distance ⁇ E between two colors (X 1 , Y 1 , Z 1 ) and (X 2 , Y 2 , Z 2 ) in the CIE-XYZ color space can be determined by the following formula (3):
  • the color distance ⁇ E between two colors (L 1 , a 1 , b 1 ) and (L 2 , a 2 , b 2 ) in the CIE-Lab color space can be determined by the following formula (4):
  • the color distance between two colors (L 1 , a 1 , b 1 ) and (L 2 , a 2 , b 2 ) in the CIE-Lab color space can also be calculated in other ways.
  • the color distance ⁇ E 94 of the two colors can be determined based on the CIE 94 color distance calculation formula (5) proposed by CIE.
  • the color distance ⁇ E CMC (I:c) between the two colors can also be calculated based on the CMC (I:c) color distance calculation formula (6) proposed by the Society's Color Measurement Committee ( CMC):
  • ⁇ L, ⁇ C ab , ⁇ H ab are defined the same as in the aforementioned formula (5), and will not be repeated here, and:
  • the color distance can also be calculated based on other distances in the two color color spaces, such as Manhattan distance, Chebyshev distance, Minkowski distance, standardized Euclidean distance, Mahalanobis distance, angle cosine, Hamming
  • Manhattan distance Chebyshev distance
  • Minkowski distance standardized Euclidean distance
  • Mahalanobis distance angle cosine
  • Hamming The distance and the like are not limited here.
  • the two colors can be converted into the same color space before calculation.
  • the user in order to improve the aesthetics of the interface displayed by the electronic device, reduce the visual complexity of the interface, etc., the user usually manually arranges multiple display elements in a certain order, such as according to the similarity of colors, the category of each display element, etc. etc., but it is cumbersome for users to manually arrange multiple display elements, which affects user experience.
  • the electronic device can obtain the layout features between the elements to be laid out in the interface, such as color features, semantic features, etc., and determine the layout feature distance between the elements to be laid out, such as color distance, semantic distance, etc., and then determine the layout scheme that makes the spatial distance between the elements to be laid out and the layout feature distance between the elements to be laid out have the highest similarity based on the layout feature distance between the elements to be laid out, and according to the determined
  • the layout scheme of is used to automatically layout the elements to be laid out. In this way, the electronic device can automatically layout the elements to be laid out according to the layout characteristics among the elements to be laid out, which improves the user experience.
  • the layout interface embodied according to the layout scheme is reflected in the aforementioned Gestalt theory.
  • the principle of similarity and proximity is highly consistent, which makes the visual complexity of the interface after the layout low and the interface simple, which further improves the user experience.
  • the electronic device 100 can obtain the main colors of each icon, and determine the color distance between the main colors of each icon, and then use an optimization algorithm, such as a genetic algorithm, to determine the color distance between the main colors of each icon.
  • the layout scheme with the highest similarity between the color distance between the main colors and the space distance between each icon and automatically adjust the position of each icon according to the determined layout scheme.
  • the electronic device 100 can automatically arrange the icons in the electronic device 100 according to the main color of each icon, which improves the user's experience of adjusting the interface layout of the electronic device 100 .
  • the main color of the icon is the main color of the icon, for example, the main color of the icon may be the most used color of the icon, or the average value of the most used color of a category.
  • the electronic device 100 can obtain the color values (such as RGB values) of each pixel of the icon, use a clustering method, such as a spectral clustering algorithm, to group pixels with similar color values into the same class, and use The color corresponding to the average value of the color values of the pixels of the category with the largest number of pixels is used as the main color of the icon.
  • the average value of the multiple main colors can be calculated as the color distance from the main colors of other icons as the color distance between the icon and other icons.
  • the electronic device 100 to which the embodiment of the present application is applicable includes, but is not limited to, a mobile phone, a laptop computer, a smart TV, a tablet computer, a server, a head-mounted display, a mobile email device, a portable game console, a portable music player,
  • the reader device and the like are not limited in this embodiment of the application.
  • the electronic device 100 is the mobile phone 100, combined with the scene shown in FIG. 1, taking the icon in the display interface of the mobile phone 100 as the element to be laid out, and the layout feature as the color feature as an example, the technical solution of the embodiment of the present application is introduced in detail.
  • FIG. 5 shows a schematic flowchart of a layout method according to some embodiments of the present application.
  • the subject of execution of the method is the mobile phone 100, as shown in Figure 5, the process includes the following steps:
  • S501 A layout instruction is detected. That is, when the mobile phone 100 detects the user's layout instruction, it automatically layouts the icons in the display interface of the mobile phone 100 through the layout method provided by the embodiment of the present application.
  • the user can enter the desktop layout adjustment interface as shown in FIG. 6B after performing the pinching operation on the touch screen of the mobile phone 100 as shown in FIG. 61 Enter the desktop setting interface shown in FIG. 6C and open the "Smart Arrange Icons" control 62.
  • the mobile phone 100 After the mobile phone 100 detects that the user has opened the "Smart Arrange Icons” control 62, the mobile phone 100 can be arranged according to the layout method provided by the embodiment of the present application. The icons in the display interface are automatically laid out.
  • the user can also shake the mobile phone 100 in the layout adjustment interface shown in FIG. 6B .
  • the mobile phone 100 After the mobile phone 100 detects the user's shaking operation, it can display the interface on the mobile phone 100 according to the layout method provided by the embodiment of the present application.
  • the user's layout instruction detected by the mobile phone 100 may also be other instructions, such as voice instructions, etc., which is not limited here.
  • S502 Obtain color data of icons to be laid out, and determine a color distance between icons to be laid out based on the color data. That is, the mobile phone 100 obtains the color data of the icons to be laid out in the display interface, such as the main color of the icons to be laid out, and determines the color distance between two icons to be laid out based on the color data of the icons to be laid out.
  • the mobile phone 100 can extract the color data of each icon to be laid out, and determine the color distance between the icons to be laid out according to any color distance calculation method shown in the aforementioned formula (1) to formula (6) .
  • the distance between the main colors of each icon can be calculated by the aforementioned formula (1), and the size shown in Figure 7 is
  • the 17 ⁇ 17 color distance matrix represents the color distance between icons.
  • the color distance between the "Gallery Icon” and the "Sports Health” icon is the value corresponding to the 4th row, 11th column or the 11th row, 4th column in the matrix shown in Figure 7, which is 31;
  • the "Music” icon and the " The color distance between the "Smart Life” icons is the value corresponding to row 8, column 14 or row 14, column 8 in the matrix shown in Figure 7, which is 4.
  • the color distance between the icon and itself is 0.
  • the elements located on the diagonal of the matrix that is, the values corresponding to the elements with the same row and column values are the color distance between the icon and itself.
  • the value of the second row and the second column of the matrix in FIG. 7 corresponds to the color distance between the "clock" icon and itself, which is 0.
  • the average value of the multiple main colors can be calculated as the color distance from the main colors of other icons as the color distance between the icon and other icons.
  • the mobile phone 100 can determine the color distance between the icons to be laid out based on the method shown in any one of the foregoing formulas (1) to formulas (6); in other embodiments, it can also use Computing in other ways, such as dividing the hue into multiple regions, the color distance between colors belonging to the same hue is 0, and the color distance between colors belonging to different hues is 1, which is not limited here.
  • the average value of the multiple main colors can be calculated, and the color between the icon and other icons can be determined according to the hue of the color corresponding to the average value distance.
  • S503 Determine a layout scheme that maximizes the similarity between the spatial distance and the color distance between the icons to be laid out. That is, the mobile phone 100 simulates and adjusts the positions of the icons to be laid out, and determines a layout scheme that maximizes the similarity of the spatial distance and the color distance between the icons to be laid out.
  • the determined layout scheme that maximizes the similarity of the spatial distance and color distance between the icons to be laid out makes the interface after the layout more consistent with the principle of similarity and proximity in the aforementioned Gestalt theory, As a result, the visual complexity of the interface after the layout is lower, and the interface is more concise.
  • the spatial distance between the icons to be laid out can be determined in the following manner: the interface to be laid out is divided into a plurality of grids, each grid is a step size D, and the grid spaced between two icons The product of the grid number and the step size D is used as the spatial distance between the two icons.
  • the space distance between the "Smooth Call” icon and the "Clock” icon and “File Management” icon is D
  • the space distance between the "Smooth Call” icon and the "Calendar” icon and "Huawei Video” icon is 2D. .
  • the spatial distance between the icons to be laid out may also be the actual distance between the centers of the icons to be laid out, such as the actual pixel distance of the icons to be laid out on the display screen 194 of the mobile phone 100, etc., for example, refer to FIG. 8B , the space distance between the "Smooth Call” icon and the "Clock” icon is 416 pixels, and the space distance between the "Smooth Call” icon and the "Browser” icon is 1096 pixels. It can be understood that, in some other embodiments, the spatial distance between the icons to be laid out may also be determined in other ways, which is not limited here.
  • the spatial distance between the icons to be laid out may be represented by a matrix, so as to determine the similarity between the space distance and the color distance between the icons to be laid out.
  • a spatial distance matrix with a size of 17 ⁇ 17 may be used to represent the spatial distance between icons in the scene shown in (A) of FIG. 1 .
  • the spatial distance between the icon of "Smooth Call” and the icon of "Calculator” is the value corresponding to row 1, column 9 or row 9, column 1 of the matrix shown in Figure 9, which is 826 pixels; another example, " The spatial distance between the “Memo” icon and the “Recorder” icon is the value corresponding to row 5, column 16 or row 16, column 5 in the matrix shown in FIG. 9 , that is, 1109 pixels.
  • the spatial distance between the icon and itself is 0.
  • the elements located on the diagonal of the matrix that is, the values corresponding to the elements with the same row and column values are the spatial distance between the icon and itself.
  • the value in the twelfth row and the twelfth column in the matrix shown in FIG. 9 corresponds to the distance between the "weather" icon and itself, which is 0.
  • the similarity between the spatial distance and the color distance between the icons to be laid out may be determined by the correlation coefficient of the space distance matrix and the color distance matrix between the icons to be laid out.
  • the Pearson correlation coefficient of the spatial distance matrix and the color distance matrix between the icons to be laid out can be calculated by formula (7) to determine the similarity of the space distance and the color distance between the icons to be laid out:
  • A is the space distance matrix between the icons to be laid out
  • B is the color distance matrix between the icons to be laid out
  • Cov(A, B) is the covariance of A and B
  • B is the average value of A
  • B is the average value of B.
  • R(A,B) represents the similarity of the spatial distance and color distance between the icons to be laid out, and the larger the absolute value of R(A,B), the greater the spatial distance and the color distance between the icons to be laid out. The higher the similarity.
  • the color distance matrix between the icons to be laid out is shown in Figure 7, and the space distance matrix is shown in Figure 9, then it can be determined from the formula (7) in Figure 1
  • the correlation coefficient between the icons to be laid out in the interface shown in (A) is 0.2078.
  • the correlation coefficient between the spatial distance matrix and the color distance matrix between the icons to be laid out in the layout interface shown in (B) of FIG. 1 is 0.6039.
  • the above-mentioned determination of the similarity of the spatial distance and color distance between the icons to be laid out by the Pearson correlation coefficient is just an example. In other embodiments, it can also be determined by other methods, for example, the Spearman correlation coefficient , Kendall correlation coefficient, etc., are not limited here.
  • the mobile phone 100 determines that the layout scheme that maximizes the similarity between the spatial distance and the color distance between the icons to be laid out can be transformed into an optimization problem, which is solved by linear programming, genetic algorithm, simulated annealing algorithm, etc.
  • An optimization problem, and the solution of the optimization problem is used as the layout scheme that maximizes the similarity between the spatial distance and the color distance between the icons to be laid out, which will be described in detail below, and will not be repeated here.
  • FIG. 10A shows a layout scheme in which the icons in the display interface shown in (A) of FIG. The distribution can refer to Fig. 10B.
  • the value of row i and column j is 1, which means that the i-th icon is laid out at the j-th position in FIG. 10B .
  • the value corresponding to the 5th row and 15th column is 1, it means that the 5th icon ("memo" icon) is laid out at the 15th position shown in Figure B.
  • the determined layout scheme may not be the layout scheme that maximizes the similarity between the spatial distance and the color distance between the layout icons, but the one that maximizes the similarity between the spatial distance and the color distance between the layout icons.
  • Layout schemes greater than the similarity threshold for example, layout schemes with a similarity threshold greater than 0.5, are not limited in this embodiment of the present application.
  • the mobile phone 100 adjusts the positions of the icons to be laid out according to the layout scheme determined in step S503 that maximizes the similarity between the spatial distance and the color distance between the icons to be laid out.
  • the mobile phone 100 can rearrange the icons in the display interface of the mobile phone 100 shown in (A) in FIG. 1 according to the layout scheme shown in FIG. 10A to obtain the display interface shown in (B) in FIG. 1 . It can be seen from the display interface shown in (B) in Figure 1 that the icons with similar colors are arranged in similar positions, which makes the similarity principle and proximity principle in the Gestalt theory reflected in this interface relatively consistent. High, reduces the visual complexity of the display interface of the mobile phone 100, and improves the interface simplicity of the display interface of the mobile phone 100.
  • step S503 determine the similarity between the spatial distance and the color distance between the icons to be laid out by converting the layout scheme that determines the maximum similarity between the space distance and the color distance between the icons to be laid out into an optimization problem.
  • the layout scheme with the highest degree of precision is introduced.
  • the mobile phone 100 can number the 17 icons to be laid out in the display interface shown in (A) in FIG. 1 (refer to the matrix shown in FIG.
  • the central coordinates of the icons to be laid out) are marked as the 17 layout positions shown in Figure 10B, so that the layout scheme that determines the maximum similarity between the space distance and the color distance between the icons to be laid out is converted into the maximum position shown in formula (8).
  • x ij , d ijmn and y ijmn are 0 or 1;
  • Constraint 1 indicates that only one icon can be laid out in one layout position
  • the constraint condition 2 indicates that an icon can only be placed in one layout position
  • d ijmn indicates the distance between the i-th icon and the m-th icon when the i-th icon is at the j-th layout position and the m-th icon is at the n-th layout position;
  • z im represents the spatial distance between the i-th icon and the m-th icon, that is, the constraint condition 4 is used to calculate the spatial distance between the j-th icon and the n-th icon;
  • c im represents the color distance between the i-th icon and the m-th icon
  • s im is the absolute value of the difference between the spatial distance and color distance between the i-th icon and the m-th icon (constraint condition 5), which is used to represent the spatial distance and color between icons to be laid out in a layout scheme through ⁇ s im
  • the layout scheme can be represented by a layout matrix, where the row indicates the layout position, and the column indicates the icon to be laid out, and the element in the i-th row and the j-th column in the layout matrix is 1, which means that the i-th icon Layout at the jth layout position.
  • the value of each x ij determined by the foregoing formula (8) is the value of row i and column j in the layout matrix.
  • the mobile phone 100 can automatically arrange the icons to be arranged according to the color characteristics between the icons to be arranged, which improves the user experience.
  • the spatial distance between the icons to be laid out in the determined layout scheme and the color distance between the icons to be laid out have the highest similarity
  • the similarity in the aforementioned Gestalt theory embodied in the interface after layout according to the layout scheme is the highest.
  • the consistency of the principle and the principle of proximity is high, so that the visual complexity of the interface after the layout is low, and the interface is simple, which further improves the user experience.
  • the elements to be laid out can also be semantic features.
  • Semantic features of each icon such as the type of application program/function corresponding to each icon, layout the icons.
  • layout the icons Combining with the scenario shown in FIG. 11 , the technical solutions of the embodiments of the present application are introduced below by taking the elements to be laid out as icons in the drop-down menu interface of the mobile phone 100 and the layout features as semantic features as examples.
  • FIG. 12 shows a schematic flowchart of a layout method according to some embodiments of the present application.
  • the subject of execution of the process is the mobile phone 100, as shown in FIG. 12, the method includes the following steps.
  • S1201 A layout instruction is detected. That is, when the mobile phone 100 detects the user's layout instruction, it automatically layouts the icons in the drop-down menu interface of the mobile phone 100 through the layout method provided by the embodiment of the present application.
  • the user can trigger the “edit menu” 132 of the drop-down menu interface by clicking the “edit control” 131 in the drop-down menu interface of the mobile phone 100, and then trigger the layout by turning on the “smart icon layout” switch 1321 instruction.
  • the mobile phone 100 detects that the user has turned on the "smart icon layout” switch 1321, it can automatically layout the icons in the drop-down menu interface of the mobile phone 100 according to the layout method provided by the embodiment of the present application.
  • the mobile phone 100 triggers the layout method provided in the embodiment of the present application after detecting the automatic layout gesture, operation, etc., which is not limited here.
  • S1202 Acquire semantic data of icons to be laid out, and determine a semantic distance between icons to be laid out based on the semantic data. That is, the mobile phone 100 obtains the semantic data of the icons to be laid out in the drop-down menu interface, such as the type of application/function corresponding to each icon, and determines the semantic distance between two icons to be laid out based on the semantic data.
  • the mobile phone 100 can obtain the type of application/function corresponding to the icon to be laid out from the application market program, and the semantic distance between icons corresponding to the same type of application/function is recorded as 0, and the semantic distance between icons of different types The semantic distance between the icons corresponding to the application program/function is recorded as 1.
  • the mobile phone 100 can also obtain data on icons classified by different users, use neural networks, unsupervised learning and other methods to classify (cluster) the semantics of different icons, and classify icons classified into the same category
  • the semantic distance between icons is recorded as 0, and the semantic distance between different types of icons is recorded as 1.
  • FIG. 14 shows a result of clustering icons in the pull-down menu interface shown in FIG. 11.
  • the horizontal axis represents different icons
  • the vertical axis represents the clustering threshold.
  • the clustering results are cropped according to the preset clustering threshold, and the icons are classified based on the number of cropped classification lines. For example, referring to FIG.
  • the mobile phone 100 can cut the clustering results with a preset clustering threshold of 15, classify icons into five categories, namely categories C1, C2, C3, C4 and C5, and classify icons belonging to the same category
  • the semantic distance between icons is recorded as 0, and the semantic distance between different types of icons is recorded as 1, such as the "WLAN” icon (that is, mywifi and the icon on the left in the figure), the "Bluetooth” icon, the “airplane mode” icon, the "mobile data " icon, "Hotspot” icon, "Huawei Sharing” icon and “5G” icon all belong to category C1, and the semantic distance between the above seven icons is recorded as 0.
  • the semantic distance between icons may also be determined in other ways, which is not limited here.
  • the semantic distance between the icons to be laid out can also be represented by a matrix.
  • the representation of the color distance in the aforementioned step S502 which will not be repeated here.
  • S1203 Determine a layout scheme that maximizes the similarity between the spatial distance and the semantic distance between the icons to be laid out. That is, the mobile phone 100 simulates and adjusts the positions of the icons to be laid out, and generates a layout scheme that maximizes the similarity between the spatial distance and the semantic distance between the icons to be laid out. For example, the mobile phone 100 may determine the similarity between the spatial distance and the semantic distance between the icons to be laid out based on the correlation coefficient of the space distance and the semantic distance between the icons to be laid out. For details, refer to the above step S503, which will not be repeated here.
  • the determined layout scheme may not be the layout scheme that maximizes the similarity between the spatial distance and the semantic distance between the icons to be laid out, but the space distance and the semantic distance between the icons to be laid out
  • FIG. 15 shows a schematic diagram of the interface after the mobile phone 100 adjusts the icons in the drop-down menu interface of the mobile phone 100 shown in FIG.
  • the icons classified into the same category in Figure 14 are arranged in adjacent positions, so that the consistency of the similarity principle and proximity principle in the aforementioned Gestalt theory embodied in this interface is relatively high, reducing the need for mobile phones.
  • the visual complexity of the display drop-down menu interface of 100 improves the interface simplicity of the drop-down menu interface of the mobile phone 100 .
  • the mobile phone 100 can automatically layout the icons to be laid out according to the semantic features between the icons to be laid out, which improves the user experience.
  • the similarity in the aforementioned Gestalt theory reflected in the interface after layout according to the layout plan The consistency of the principle and the principle of proximity is high, so that the visual complexity of the interface after the layout is low, and the interface is simple, which further improves the user experience.
  • the mobile phone 100 can also perform automatic layout based on multiple layout features at the same time.
  • the elements to be laid out are first laid out according to the semantic features, and then the layout elements that belong to the same category semantically are laid out according to the color features;
  • the vector represents the layout feature of an element to be laid out, and the distance between multidimensional vectors corresponding to each element to be laid out, such as Euclidean distance, Manhattan distance, Chebyshev distance, Minkowski distance, standardized Euclidean distance, Mahalanobis distance , included angle cosine, Hamming distance, etc. are the layout feature distances between the elements to be laid out, and then determine the layout scheme that maximizes the similarity between the layout feature distance and the space distance between the elements to be laid out, and treat the layout according to the layout scheme Elements are automatically laid out.
  • the embodiment of the present application also provides a layout evaluation method, which determines the layout feature distance between each layout element, such as color distance, semantic feature, etc.
  • the evaluation value of the layout to be evaluated is determined based on the similarity between the layout feature distance between each layout element and the spatial distance between each layout element.
  • the higher the similarity between the layout feature distance between each layout element and the spatial distance between each layout element the higher the evaluation value, the lower the visual complexity and the higher the interface simplicity of the interface corresponding to the layout to be evaluated.
  • the electronic device can adjust the layout based on the evaluation value of the layout to be evaluated to obtain a layout scheme with a higher evaluation value, that is, a layout scheme with low visual complexity and high simplicity of the interface displayed by the electronic device.
  • Fig. 16 shows a schematic flowchart of a layout evaluation method according to some embodiments of the present application.
  • the subject of execution of the process is the mobile phone 100, as shown in FIG. 16, the method includes the following steps.
  • S1601 Obtain layout features and position data between layout elements in the layout to be evaluated. That is, the mobile phone 100 obtains the layout features of each layout element in the layout to be evaluated, such as the color data, semantic data, etc. of each layout element, and the position data of each layout element, such as the center of each layout element on the display screen Pixel coordinates on 194.
  • the layout elements can be icons in the display interface shown in (A) in Figure 1
  • the layout feature can be the main feature of each icon.
  • the position data of each icon may be the grid position shown in FIG. 8A, or the pixel coordinates shown in FIG. 8B, which is not limited here.
  • the layout elements can be the icons in the drop-down menu interface shown in Figure 11
  • the layout feature can be the semantics of the application/function corresponding to each icon
  • each The location data of the icons may be the pixel coordinates of the center of each icon on the display screen of the mobile phone 100 .
  • the layout to be evaluated is layout A in which grayscale graphics 1 to grayscale graphics 8 are arranged in the order of 1, 2, ..., 7, 8, and the layout elements are graphics 1 to 8 , the layout feature is the gray value of graph 1 to graph 8.
  • S1602 Determine the layout feature distance and spatial distance of each layout element. That is, the mobile phone 100 determines the layout feature distance of each layout element according to the layout feature of each layout element, and determines the spatial distance of each layout element according to the position data of each layout element.
  • the layout feature distance between icons can be the color distance of the main color of each icon.
  • the spatial distance of each icon may be the distance between the pixel coordinates of each icon, for details, refer to the relevant description in FIG. 9 above, and will not be described in detail here.
  • the characteristic distance between the icons may be the semantic distance of each icon, for details, refer to the relevant description in Figure 14 above, which will not be repeated here; each icon The spatial distance between them may be the distance between the pixel coordinates of the center of each icon on the display screen 194 of the mobile phone 100, for details, please refer to the relevant description of the aforementioned step S1203, which will not be repeated here.
  • the layout feature distance of each layout element in layout A can be the absolute value of the gray value difference between the graphics, for example, the gray value of graphic 1 is 0, and the gray value of graphic 2 If the grayscale value is 180, then the layout feature distance between graphic 1 and graphic 2 is 180; the spatial distance of each layout element can be the distance between the graphic sorting numbers, for example, the sorting number of graphic 1 is 1, and the sorting number of graphic 4 number is 4, then the spatial distance between graphic 1 and graphic 4 is 3. It can be understood that, referring to FIG. 17 , in some embodiments, the layout feature distances and spatial distances between two figures 1 to 8 can be represented by a matrix.
  • S1603 Determine the evaluation value of the layout to be evaluated according to the similarity between the layout feature distance and the spatial distance of each layout element. That is, the mobile phone 100 determines the evaluation value of the layout to be evaluated according to the similarity between the layout feature distance and the spatial distance of each layout element in the layout to be evaluated, and the higher the similarity between the layout feature distance and the spatial distance of each layout element, the higher the evaluation value , the lower the visual complexity of the display interface corresponding to the layout to be evaluated, the higher the simplicity of the interface.
  • the correlation coefficient between the layout feature distance and the spatial distance of each layout element can be calculated based on the method in the aforementioned step S503, and the calculated correlation coefficient can be used as the evaluation value of the layout to be evaluated.
  • the evaluation value of the layout may be the correlation coefficient 0.2078 calculated in the foregoing example.
  • the evaluation value of the layout may be the correlation coefficient 0.6039 calculated in the foregoing example.
  • FIG. 18 shows layout A, layout B, layout C, layout D, layout E, and layout F arranged in different order from graph 1 to graph 8 shown in FIG. 17 and the evaluation values corresponding to each layout.
  • the evaluation values of layout A, layout B, layout C, layout D, layout E, and layout F are 0.0658, 0.2632, 0.329, 0.329, 0.5264, and 0.7237, respectively.
  • the similarity between the gray value distance and the spatial distance increases, that is, the higher the evaluation value, the lower the visual complexity and the higher the simplicity of the layout.
  • the mobile phone 100 can evaluate the visual complexity and simplicity of the interface corresponding to the layout to be evaluated, and generate an evaluation value of the layout, so that the mobile phone 100 can adjust the layout based on the evaluation value, A layout scheme with a higher evaluation value is obtained, that is, a layout scheme with low visual complexity and high simplicity of the interface displayed on the mobile phone 100 .
  • Fig. 19 shows a schematic structural diagram of a mobile phone 100 according to some embodiments of the present application.
  • the mobile phone 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (universal serial bus, USB) interface 130, a charging management module 140, a power management module 141, a battery 142, Antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, audio module 170, speaker 170A, receiver 170B, microphone 170C, earphone interface 170D, sensor module 180, button 190, motor 191, indicator 192, camera 193, A display screen 194, and a subscriber identification module (subscriber identification module, SIM) card interface 195, etc.
  • the sensor module 180 may include a pressure sensor 180A, an acceleration sensor 180E, a fingerprint sensor 180H, a touch sensor 180K, an ambient light sensor 180L and the like.
  • the structure shown in the embodiment of the present application does not constitute a specific limitation on the mobile phone 100 .
  • the mobile phone 100 may include more or fewer components than shown in the figure, or combine certain components, or separate certain components, or arrange different components.
  • the illustrated components can be realized in hardware, software or a combination of software and hardware.
  • the processor 110 may include one or more processing units, for example: the processor 110 may include an application processor (application processor, AP), a modem processor, a graphics processing unit (graphics processing unit, GPU), an image signal processor (image signal processor, ISP), controller, video codec, digital signal processor (digital signal processor, DSP), baseband processor, and/or neural network processor (neural-network processing unit, NPU), etc. Wherein, different processing units may be independent devices, or may be integrated in one or more processors.
  • application processor application processor, AP
  • modem processor graphics processing unit
  • GPU graphics processing unit
  • image signal processor image signal processor
  • ISP image signal processor
  • controller video codec
  • digital signal processor digital signal processor
  • baseband processor baseband processor
  • neural network processor neural-network processing unit
  • the controller can generate an operation control signal according to the instruction opcode and timing signal, and complete the control of fetching and executing the instruction.
  • a memory may also be provided in the processor 110 for storing instructions and data.
  • the memory in processor 110 is a cache memory.
  • the memory may hold instructions or data that the processor 110 has just used or recycled. If the processor 110 needs to use the instruction or data again, it can be called directly from the memory. Repeated access is avoided, and the waiting time of the processor 110 is reduced, thereby improving the efficiency of the system.
  • the processor 110 may call and execute execution instructions of the layout method/layout evaluation method provided by the embodiments of the present application stored in the memory, so as to implement the layout method/layout evaluation method provided by the embodiments of the present application.
  • processor 110 may include one or more interfaces.
  • the interface may include an integrated circuit (inter-integrated circuit, I2C) interface, an integrated circuit built-in audio (inter-integrated circuit sound, I2S) interface, a pulse code modulation (pulse code modulation, PCM) interface, a universal asynchronous transmitter (universal asynchronous receiver/transmitter, UART) interface, mobile industry processor interface (mobile industry processor interface, MIPI), general-purpose input and output (general-purpose input/output, GPIO) interface, subscriber identity module (subscriber identity module, SIM) interface, and /or universal serial bus (universal serial bus, USB) interface, etc.
  • I2C integrated circuit
  • I2S integrated circuit built-in audio
  • PCM pulse code modulation
  • PCM pulse code modulation
  • UART universal asynchronous transmitter
  • MIPI mobile industry processor interface
  • GPIO general-purpose input and output
  • subscriber identity module subscriber identity module
  • SIM subscriber identity module
  • USB universal serial bus
  • the I2C interface is a bidirectional synchronous serial bus, including a serial data line (serial data line, SDA) and a serial clock line (derail clock line, SCL).
  • processor 110 may include multiple sets of I2C buses.
  • the processor 110 can be respectively coupled to the touch sensor 180K, the charger, the flashlight, the camera 193 and the like through different I2C bus interfaces.
  • the processor 110 may be coupled to the touch sensor 180K through the I2C interface, so that the processor 110 and the touch sensor 180K communicate through the I2C bus interface to realize the touch function of the mobile phone 100 .
  • the I2S interface can be used for audio communication.
  • processor 110 may include multiple sets of I2S buses.
  • the processor 110 may be coupled to the audio module 170 through an I2S bus to implement communication between the processor 110 and the audio module 170 .
  • the audio module 170 can transmit audio signals to the wireless communication module 160 through the I2S interface, so as to realize the function of answering calls through the Bluetooth headset.
  • the PCM interface can also be used for audio communication, sampling, quantizing and encoding the analog signal.
  • the audio module 170 and the wireless communication module 160 may be coupled through a PCM bus interface.
  • the audio module 170 can also transmit audio signals to the wireless communication module 160 through the PCM interface, so as to realize the function of answering calls through the Bluetooth headset. Both the I2S interface and the PCM interface can be used for audio communication.
  • the UART interface is a universal serial data bus used for asynchronous communication.
  • the bus can be a bidirectional communication bus. It converts the data to be transmitted between serial communication and parallel communication.
  • a UART interface is generally used to connect the processor 110 and the wireless communication module 160 .
  • the processor 110 communicates with the Bluetooth module in the wireless communication module 160 through the UART interface to realize the Bluetooth function.
  • the audio module 170 can transmit audio signals to the wireless communication module 160 through the UART interface, so as to realize the function of playing music through the Bluetooth headset.
  • the MIPI interface can be used to connect the processor 110 with peripheral devices such as the display screen 194 and the camera 193 .
  • MIPI interface includes camera serial interface (camera serial interface, CSI), display serial interface (display serial interface, DSI), etc.
  • the processor 110 communicates with the camera 193 through the CSI interface to realize the shooting function of the mobile phone 100 .
  • the processor 110 communicates with the display screen 194 through the DSI interface to realize the display function of the mobile phone 100 .
  • the GPIO interface can be configured by software.
  • the GPIO interface can be configured as a control signal or as a data signal.
  • the GPIO interface can be used to connect the processor 110 with the camera 193 , the display screen 194 , the wireless communication module 160 , the audio module 170 , the sensor module 180 and so on.
  • the GPIO interface can also be configured as an I2C interface, I2S interface, UART interface, MIPI interface, etc.
  • the USB interface 130 is an interface conforming to the USB standard specification, specifically, it can be a Mini USB interface, a Micro USB interface, a USB Type C interface, and the like.
  • the USB interface 130 can be used to connect a charger to charge the mobile phone 100, and can also be used to transmit data between the mobile phone 100 and peripheral devices. It can also be used to connect headphones and play audio through them. This interface can also be used to connect other electronic devices, such as AR devices.
  • the charging management module 140 is configured to receive a charging input from a charger. While the charging management module 140 is charging the battery 142 , it can also supply power to the electronic device through the power management module 141 .
  • the power management module 141 is used for connecting the battery 142 , the charging management module 140 and the processor 110 .
  • the power management module 141 receives the input from the battery 142 and/or the charging management module 140 to provide power for the processor 110 , the internal memory 121 , the display screen 194 , the camera 193 , and the wireless communication module 160 .
  • the wireless communication function of the mobile phone 100 can be realized by the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.
  • Antenna 1 and Antenna 2 are used to transmit and receive electromagnetic wave signals.
  • the mobile communication module 150 can provide wireless communication solutions including 2G/3G/4G/5G applied on the mobile phone 100 .
  • the mobile communication module 150 may include at least one filter, switch, power amplifier, low noise amplifier (low noise amplifier, LNA) and the like.
  • the mobile communication module 150 can receive electromagnetic waves through the antenna 1, filter and amplify the received electromagnetic waves, and send them to the modem processor for demodulation.
  • the mobile communication module 150 can also amplify the signal modulated by the modem processor, convert it into electromagnetic wave and radiate it through the antenna 1 .
  • at least part of the functional modules of the mobile communication module 150 may be set in the processor 110 .
  • at least part of the functional modules of the mobile communication module 150 and at least part of the modules of the processor 110 may be set in the same device.
  • the wireless communication module 160 can provide wireless local area networks (wireless local area networks, WLAN) (such as wireless fidelity (Wireless Fidelity, Wi-Fi) network), bluetooth (bluetooth, BT), global navigation satellite system, etc. applied on the mobile phone 100 (global navigation satellite system, GNSS), frequency modulation (frequency modulation, FM), near field communication technology (near field communication, NFC), infrared technology (infrared, IR) and other wireless communication solutions.
  • the wireless communication module 160 may be one or more devices integrating at least one communication processing module.
  • the wireless communication module 160 receives electromagnetic waves via the antenna 2 , frequency-modulates and filters the electromagnetic wave signals, and sends the processed signals to the processor 110 .
  • the wireless communication module 160 can also receive the signal to be sent from the processor 110 , frequency-modulate it, amplify it, and convert it into electromagnetic waves through the antenna 2 for radiation.
  • the mobile phone 100 realizes the display function through the GPU, the display screen 194, and the application processor.
  • the GPU is a microprocessor for image processing, and is connected to the display screen 194 and the application processor. GPUs are used to perform mathematical and geometric calculations for graphics rendering.
  • Processor 110 may include one or more GPUs that execute program instructions to generate or change display information.
  • the display screen 194 is used to display images, videos and the like.
  • the display screen 194 includes a display panel.
  • the display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active matrix organic light emitting diode or an active matrix organic light emitting diode (active-matrix organic light emitting diode, AMOLED), flexible light-emitting diode (flex light-emitting diode, FLED), Mini-LED, Micro-LED, Micro-OLED, quantum dot light emitting diodes (quantum dot light emitting diodes, QLED), etc.
  • the mobile phone 100 may include 1 or N display screens 194, where N is a positive integer greater than 1.
  • Camera 193 is used to capture still images or video.
  • the object generates an optical image through the lens and projects it to the photosensitive element.
  • the photosensitive element may be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor.
  • CMOS complementary metal-oxide-semiconductor
  • the photosensitive element converts the light signal into an electrical signal, and then transmits the electrical signal to the ISP to convert it into a digital image signal.
  • the ISP outputs the digital image signal to the DSP for processing.
  • DSP converts digital image signals into standard RGB, YUV and other image signals.
  • the mobile phone 100 may include 1 or N cameras 193, where N is a positive integer greater than 1.
  • the external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the mobile phone 100.
  • the external memory card communicates with the processor 110 through the external memory interface 120 to implement a data storage function. Such as saving music, video and other files in the external memory card.
  • the internal memory 121 may be used to store computer-executable program codes including instructions.
  • the internal memory 121 may include an area for storing programs and an area for storing data.
  • the stored program area can store an operating system, at least one application program required by a function (such as a sound playing function, an image playing function, etc.) and the like.
  • the storage data area can store data (such as audio data, phone book, etc.) created during the use of the mobile phone 100 .
  • the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one magnetic disk storage device, flash memory device, universal flash storage (universal flash storage, UFS) and the like.
  • the processor 110 executes various functional applications of the mobile phone 100 by executing instructions stored in the internal memory 121 and/or instructions stored in a memory provided in the processor 110 .
  • the mobile phone 100 can realize the audio function through the audio module 170 , the speaker 170A, the receiver 170B, the microphone 170C, the earphone interface 170D, and the application processor. Such as music playback, recording, etc.
  • the audio module 170 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signal.
  • the audio module 170 may also be used to encode and decode audio signals.
  • Speaker 170A also referred to as a "horn" is used to convert audio electrical signals into sound signals.
  • Receiver 170B also called “earpiece” is used to convert audio electrical signals into sound signals.
  • the microphone 170C also called “microphone” or “microphone”, is used to convert sound signals into electrical signals.
  • the earphone interface 170D is used for connecting wired earphones.
  • the pressure sensor 180A is used to sense the pressure signal and convert the pressure signal into an electrical signal.
  • pressure sensor 180A may be disposed on display screen 194 .
  • pressure sensors 180A such as resistive pressure sensors, inductive pressure sensors, and capacitive pressure sensors.
  • a capacitive pressure sensor may be comprised of at least two parallel plates with conductive material.
  • touch operations acting on the same touch position but with different touch operation intensities may correspond to different operation instructions. For example: when a touch operation whose intensity is less than the first pressure threshold is applied to the “smart icon layout” switch 1321 shown in FIG. When the touch operation of the pressure threshold is applied to the “smart icon layout” switch 1321 shown in FIG. 13 , the instructions arranged according to the color characteristics are executed.
  • the acceleration sensor 180E can detect the acceleration of the mobile phone 100 in various directions (generally three axes). When the mobile phone 100 is stationary, the magnitude and direction of gravity can be detected. It can also be used to identify the posture of electronic devices, and can be used in applications such as horizontal and vertical screen switching, pedometers, etc.
  • the ambient light sensor 180L is used for sensing ambient light brightness.
  • the mobile phone 100 can adaptively adjust the brightness of the display screen 194 according to the perceived ambient light brightness.
  • the ambient light sensor 180L can also be used to automatically adjust the white balance when taking pictures.
  • the ambient light sensor 180L can also cooperate with the proximity light sensor 180G to detect whether the mobile phone 100 is in the pocket, so as to prevent accidental touch.
  • the fingerprint sensor 180H is used to collect fingerprints.
  • the mobile phone 100 can use the collected fingerprint features to realize fingerprint unlocking, access to the application lock, take pictures with the fingerprint, answer calls with the fingerprint, and the like.
  • the main space and the private space may use different unlocking fingerprints for verification and login, so as to ensure the security of private data in the private space.
  • the user can verify different fingerprints on the mobile phone 100 to enter the main space and the private space respectively.
  • the touch sensor 180K is also called “touch device”.
  • the touch sensor 180K can be disposed on the display screen 194, and the touch sensor 180K and the display screen 194 form a touch screen, also called a “touch screen”.
  • the touch sensor 180K is used to detect a touch operation on or near it.
  • the touch sensor can pass the detected touch operation to the application processor to determine the type of touch event.
  • Visual output related to the touch operation can be provided through the display screen 194 .
  • the touch sensor 180K may also be disposed on the surface of the mobile phone 100 , which is different from the position of the display screen 194 .
  • the motor 191 can generate a vibrating reminder.
  • the indicator 192 can be an indicator light, and can be used to indicate charging status, power change, and can also be used to indicate messages, missed calls, notifications, and the like.
  • the SIM card interface 195 is used for connecting a SIM card.
  • the operating system used by the above-mentioned mobile phone 100 may be one of Android TM , IOS TM (Input Output System) and Harmony OS (Harmony OS), which is not limited here.
  • IOS TM Input Output System
  • Harmony OS Harmony OS
  • Fig. 20 shows a schematic diagram of a software architecture of a mobile phone 100 according to some embodiments of the present application.
  • the software architecture of the mobile phone 100 mainly includes:
  • Application layer 20 may include system applications 21 and extension applications 22 (or third-party applications).
  • the system application 21 may include a desktop 211, a setting 212, and the like.
  • the layout method/layout evaluation method provided by each embodiment of the present application can be executed by the desktop application, that is, the desktop 211 can obtain the layout characteristics of the icons in the display interface of the mobile phone 100 or in the drop-down menu, such as color characteristics and semantics.
  • the desktop 211 can obtain the layout characteristics of the icons in the display interface of the mobile phone 100 or in the drop-down menu, such as color characteristics and semantics.
  • the framework layer 30 provides a multilingual framework for the application layer, including an interface (User Interface, UI) framework 31, a user program framework 32, and a capability framework 33.
  • UI User Interface
  • the UI framework 31 includes a window manager, a content provider, a view system, a phone manager, a resource manager, a notification manager, etc., which will not be repeated here.
  • the user program framework 32 and the capability framework 33 can provide application programs with the capabilities of various capability components required by the application, such as computing power (may include CPU computing power, graphics processing unit (Graphics Processing Unit, GPU) computing power, image signal processor (Image Signal Processor, ISP) computing power, etc.), sound pickup capabilities (including microphone pickup capabilities, voice recognition capabilities, etc.), security capabilities in equipment security protection (including trusted operating environment security levels, etc.), display capabilities (can include screen resolution, screen size, etc.), playback capability (including sound amplification capability, stereo effect capability, etc., and storage capability (may include device memory capability, random access memory (RAM) capability, etc.) etc., no limitation here.
  • computing power may include CPU computing power, graphics processing unit (Graphics Processing Unit, GPU) computing power, image signal processor (Image Signal Processor, ISP) computing power, etc.
  • sound pickup capabilities including microphone pickup capabilities, voice recognition capabilities, etc.
  • security capabilities in equipment security protection including trusted operating environment security levels, etc.
  • display capabilities can include screen resolution, screen size, etc.
  • the system service layer 40 is the core of the software system of the mobile phone 100 and can provide services to the application programs of the application layer 20 through the framework layer 30 .
  • the system service layer 40 includes a distributed soft bus 41, a distributed data management module 42, a distributed task scheduling module 43, and the like.
  • the distributed soft bus 41 is used to couple the mobile phone 10 with other electronic devices to form a distributed system;
  • the distributed data management module 42 is based on the distributed soft bus 41 to realize distributed management of application program data and user data.
  • the distributed task scheduling module 43 builds a unified distributed service management (discovery, synchronization, registration, call) mechanism based on technical characteristics such as distributed soft bus and distributed data management, and supports remote startup and remote call of cross-device applications , remote connection, migration and other operations, can select the appropriate device to run tasks according to the capabilities, locations, business operation status, resource usage, and user habits and intentions of different devices.
  • distributed service management discovery, synchronization, registration, call
  • the kernel layer 50 includes a kernel subsystem 51 and a driver subsystem 52 .
  • the kernel subsystem 51 provides basic kernel capabilities to the upper layer by shielding differences between multiple kernels, including process/thread management, memory management, file system, network management, and peripheral management.
  • the driver subsystem 52 includes a hardware driver framework, which can provide the mobile phone 100 with a unified peripheral device access capability and a management framework.
  • the software architecture of the mobile phone 100 shown in FIG. 20 is just an example.
  • the mobile phone 10 may also adopt other software architectures different from the above-mentioned FIG. 20 , which is not limited in this embodiment of the present application.
  • Embodiments of the mechanisms disclosed in this application may be implemented in hardware, software, firmware, or a combination of these implementation methods.
  • Embodiments of the present application may be implemented as a computer program or program code executed on a programmable system comprising at least one processor, a storage system (including volatile and non-volatile memory and/or storage elements) , at least one input device, and at least one output device.
  • Program code can be applied to input instructions to perform the functions described herein and to generate output information.
  • the output information may be applied to one or more output devices in known manner.
  • a processing system includes any system having a processor such as, for example, a digital signal processor (DSP), microcontroller, application specific integrated circuit (ASIC), or microprocessor.
  • DSP digital signal processor
  • ASIC application specific integrated circuit
  • the program code can be implemented in a high-level procedural language or an object-oriented programming language to communicate with the processing system.
  • Program code can also be implemented in assembly or machine language, if desired.
  • the mechanisms described in this application are not limited in scope to any particular programming language. In either case, the language may be a compiled or interpreted language.
  • the disclosed embodiments may be implemented in hardware, firmware, software, or any combination thereof.
  • the disclosed embodiments can also be implemented as instructions carried by or stored on one or more transitory or non-transitory machine-readable (e.g., computer-readable) storage media, which can be executed by one or more processors read and execute.
  • instructions may be distributed over a network or via other computer-readable media.
  • a machine-readable medium may include any mechanism for storing or transmitting information in a form readable by a machine (e.g., a computer), including, but not limited to, floppy disks, optical disks, optical disks, read-only memories (CD-ROMs), magnetic Optical discs, read-only memory (ROM), random-access memory (RAM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), magnetic or optical cards, flash memory, or A tangible, machine-readable memory used to transmit information (eg, carrier waves, infrared signals, digital signals, etc.) by electrical, optical, acoustic, or other forms of propagating signals using the Internet.
  • a machine-readable medium includes any type of machine-readable medium suitable for storing or transmitting electronic instructions or information in a form readable by a machine (eg, a computer).
  • each unit/module mentioned in each device embodiment of this application is a logical unit/module.
  • a logical unit/module can be a physical unit/module, or a physical unit/module.
  • a part of the module can also be realized with a combination of multiple physical units/modules, the physical implementation of these logical units/modules is not the most important, the combination of functions realized by these logical units/modules is the solution The key to the technical issues raised.
  • the above-mentioned device embodiments of this application do not introduce units/modules that are not closely related to solving the technical problems proposed by this application, which does not mean that the above-mentioned device embodiments do not exist other units/modules.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • User Interface Of Digital Computer (AREA)

Abstract

La présente demande se rapporte au domaine technique de l'électronique. Un procédé de disposition, un support lisible et un dispositif électronique sont divulgués. Le procédé comprend les étapes consistant à : acquérir des caractéristiques de disposition d'une pluralité d'éléments à disposer ; sur la base des caractéristiques de disposition desdits éléments, déterminer une distance de caractéristiques de disposition entre deux desdits éléments, la distance de caractéristiques de disposition entre lesdits éléments étant d'autant plus petite que la similarité entre les caractéristiques de disposition desdits éléments est grande ; déterminer un schéma de disposition sur la base de la similarité entre la distance de caractéristiques de disposition entre deux quelconques desdits éléments et une distance spatiale ces deux éléments ; et ajuster les positions desdits éléments selon le schéma de disposition déterminé. La cohérence du principe de similarité et du principe de proximité dans la théorie de la Gestalt matérialisée par une interface qui a été disposée au moyen d'un schéma de disposition déterminé par le procédé décrit dans la présente demande est relativement grande, de telle sorte que la complexité visuelle d'une interface d'affichage d'un dispositif électronique disposée au moyen du schéma de disposition peut être réduite, ce qui permet d'améliorer la simplicité d'interface de l'interface d'affichage, et d'améliorer l'expérience utilisateur.
PCT/CN2022/126443 2021-11-24 2022-10-20 Procédé de disposition, support lisible et dispositif électronique WO2023093390A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202111405667.6A CN116166155A (zh) 2021-11-24 2021-11-24 布局方法、可读介质和电子设备
CN202111405667.6 2021-11-24

Publications (1)

Publication Number Publication Date
WO2023093390A1 true WO2023093390A1 (fr) 2023-06-01

Family

ID=86416870

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/126443 WO2023093390A1 (fr) 2021-11-24 2022-10-20 Procédé de disposition, support lisible et dispositif électronique

Country Status (2)

Country Link
CN (1) CN116166155A (fr)
WO (1) WO2023093390A1 (fr)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116627293A (zh) * 2023-07-26 2023-08-22 荣耀终端有限公司 一种桌面内容整理方法及装置

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20130093764A1 (en) * 2011-10-18 2013-04-18 Research In Motion Limited Method of animating a rearrangement of ui elements on a display screen of an electronic device
US20130132874A1 (en) * 2011-11-22 2013-05-23 Yahoo! Inc. Automatically arranging of icons on a user interface
CN103488397A (zh) * 2013-09-13 2014-01-01 深圳市欧珀通信软件有限公司 一种图标的排列方法、装置及移动终端
CN103955332A (zh) * 2014-05-15 2014-07-30 深圳市中兴移动通信有限公司 一种移动终端及其桌面图标的排列方法和装置
CN106527853A (zh) * 2016-10-31 2017-03-22 珠海市魅族科技有限公司 一种应用图标展示方法及装置
CN106873857A (zh) * 2017-02-22 2017-06-20 Tcl集团股份有限公司 一种应用图标自动布局方法及装置

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20130093764A1 (en) * 2011-10-18 2013-04-18 Research In Motion Limited Method of animating a rearrangement of ui elements on a display screen of an electronic device
US20130132874A1 (en) * 2011-11-22 2013-05-23 Yahoo! Inc. Automatically arranging of icons on a user interface
CN103488397A (zh) * 2013-09-13 2014-01-01 深圳市欧珀通信软件有限公司 一种图标的排列方法、装置及移动终端
CN103955332A (zh) * 2014-05-15 2014-07-30 深圳市中兴移动通信有限公司 一种移动终端及其桌面图标的排列方法和装置
CN106527853A (zh) * 2016-10-31 2017-03-22 珠海市魅族科技有限公司 一种应用图标展示方法及装置
CN106873857A (zh) * 2017-02-22 2017-06-20 Tcl集团股份有限公司 一种应用图标自动布局方法及装置

Also Published As

Publication number Publication date
CN116166155A (zh) 2023-05-26

Similar Documents

Publication Publication Date Title
WO2021027649A1 (fr) Procédé de traitement d'interface d'affichage en mode sombre, dispositif électronique et support d'informations
WO2020187157A1 (fr) Procédé de commande et dispositif électronique
US12079974B2 (en) Image processing method and apparatus, and device
US9607421B2 (en) Displaying particle effect on screen of electronic device
US20220319077A1 (en) Image-text fusion method and apparatus, and electronic device
WO2022042285A1 (fr) Procédé d'affichage d'interface de programme d'application et dispositif électronique
WO2021063098A1 (fr) Procédé de réponse d'écran tactile, et dispositif électronique
US20220283610A1 (en) Electronic Device Control Method and Electronic Device
WO2021052344A1 (fr) Procédé de réglage de paramètres et dispositif électronique
WO2021147396A1 (fr) Procédé de gestion d'icônes et terminal intelligent
KR102359276B1 (ko) 전자 장치의 화이트 밸런스 기능 제어 방법 및 장치
WO2021013132A1 (fr) Procédé d'entrée et dispositif électronique
CN110401768B (zh) 调节电子设备的工作状态的方法和装置
CN110100251A (zh) 用于处理文档的设备、方法和图形用户界面
WO2022001619A1 (fr) Procédé de capture d'écran et dispositif électronique
CN114579016A (zh) 一种共享输入设备的方法、电子设备及系统
CN112887582A (zh) 一种图像色彩处理方法、装置及相关设备
WO2021000943A1 (fr) Procédé et appareil de gestion d'un commutateur à empreinte digitale
WO2021093595A1 (fr) Procédé de vérification d'identité d'utilisateur et dispositif électronique
US20230244507A1 (en) Method and Apparatus for Processing Interaction Event
WO2023142916A1 (fr) Procédé, appareil et dispositif de traitement d'image ainsi que support de stockage
WO2023093390A1 (fr) Procédé de disposition, support lisible et dispositif électronique
WO2023001043A1 (fr) Procédé d'affichage de contenu, dispositif électronique et système
WO2018098959A2 (fr) Procédé d'affichage d'image et dispositif électronique
WO2019184017A1 (fr) Procédé d'affichage de commande et dispositif électronique

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22897473

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE