CN115309299A - Desktop card display method, device, terminal, storage medium and program product - Google Patents

Desktop card display method, device, terminal, storage medium and program product Download PDF

Info

Publication number
CN115309299A
CN115309299A CN202211117854.9A CN202211117854A CN115309299A CN 115309299 A CN115309299 A CN 115309299A CN 202211117854 A CN202211117854 A CN 202211117854A CN 115309299 A CN115309299 A CN 115309299A
Authority
CN
China
Prior art keywords
card
desktop
control
cards
content
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202211117854.9A
Other languages
Chinese (zh)
Other versions
CN115309299B (en
Inventor
陈科鑫
张晓帆
董伟鑫
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Guangdong Oppo Mobile Telecommunications Corp Ltd
Original Assignee
Guangdong Oppo Mobile Telecommunications Corp Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Guangdong Oppo Mobile Telecommunications Corp Ltd filed Critical Guangdong Oppo Mobile Telecommunications Corp Ltd
Priority to CN202211117854.9A priority Critical patent/CN115309299B/en
Publication of CN115309299A publication Critical patent/CN115309299A/en
Application granted granted Critical
Publication of CN115309299B publication Critical patent/CN115309299B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0481—Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44—Arrangements for executing specific programs
    • G06F9/451—Execution arrangements for user interfaces
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Software Systems (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • User Interface Of Digital Computer (AREA)

Abstract

The embodiment of the application discloses a desktop card display method, a desktop card display device, a desktop card display terminal, a desktop card display storage medium and a desktop card program product, and belongs to the field of human-computer interaction. The method comprises the following steps: the method comprises the steps of obtaining a desktop card, wherein the desktop card is a card-like component used for information display on a desktop; performing card clustering on the desktop cards based on the card content of the desktop cards to obtain card clustering results, wherein the card clustering results comprise a plurality of card clusters; integrating the desktop cards in the same card cluster to obtain a desktop integration component, wherein the desktop integration component is an independent desktop card or a desktop card group consisting of a plurality of desktop cards; and displaying the desktop integration component. By adopting the scheme provided by the embodiment of the application, the terminal desktop cards are clustered and integrated, so that the desktop cards are more orderly displayed, and the application information pushed by the desktop cards is clearer.

Description

Desktop card display method, device, terminal, storage medium and program product
Technical Field
The embodiment of the application relates to the technical field of human-computer interaction, in particular to a desktop card display method, a desktop card display device, a desktop card display terminal, a desktop card storage medium and a program product.
Background
The intelligent terminal technology is mature, so that the information recommended by the intelligent terminal for the user is richer and more various, and the desktop card is a classic mode for presenting the information for the user in the intelligent terminal.
Under a common condition, the desktop card is arranged on a negative screen, application information is displayed for a user, the user can change the display position and the display state of the desktop card through sliding or long-time pressing operation, and after the desktop card is triggered, the desktop card can jump to an application interface to provide corresponding services for the user.
However, as the application information tends to be diversified, the number and the types of the desktop cards are gradually increased, the display interface is more complicated, the desktop card display is less organized, and the problem of difficulty in selecting the application information is not facilitated for the user to visually acquire the application information.
Disclosure of Invention
The embodiment of the application provides a desktop card display method, a desktop card display device, a desktop card display terminal, a desktop card storage medium and a desktop card program product. The technical scheme is as follows:
in one aspect, the present application provides a method for displaying a desktop card, where the method includes:
the method comprises the steps of obtaining a desktop card, wherein the desktop card is a card-type component used for information display on a desktop;
performing card clustering on the desktop card based on the card content of the desktop card to obtain a card clustering result, wherein the card clustering result comprises a plurality of card clusters;
integrating the desktop cards in the same card cluster to obtain a desktop integration component, wherein the desktop integration component is an independent desktop card or a desktop card group consisting of a plurality of desktop cards;
and displaying the desktop integration component.
In another aspect, the present application provides a desktop card display apparatus, comprising:
the desktop card acquisition module is used for acquiring a desktop card, and the desktop card is a card-based component used for information display on a desktop;
the desktop card clustering module is used for clustering cards of the desktop card based on the card content of the desktop card to obtain a card clustering result, and the card clustering result comprises a plurality of card clusters;
the integration module is used for integrating the desktop cards in the same card cluster to obtain a desktop integration component, and the desktop integration component is an independent desktop card or a desktop card group consisting of a plurality of desktop cards;
and the display module is used for displaying the desktop integration component.
In another aspect, an embodiment of the present application provides a terminal, where the terminal includes a processor and a memory; the memory stores at least one instruction for execution by the processor to implement a method of displaying a desktop card as described in the above aspect.
In another aspect, an embodiment of the present application provides a computer-readable storage medium, where at least one program code is stored, and the program code is loaded and executed by a processor to implement the desktop card display method according to the above aspect.
In another aspect, embodiments of the present application provide a computer program product including computer instructions, which are stored in a computer-readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions to cause the computer device to perform the display method of the desktop card provided in the various alternative implementations of the above aspect.
In the embodiment of the application, after the terminal acquires the desktop card, the desktop card is clustered and integrated according to the card content of the desktop card, the desktop integration component after integration is finally displayed, and the terminal can integrate the desktop card in a self-adaptive manner according to the content of the desktop card, so that the organization form of the desktop card is not limited to a fixed form or an irregular display form, and the desktop card is more favorable for clearly displaying application information to a user.
Drawings
FIG. 1 is a flowchart illustrating a method for displaying a desktop card according to an exemplary embodiment of the present application;
FIG. 2 illustrates a flow chart of a desktop card display method provided by another exemplary embodiment of the present application;
FIG. 3 illustrates a schematic diagram of desktop card integration provided by an exemplary embodiment of the present application;
FIG. 4 illustrates a schematic diagram of desktop card integration provided by another exemplary embodiment of the present application;
FIG. 5 is a flowchart illustrating a process for obtaining a desktop card provided by another exemplary embodiment of the present application;
FIG. 6 is a schematic diagram illustrating a process for obtaining card content provided by an exemplary embodiment of the present application;
FIG. 7 shows a flowchart of a process for generating a card feature vector provided by another exemplary embodiment of the present application;
FIG. 8 is a diagram illustrating a determination of a first key control provided by an exemplary embodiment of the present application;
FIG. 9 is a diagram illustrating a determination of a second key control provided by an exemplary embodiment of the present application;
fig. 10 is a schematic structural diagram illustrating that a terminal obtains an association degree of an application interface control according to an exemplary embodiment of the present application;
FIG. 11 is a flowchart illustrating a method for displaying a desktop card according to another exemplary embodiment of the present application;
FIG. 12 is a diagram illustrating display area partitioning provided by an exemplary embodiment of the present application;
FIG. 13 illustrates a desktop card update layout provided by an exemplary embodiment of the present application;
FIG. 14 illustrates a desktop card update layout provided by another exemplary embodiment of the present application;
FIG. 15 is a flow chart illustrating a manner in which a desktop card is pushed as provided by an exemplary embodiment of the present application;
FIG. 16 illustrates a schematic diagram of a desktop card push provided by an exemplary embodiment of the present application;
FIG. 17 is a block diagram illustrating a display device of a desktop card according to an embodiment of the present application;
fig. 18 is a block diagram illustrating a structure of a terminal according to an exemplary embodiment of the present application.
Detailed Description
To make the objects, technical solutions and advantages of the present application more clear, embodiments of the present application will be described in further detail below with reference to the accompanying drawings.
It should be noted that all the application information collected in the present application is collected under the condition that the user agrees and authorizes, and the collection, use and processing of the relevant application data need to comply with relevant laws and regulations and standards of relevant countries and regions.
Fig. 1 is a flowchart illustrating a display method of a desktop card according to an exemplary embodiment of the present application, where the method includes:
step 101, obtaining a desktop card, where the desktop card is a card-based component for displaying information on a desktop.
A great amount of application information in the terminal needs to be displayed to a user, and a desktop card is a card-type component for displaying information to the user, is usually arranged on a negative screen and is an interface control. After the desktop card is triggered, the desktop card can jump to an application interface corresponding to the card content.
The terminal has two modes of acquiring the desktop card, and firstly, the desktop card is generated based on the application interface, and the desktop card is generated by acquiring the information of the application interface and performing art rendering after processing. The other mode is that content filling is carried out based on a card template, after the terminal obtains application interface information, corresponding content is filled into a proper template according to a set template, and therefore the desktop card is generated.
And 102, carrying out card clustering on the desktop cards based on the card contents of the desktop cards to obtain card clustering results, wherein the card clustering results comprise a plurality of card clusters.
The card content of the desktop card comprises explicit content and implicit content, wherein the explicit content refers to content displayed by the desktop card and can be seen by a user, for example, information displayed by the desktop card, application corresponding to the desktop card and the like; the implicit content refers to content which is stored in a cache inside the terminal and is invisible to a user, for example, a jump link of an application interface corresponding to the desktop card and important data of a process of generating the desktop card and the like.
After the terminal obtains the desktop cards, clustering the desktop cards according to the card contents of the desktop cards, wherein clustering means that a plurality of desktop cards are divided into different clusters according to a certain specific standard (such as distance), so that the similarity of the desktop cards in the same card cluster is as large as possible, and the difference of the desktop cards not in one cluster is as large as possible, namely the desktop cards of the same type are clustered together as much as possible after clustering, and the desktop cards of different types are separated as much as possible.
And 103, integrating the desktop cards in the same card cluster to obtain a desktop integration component, wherein the desktop integration component is an independent desktop card or a desktop card group consisting of a plurality of desktop cards.
Because the variety and the quantity of the application information are very abundant, and the interface size of the terminal for displaying the desktop card is limited, the desktop card needs to be integrated.
The card clustering result can be used as the basis for integrating the desktop cards, and the desktop cards in the same card type cluster in the card clustering result are integrated to obtain a desktop integration component.
The integration of the desktop cards is essentially to integrate similar desktop cards into a new card or a new card group according to the relevance of the card contents of the desktop cards. Optionally, the desktop integration component may integrate the card content with the desktop cards in the same card cluster to generate a new desktop card; the irregularly arranged desktop cards can be integrated into a new desktop card group to serve as a desktop integration component, so that the related application information can be conveniently displayed for the user.
And 104, displaying the desktop integration component.
And displaying the desktop integration component obtained after integrating the desktop card on the desktop or a negative screen to display the application information for the user.
To sum up, in the embodiment of the present application, after the terminal acquires the desktop card, the desktop card is clustered and integrated according to the card content of the desktop card, and finally, the desktop integration component after integration is displayed, and the terminal can adaptively integrate the desktop card according to the content of the desktop card, so that the organization form of the desktop card is not limited to a fixed or irregular display form, which is more favorable for clearly displaying the application information to the user.
Fig. 2 is a flowchart illustrating a desktop card display method according to another exemplary embodiment of the present application, where the method includes:
step 201, obtaining a desktop card.
The step 101 may be referred to in the implementation manner of this step, and this embodiment is not described herein again.
Step 202, card feature vectors of the desktop card are obtained, the card feature vectors are generated based on card content vectors corresponding to card contents of at least one content dimension, and the content dimension includes texts and controls.
When the terminal clusters the desktop cards, clustering needs to be performed according to the similarity of the desktop cards, and the similarity of the desktop cards needs to be measured through the numerical value of the similarity of the desktop cards. The similarity value of the desktop card needs to be calculated according to the feature vector of the desktop card.
The terminal obtains the card feature vector of the desktop card in different modes according to different desktop card generation modes.
In one possible implementation, the desktop card is generated based on the application interface, and the card feature vector of the desktop card is generated in the process of generating the desktop card.
In another possible implementation, the desktop card is obtained by content filling based on the card template, and when the desktop card is generated, the card feature vector of the desktop card is not generated. Therefore, it is necessary to perform vectorization processing on the card content line of at least one content dimension in the desktop card to obtain a card content vector. And then acquiring a card feature vector for generating the desktop card based on the card content vector.
Wherein the card content dimension comprises two dimensions of text and control. The terminal acquires the text content of the desktop card as the card content under the condition that the desktop card only contains the text dimension, and takes the text content vector obtained by encoding the text content as the card feature vector. Under the condition that the desktop card contains both the text dimension and the control dimension, the text content of the text dimension and the control content of the control dimension are obtained and used as card contents, the card contents of the two dimensions are respectively encoded in different encoding modes to obtain card content vectors, and then the text content vectors and the control content vectors are weighted and averaged to obtain card characteristic vectors. The Text content refers to a Text attribute of the desktop card, namely a Text attribute. The control content may contain image information of the control and structured information of the control.
In a possible implementation manner, after the terminal acquires the text content of the desktop card, a word segmentation algorithm is used for segmenting the text content into a plurality of independent words, and the words are segmented texts. And then, performing sliding pooling on the word vectors corresponding to the word segmentation texts to obtain the text vectors corresponding to the text content. On one hand, the terminal can obtain the structured information of the desktop card by obtaining the control tree of the display interface of the desktop card, and on the other hand, the terminal can obtain the control icon of the control content of the desktop card in a screenshot mode after confirming the position of the desktop card in the display interface. And after the control content is obtained, obtaining the control content code of the desktop card through image processing, image coding and structured coding. And carrying out weighted average on the control content codes and the text content codes to obtain card feature vectors of the desktop card.
And step 203, clustering the desktop cards based on the vector distance between the card feature vectors to obtain a card clustering result.
After the terminal acquires the feature vectors of the desktop cards again, the desktop cards need to be clustered according to the similarity among the desktop cards, and the similarity of the desktop cards is determined by the feature of the cardsThe vectors are obtained through calculation, and because the embodiment performs clustering based on the card content, the cosine distance between the card feature vectors corresponding to the two desktop cards can be used as the similarity of the two desktop cards. The similarity between the two desktop cards can be determined by Sim (C) 1 ，C 2 )＝cosine(F(C 1 )，F(C 2 ) C represents a desktop card, and F (C) represents a card feature vector corresponding to the desktop card.
After the terminal obtains the similarity between the desktop cards, the desktop cards need to be clustered according to a similarity threshold value. Desktop cards with the similarity higher than the similarity threshold value can be clustered to obtain a card cluster, and desktop cards with the similarity lower than the similarity threshold value cannot be clustered into the card cluster. The similarity threshold is set, the numerical value of the similarity threshold can be adjusted according to the display interface state of the terminal, and when the complexity of the display interface of the terminal is high, the similarity threshold can be correspondingly reduced, so that more desktop cards can be clustered together, and the effect of saving display interface controls is achieved. When the complexity of the terminal display interface is low, the similarity threshold value is correspondingly increased through the terminal, so that the desktop cards with high card content correlation degree are clustered together, the desktop card display orderliness is strong, and various information can be provided more clearly.
And 204, integrating the pure text desktop cards based on the texts contained in the pure text desktop cards in the same card type cluster to obtain integrated desktop cards, wherein the integrated desktop cards are independent desktop cards, and the pure text desktop cards do not contain controls.
After the terminal clusters the desktop cards, all the desktop cards that only contain text content, that is, plain text desktop cards, may be in a card cluster. In this case, the terminal obtains texts contained in all desktop cards in the card cluster, extracts similar texts from the text contents of all desktop cards in the card cluster, and extracts key words from the similar texts to serve as key texts of the card cluster and also serve as key texts for integrating the desktop cards.
The method includes the steps that a plurality of desktop cards are still contained in each card type cluster, before desktop integrated cards are generated, the terminal needs to refine and classify the desktop cards in each card type cluster, so that the similarity of the desktop cards contained in each classification result after refinement and classification is higher, and then corresponding type names are marked for different classification results and used for displaying in the desktop integrated cards.
As shown in fig. 3, after clustering desktop cards, a terminal integrates a plurality of pure text desktop cards 301 of a notification class obtained by clustering, so as to obtain a second desktop card 302, where "message reminder" is a key text, "message notification", "news notification", and "health notification" are obtained by performing detailed classification on desktop cards in the card class cluster, and the terminal marks different types of names, such as "message notification", "news notification", and "health notification", for classification results of the detailed classification, and displays the names in the desktop integration cards.
Step 205, integrating the controllable desktop cards in the same card cluster into a desktop card group, where the controllable desktop cards include texts and controls.
When the terminal clusters the desktop cards, the card cluster may contain the desktop cards with controls, and at this time, if the same method as that of the plain text desktop cards is adopted, the desktop cards with controls cannot realize the function of directly controlling the application through the desktop cards after being integrated. Therefore, for the desktop card with the control, the terminal only needs to adjust the original desktop card to be a proper size and integrate the desktop card into a new desktop card group. The desktop deck is a desktop integration component, which is essentially a box-type control, whose key text is extracted from the text of the desktop cards in the card-like cluster.
As shown in fig. 4, the controllable desktop cards of the same card cluster, the first desktop card 401 with the key text of "map navigation", and the second desktop card 402 with the key text of "travel scene" are integrated with the third desktop card 403 with the key text of "travel assistant", so as to obtain the desktop card group 404 with the key text of "convenient travel".
Step 206, displaying the desktop integration component.
And displaying the integrated desktop card or the desktop card group in the display interface.
In the embodiment of the application, the terminal clusters the desktop cards based on the similarity of the card contents of the desktop cards, so that the desktop cards are more orderly to be displayed. Different desktop cards in the card clustering result are integrated into desktop integration components in different forms, so that the complexity of the desktop cards is reduced, the integration modes of the desktop cards are more flexible and various, and different types of application information can be displayed more clearly.
Due to different types of application information, the display content of the corresponding desktop card is different. And a part of desktop cards only need to show text information to a user, and the user can jump to a corresponding application interface by triggering the corresponding desktop cards. The other part of the desktop card needs to be provided with a corresponding control for directly controlling the application, for example, a player desktop card with a play key and a pause key, and a user can directly control the application through the control in the desktop card or can jump to a corresponding application interface by triggering the desktop card. Accordingly, the generation method is different for different desktop card types.
The manner in which the desktop card is generated will now be described with an exemplary embodiment.
FIG. 5 is a flow chart illustrating a process for obtaining a desktop card as provided by another example of the present application, the process comprising:
step 501, extracting key interface content of at least one content dimension from the interface content of the application interface.
And the terminal acquires at least one content dimension of the text dimension and the control dimension from the application interface as the content of the key interface. In general, text content exists in an application interface, and the text content is more specific than other forms of content, so that the terminal first acquires text dimension content as key interface content.
Further, when the terminal includes the control, the control dimension content can also be used as key interface content and displayed in the interface card.
In a possible implementation manner, after the terminal obtains the control tree of the application interface, text attributes of the control are extracted from control information corresponding to the control by traversing each node in the control tree, wherein the text attributes include at least one of explicit text attributes and implicit text attributes. The explicit Text attribute refers to a Text attribute of the control, wherein the Text attribute is visible to a user and is displayed on an application interface in a Text form. The implicit text attribute, i.e., the non-visible text attribute, refers to a ContentDescription text attribute of the control. The ContentDescription text attribute is an attribute written in the control by a developer, is used for describing the text of the control function, is invisible in an application interface, and enables the interface to be attractive and part of the text to be hidden in the ContentDescription text attribute by the developer.
The terminal acquires the Text content by traversing each node of the control tree, so that each control of the application interface can acquire the Text attribute and the ContentDescription Text attribute of the control.
In another possible implementation manner, the terminal extracts control coordinates from control information corresponding to an application interface control in a control tree, then intercepts a control image of the application interface control from the application interface based on the control coordinates, and meanwhile, the terminal extracts structured information from the control information of the interface control in the control tree. And taking the structured information and the control image together as control dimension content.
Step 502, text summarization processing is performed on the text to obtain a key text.
For the text dimension content in the interface content, a corresponding algorithm is required to simplify the information. For display in a desktop card.
Optionally, the terminal extracts a key of the text dimensional content from the text dimensional content by using MMR (maximum boundary Relevance)Text. The text summarization process is to calculate a piece of text which can represent the subject of a given text sequence most. Common algorithms are divided into generative and decimated, where a decimated is the subject of representing an entire piece of text using a portion of the text in the original text. Because the terminal application information is updated quickly, the MMR extraction algorithm with high operation efficiency is adopted to carry out text abstract processing. Setting original text dimension content as C, d i For a subset of short sentences in the text dimension content, λ sim (C, d) i ) Representing the similarity of a certain subset of phrases with the original text dimension content C,
Figure BDA0003846106640000051
representing the similarity between a certain subset of short sentences and the processed abstract text,
Figure BDA0003846106640000052
the index of k sentences returned by searching is represented, wherein the two formulas are connected by a minus sign, so that the extracted text abstract can express the meaning of the whole text dimension content and has diversity. Where λ is a parameter used to control the diversity of the digests. The MMR algorithm formula for text summarization processing is as follows:
Figure BDA0003846106640000053
and the terminal takes the text abstract processing result as a key text.
Step 503, under the condition that the application interface comprises the control, extracting the key control from the control.
For the control dimension content in the application interface, the terminal needs to extract the key control in the application interface and the control associated with the key control as the key control.
In a possible implementation manner, the terminal may perform the acquisition of the key control through a key control prediction algorithm.
For example, a play control of a video player page may serve as a key control, and a pause control is a control associated with the play control.
Step 504, the extracted key interface content is used as card content, and a desktop card is generated.
The key interface content comprises at least one of key text and key controls. After the terminal obtains the key Interface content, the key Interface content is used as card content, and a desktop card corresponding to the application Interface can be obtained through rendering of a corresponding art UI (User Interface). This embodiment does not describe the art UI rendering again.
And 505, generating a card feature vector of the desktop card based on the content of the key interface.
The key interface content is the card content of the desktop card, and the implementation manner of this step may refer to step 101, which is not described herein again.
As shown in fig. 6, in a possible implementation, the information of the message notification class, the information of the consulting summary class, the information of the event reminder class, and the information of the figure recommendation class may correspond to a plain text desktop card, so when the desktop card is generated, plain text information is extracted from an application interface, and a text information summarization algorithm is used to obtain a key text for displaying in the corresponding desktop card. The reminding item information, the service push information and the service control information can correspond to the controllable desktop card, so that when the desktop card is generated, the application information with the control relation is extracted from the application interface, the calculation is carried out through the key association control prediction algorithm, the key text and the key control are obtained, and the key text and the key control are displayed on the desktop card after being processed.
It should be noted that the type of the application information may be the same as the type of the application information divided in fig. 6, that is: a message notification class, a consultation abstract class, a reminder item class, a service push class and a service control class. Optionally, more application information types may be divided according to different dimensions in the iterative update of the application information. The method for dividing the application information type and the divided application information type are not limited in the embodiment of the application.
According to the method and the device, the key text and the key control of the application interface are obtained in different processing modes respectively according to the text dimension and the control dimension of the application interface and are used as the key content of the application interface, and the desktop card is generated in an art UI rendering mode according to the key content, so that the generation of the desktop card is not limited to the fixed mould filling, and the problem of single desktop card form is solved.
For the control dimension content in the application interface, the terminal needs to determine the first control from the control tree and then determine the second control associated with the first control, and the first control and the second control are used as key controls together. The process of determining key controls in the case of an application interface containing control dimensions is described below in an exemplary embodiment.
Fig. 7 is a flowchart illustrating a process of generating a card feature vector according to another exemplary embodiment of the present application, where the process includes:
step 701, a control tree of the application interface is obtained, wherein the control tree is used for representing the topological structure of each control in the application interface.
The terminal can directly obtain the control tree corresponding to the application interface through an Accessibility (barrier-free service) interface defined by the system.
The control tree is formed by interface controls of the application interface. The control tree is a tree structure and comprises all controls in the application interface, each control has control information corresponding to the control, and each control corresponds to one node in the control tree. The terminal can simultaneously obtain the control coordinates of the corresponding control of each node in the control tree when obtaining the control tree, and each control has one control coordinate corresponding to the control coordinate.
After the terminal acquires the control tree, a first key control and a second key control associated with the first key control need to be determined from the control tree, and the first key control and the second key control are extracted for displaying in the desktop card. The specific implementation steps are as follows.
Step 702, acquiring control information of each control from a control tree, where the control information includes at least one of text information, image information, and structural information, and the structural information is used to represent a hierarchical structure of the control in the control tree.
The Text information in the control information refers to Text characters in the application interface, and the Text attributes of the control comprise visible Text attributes (namely Text attributes) and non-visible Text attributes (namely ContentDescription attributes).
The image information refers to images in the application interface, and comprises icons, pure images and text images displayed in the form of images.
The structured information is used for representing the hierarchical structure of the controls in the application interface, each control is a container, and developers can set the controls according to a certain hierarchical relationship when developing the application interface. The terminal can determine other controls which have a certain association with a certain control, such as a child control or a parent control of the control, by acquiring the structured modal information of the certain control.
And 703, coding the control information to obtain the control original coding vector of the control.
After the terminal acquires different types of control information, because the different types of information have different characteristics, the terminal needs to encode each type of control information respectively in different encoding modes.
Wherein, the text information is encoded by a text encoder; the image information is coded by adopting an image coder; and the structured information is encoded by adopting a structured encoder to obtain respective modal codes corresponding to different modal information.
In a possible implementation mode, binary coding is adopted for the structured information, and the position codes of the control in the control tree in the first-order traversal sequence are coded and the position codes of the control in the control tree in the second-order traversal sequence are spliced to obtain a structured coding vector; for text information, firstly, performing a word segmentation algorithm on the text information of the control, then, inquiring a word vector of each word by taking the word as a unit, and performing weighted average on the word vectors to obtain a text coding vector of the control; for an image information terminal, a pre-trained CNN (Neural Network) is used for extracting depth features of an image, and then the depth features are converted into one-dimensional vectors. I.e. the image coding vector.
And after the terminal acquires the coding vectors corresponding to the control information of each type, fusing different coding vectors to obtain the original coding vector of the control. Optionally, the terminal fuses coding vectors corresponding to different types of control information through the feature fusion network to obtain an original coding vector of the control.
And 704, performing self-attention coding on the original control coding vector to obtain a control self-attention coding vector.
After the terminal acquires the original coding vector of the control, extracting the self-attention coding vector of the control by using a self-attention mechanism of a Transformer model.
Step 705, determining a first key control based on the control self-attention coding vector of the control corresponding to the key text and the control self-attention coding vectors of other controls.
After the terminal determines the key text, the association degree between the two controls can be calculated according to the self-attention coding vector of the corresponding control of the key text and the self-attention coding vector of other controls in the application interface. And determining the control with the relevance degree higher than a set threshold value to be the first key control.
As shown in fig. 8, the two controls connected by a solid line are associated with a degree higher than the set threshold. The key text in the application interface 801 corresponds to the text control 802, and the video control 803, the first icon control 804, the second icon control 805, and the third icon control 806 all have a correlation degree relationship through calculation, wherein the correlation degree between the video control 803 and the text control 802 corresponding to the key text is higher than a set threshold value, so that the video control 803 is determined to be the first key control.
In one possible embodiment, the terminal may use MLP (Multi-Layer Perceptron) to calculate the association between two widgets, and the association between two widgets is calculated based on the self-attention-encoding vectors of the two widgets. And inputting the self-attention coding vectors of the two controls into an MLP model to obtain the association degree of the two self-attention coding vectors, namely the association degree of the two controls. And after the association degrees among the application interface controls are obtained, sorting the association degrees from high to low, and determining the control of which the association degree with the control corresponding to the key text is higher than a set threshold value as a first key control.
Step 706, determining a second key control associated with the first key control based on the control self-attention encoding vector of the first key control and the control self-attention encoding vectors of the other controls.
After the terminal determines the first key, the terminal calculates the self-attention coding vectors of the MLP model and other controls based on the self-attention coding vector of the first key control to obtain the association degree between the first key control and other controls, and determines the control with the association degree higher than a set threshold value with the first key control as a second key control.
When the same application interface has a plurality of key texts, each control corresponding to the key text has a control having an association degree with the key text, and a control having an association degree higher than a threshold value in the controls having an association degree corresponding to each key text can be respectively determined as a first key control, and at this time, one application interface can generate a plurality of first key controls. Similarly, the control with the association degree higher than the set threshold is respectively determined from the controls with the association degree of each first key control as a second key control, and the second key control, the key text and the first key control can be displayed together in the desktop card.
Each group of associated controls and the corresponding key texts can be displayed as the card content of one controllable desktop card. As shown in fig. 9, in an application interface 901, three key text "personalized", "display and brightness", and "notification and status bar" corresponding controls are a first control 902, a second control 903, and a third control 904, respectively, three first key controls 905 corresponding to the three controls, and three second key controls 906 associated with the first key controls, respectively, which can be obtained by calculating the association degree of the current interface.
Fig. 10 shows a schematic structural diagram of a terminal for obtaining an association degree of an application interface control, where the terminal encodes text information, image information, and structured information in an application interface 1001 through a structural encoder 1002, a text encoder 1003, and an image encoder 1004, respectively, to obtain three corresponding encoding vectors. And splicing and fusing the three coding vectors to obtain the original coding vector of the control. And performing self-attention coding on the control original coding vector through a transform model 1005 to obtain a control self-attention coding vector. And finally, calculating the relevance of the control self-attention coding vector and the self-attention coding vectors of other controls through an MLP model 1006 to obtain the relevance among the controls.
And 707, vectorizing the key text to obtain a text feature vector.
The text vectorization process expresses the key text into real number vectors which can be recognized by a computer, and the key text can be expressed into text feature vectors of the levels of words, sentences and the like according to different sizes of fine granularity.
Optionally, the text vectorization may be implemented by a statistical-based method, or may be implemented by a neural network-based method.
Step 708, generating a card feature vector of the desktop card based on the text feature vector and the control self-attention coding vector of the key control.
And carrying out weighted average on the control self-attention coding vector and the text feature vector of the key text to obtain a card feature vector of the desktop card.
According to the embodiment of the application, after the terminal acquires the information of the application interface, the self-attention coding vector of the control is generated through the Transformer model, the first key control and the second key control are determined through the MLP model to calculate the association degree, the card characteristic vector of the desktop card can be generated, the controllable desktop control can be generated according to the association relation of the controls in the application interface, the completeness of the information displayed by the desktop control is benefited, meanwhile, the user can realize control over multiple functions of the application through the desktop control, and the practicability is higher.
In the above embodiment, after the terminal clusters the desktop cards, the desktop cards are directly integrated based on the card clustering result.
In another possible implementation manner, after obtaining the desktop card clustering result, the terminal needs to determine a relative loading factor based on the current card integration ratio and the display ratio, and integrate the desktop cards in the same card cluster to obtain a desktop card integration component when the relative loading factor is greater than a threshold value. The card display ratio is used for representing the current integration degree of the desktop card, and the card display ratio is used for representing the screen ratio of the currently displayed desktop card.
n cur For indicating the number of desktop cards (individual desktop cards or groups of desktop cards) displayed independently in a display page, n total Used for representing the total number of the desktop cards (including the desktop cards in the desktop card group) displayed on the display interface, S cur The area of the screen occupied by the currently displayed desktop card, S is the total resolution area of the screen, and the relative loading factor
Figure BDA0003846106640000081
Figure BDA0003846106640000082
Where λ is an adjustable parameter for controlling the tendency to integrate into a stand-alone desktop card or set of desktop cards.
The relative loading factor can be used to characterize the saturation level of the display interface, and the larger the relative loading factor, the more crowded and complex the desktop card representing the display interface. When the relative loading factor is too large, the terminal can automatically optimize the direction in which the relative loading factor can be greatly reduced by adopting the method of the embodiment of the application, for example, a plurality of desktop cards are integrated into a new independent desktop card.
The desktop display cards are arranged according to the browsing sequence of the user, and according to the modern reading habit, the desktop cards can be generally sorted from top to bottom and from left to right according to the frequency of the user using the desktop cards. However, the arrangement of the desktop cards according to the usage frequency of the desktop cards only results in that the display positions of the desktop cards with low frequency but important are not easily noticed by the user, and the problem that the display positions of the desktop cards are not proper is also caused. For example, the terminal acquires that the user will carry out an important meeting tomorrow, but the number of times that the terminal receives the information is small, and the desktop card usage frequency corresponding to the reminding information of the important meeting is very low, so that the desktop cards are arranged only according to the desktop card usage frequency, which may cause that the important meeting reminding cards with low frequency and high priority are arranged behind the news cards with high frequency and low priority, and cannot play a role in effective information recommendation.
Fig. 11 is a flowchart illustrating a method for displaying a desktop card according to another exemplary embodiment of the present application, where the method includes:
in step 1101, a desktop card is obtained.
The step 101 may be referred to in the implementation manner of this step, and this embodiment is not described herein again.
Step 1102, performing card clustering on the desktop cards based on the card contents of the desktop cards to obtain card clustering results, wherein the card clustering results comprise a plurality of card clusters.
The step 102 may be referred to in the implementation manner of this step, and this embodiment is not described herein again.
Step 1103, integrating the desktop cards in the same card type cluster to obtain a desktop integration component, where the desktop integration component is an independent desktop card or a desktop card group composed of multiple desktop cards.
The step 103 may be referred to in the implementation manner of this step, and this embodiment is not described herein again.
And 1104, determining a target display area to which the desktop integration component belongs based on the card content of the desktop card in the desktop integration component, wherein different display areas correspond to different display priorities, and the display priorities are related to the card content.
The terminal adopts the mode of data bucket, divides desktop card display area into the display area of a plurality of display priority from top to bottom, and wherein the higher the display priority in the display area of top is. As shown in fig. 12, the display interface 1201 is divided into different priority levels, which are a first priority level area 1202, a second priority level area 1203, a third priority level area 1204, and so on from top to bottom.
The display area division potential in the figure is virtualized at the system level and is not visible in the display interface.
The display areas with different display priorities correspondingly display different desktop integration components, the terminal divides the desktop cards belonging to the same priority into the display areas with the same display priority based on the priority of the card contents of the desktop cards in the desktop integration components, the display priority of the display areas corresponds to the priority of the card contents in the desktop integration components, and for example, the target display area of the desktop card component with the highest card content priority is a first priority area.
Step 1105, determining a target display position of the desktop integration component in the target display area based on the historical use frequency of the desktop card in the desktop integration component, wherein different display positions correspond to different display priorities, and the display priorities and the historical use frequency are in a positive correlation.
In the display area corresponding to each display priority, the terminal sorts the desktop integration components again according to the use frequency of the user, and determines the target display position of the desktop integration components in the target display area.
The priority of the display position is in positive correlation with the use frequency of the user, the higher the use frequency of the user is, the higher the display priority corresponding to the desktop card is, and the display interface has a more striking display position.
The display area range corresponding to each display priority is not fixed, telescopic adjustment can be performed according to actual use conditions, and the display areas with different display priorities can be overlapped, so that the maximum use efficiency of the display interface is ensured. However, the display area ranges with different display priorities have set upper limits, and when the desktop cards in the display area corresponding to a certain display priority are filled up and cannot be further integrated, the desktop cards with relatively low frequency are stored in the system cache area.
In step 1106, the desktop integration component is displayed at the target display position of the target display area.
After the terminal determines the target display position of the desktop card, the corresponding desktop control is displayed at the position, and the display position can be changed according to the update of the terminal to the display interface.
In the embodiment of the application, the display interface is divided according to the card content priority of the desktop card by the terminal, the display positions in the corresponding region of each display priority are arranged according to the user use frequency, the problem that the low-frequency high-priority card is divided and is positioned at the back is avoided, the desktop card can be adaptively adjusted in a targeted mode, the desktop card recommendation information is clearer, and the highest display interface utilization rate is ensured.
In addition, the terminal can adaptively adjust the display layout and the display content of the desktop card based on the screen display state, wherein the screen display state comprises the screen display size and/or the available display area of the screen.
In a possible implementation manner, when the display interface is occupied by other controls, such as a video playing window, or the display interface is occupied by a split-screen application, the available display area of the desktop card is limited, and at this time, the terminal performs adaptive adjustment on the display layout of the desktop card again according to the current available display area of the screen. For example, in fig. 13, the original layout manner of the terminal is as shown in a first display interface 1301, after a video playing window 1303 is added to the display interface, the terminal performs adaptive adjustment of the display layout again on the desktop cards in the application suggestions, obtains a desktop card group 1304 with a key text of "application suggestion" after adjustment, and obtains a second display interface 1302 after adjustment.
In another possible implementation, when the terminal has a folding screen, the size of the display screen may be changed according to the operation of the user, and at this time, the desktop card also needs to be adaptively arranged.
In the process of generating the desktop card, the card content of the desktop card is stored in a buffer area of the system. When the desktop cards are laid out, the desktop cards to be displayed are selected according to the actual screen size, and the display forms and the display positions of the desktop cards are planned.
When the screen display area is adjusted, the desktop card on the original display interface and the desktop card in the cache area are adjusted in a self-adaptive mode together, and the desktop card is integrated again. When the display area of the desktop card is changed from big to small, the terminal further integrates the desktop card and stores the desktop card which is not displayed in the display area into a cache area; and when the display area of the desktop card is changed from small to large, the terminal executes the reverse process of integration or redisplays the desktop card in the cache area. If the display area still cannot be filled, the terminal can execute the process of generating the desktop card again, modify the relevance threshold value and fill more display contents for each desktop card.
As shown in fig. 14, a display interface 1401 is a display area of a desktop card under a small screen, a second display interface 1402 is a display area of the desktop card after the foldable screen is opened, when the display area of the desktop card is changed from small to large, the terminal releases and displays the desktop card corresponding to the keywords stored in the cache area, such as "hot application", "hot good things", "poetry every day", and the like, and at the same time, the terminal executes the desktop card generation flow again to generate the desktop card again, adds music information to the card content, and fills the display content of the music playing card. Accordingly, when the size of the display screen becomes smaller, the second display interface 1402 is converted into the display interface 1401, and the desktop card for which display is cancelled is stored in the buffer area.
FIG. 15 illustrates a desktop card push approach provided by an exemplary embodiment of the present application, which includes:
at step 1501, the target desktop card is screened from the desktop cards.
On one hand, the terminal screens out target desktop cards from the desktop cards based on the control relation between the current equipment and the cross-terminal equipment. And under the condition that the cross-terminal equipment is controlled by the current equipment, the target desktop card does not contain a desktop card for controlling the current equipment.
The desktop card is pushed to the controlled equipment by the master control equipment, and the terminal screens out the uncontrollable desktop card from the desktop card as the target desktop card due to the fact that the controlled equipment is low in grade.
For example, the current device is a smart phone, the cross-terminal device is a smart air conditioner, the device level of the smart phone is higher than that of the smart air conditioner, and the smart air conditioner is a controlled device.
Under the condition that the desktop card is sent to the master control equipment by the controlled equipment, because the master control equipment is higher in grade, the controlled equipment can send all the desktop cards to the master control equipment, and the master control equipment can control the controlled equipment through the controllable desktop card.
For example, the smart refrigerator pushes desktop cards to the smart phone, and since the smart phone is higher in device level, the smart refrigerator takes all the desktop cards as target desktop cards, and a user can control the smart refrigerator by operating the smart phone.
Under the condition that the device grades of the two devices are the same, when one device pushes the desktop card to the other device, all the desktop cards are also used as target desktop cards, and a user can control the other device by controlling any device. For example, in fig. 16, the display interface 1601 is a display interface of a smart phone, the second display interface 1602 is a display interface of a smart watch, and the two devices have the same level, so that a music playing card 1603 capable of controlling the smart phone can be displayed on the smart watch, and meanwhile, the smart phone also integrates the sports card 1604 and the weather card 1605 to obtain a desktop card group 1607, which is displayed in the second display interface 1602 of the smart watch as a target desktop card.
On the other hand, the terminal determines the type of the target card supported and displayed by the cross-terminal equipment, and then selects the target desktop card from the desktop cards based on the type of the target card.
Because the processing performance of different devices is different, the type of the card supported and displayed by the cross-end device is different from that of the current device, and before the card is pushed, the terminal needs to filter the desktop card of the current device according to the type of the target card supported by the cross-end device and then push the desktop card to the cross-end device.
For example, the current device is a smart phone, the cross-terminal device is a smart refrigerator, the smart refrigerator does not support a music playing function, and correspondingly does not support a music playing desktop card, so that before the desktop card is pushed, the terminal filters out the music playing desktop card of the smart phone, and then takes other desktop cards as a target desktop card.
Step 1502, pushing the target desktop card to the cross-end device so that the cross-end device can display the target desktop card, wherein the cross-end device supports display of card contents in the target desktop card and has control authority of the target desktop card.
After determining the target desktop card, the terminal pushes the target desktop card to the cross-end device for display, and the cross-end device should be capable of supporting the card content in the target desktop card, such as a music playing function, a video playing function, a message notification function, and the like.
After the target desktop card is displayed by the cross-end device, the cross-end device may control the target desktop card, for example, a user may manually remove, stretch, shift, and the like the target desktop card in the cross-end device display page.
In the embodiment of the application, the terminal can screen the desktop card according to the actual situation of crossing the end equipment, and the propelling movement shows to crossing the end equipment, has realized crossing the equipment propelling movement of desktop card, and the range of application is wider, and can realize crossing the control of end equipment to current equipment, and the practicality is strong.
After the desktop card is generated, the content of the card can be dynamically updated along with the change of time and application information. Correspondingly, the terminal can adjust the integration mode and the layout mode of the card according to the use condition of the display interface. The dynamic update of the card content may change according to the change of the application information, for example, the desktop card content of the music list may be dynamically updated along with the update of the music list in the music playing application. The content of the desktop card of the home monitoring video can be changed according to the actual shooting video. In addition, the display state of the desktop card can be automatically re-rendered according to operations of dragging and zooming the card by a user and the like.
The following are embodiments of the apparatus of the present application that may be used to perform embodiments of the method of the present application. For details which are not disclosed in the embodiments of the apparatus of the present application, reference is made to the embodiments of the method of the present application.
Fig. 17 is a block diagram illustrating a display device of a desktop card according to an embodiment of the present application. The apparatus may include:
an obtaining module 1701 for obtaining a desktop card, wherein the desktop card is a card-based component for displaying information on a desktop;
a clustering module 1702, configured to perform card clustering on the desktop card based on the card content of the desktop card to obtain a card clustering result, where the card clustering result includes a plurality of card clusters;
an integration module 1703, configured to integrate the desktop cards in the same card cluster to obtain a desktop integration component, where the desktop integration component is an independent desktop card or a desktop card group formed by multiple desktop cards;
a display module 1704 for displaying the desktop integration component.
Optionally, the clustering module 1702 includes:
the system comprises a vector acquisition unit, a display unit and a control unit, wherein the vector acquisition unit is used for acquiring a card feature vector of the desktop card, the card feature vector is generated based on a card content vector corresponding to the card content of at least one content dimension, and the content dimension comprises a text and a control;
and the card clustering unit is used for clustering the desktop cards based on the vector distance between the card feature vectors to obtain the card clustering result.
Optionally, the vector obtaining unit is configured to:
under the condition that the desktop card is generated based on an application interface, obtaining the card feature vector of the desktop card in the card generation process;
under the condition that the desktop card is obtained by filling content based on a card template, vectorizing the card content of at least one content dimension in the desktop card to obtain a card content vector; generating the card feature vector based on the card content vector.
Optionally, in a case that the desktop card is generated based on the application interface, the apparatus further includes:
the content extraction module is used for extracting key interface content of at least one content dimension from the interface content of the application interface;
the card generating module is used for taking the extracted key interface content as the card content and generating the desktop card;
and the vector generation module is used for generating the card feature vector of the desktop card based on the key interface content.
Optionally, the content extracting module is configured to:
performing text abstract processing on the text to obtain a key text;
and under the condition that the application interface contains the control, extracting the key control from the control.
Optionally, the content extracting module is configured to:
acquiring a control tree of the application interface, wherein the control tree is used for representing the topological structure of each control in the application interface;
determining a first key control and a second key control associated with the first key control from the control tree;
and extracting the first key control and the second key control.
Optionally, the content extracting module is configured to:
acquiring control information of each control from the control tree, wherein the control information comprises at least one of text information, image information and structural information, and the structural information is used for representing the hierarchical structure of the control in the control;
coding the control information to obtain a control original coding vector of the control;
carrying out self-attention coding on the original coding vector of the control to obtain a self-attention coding vector of the control;
determining the first key control based on the control self-attention coding vector of the control corresponding to the key text and the control self-attention coding vectors of other controls;
determining the second key control associated with the first key control based on the control self-attention encoding vector of the first key control and the control self-attention encoding vectors of other controls.
Optionally, the vector generating module is configured to:
vectorizing the key text to obtain a text feature vector;
generating the card feature vector of the desktop card based on the text feature vector and the control self-attention coding vector of the key control.
Optionally, the integrating module 1703 is configured to:
integrating the plain text desktop cards based on texts contained in the plain text desktop cards in the same card cluster to obtain integrated desktop cards, wherein the integrated desktop cards are independent desktop cards and the plain text desktop cards do not contain controls;
and integrating the controllable desktop cards in the same card cluster into the desktop card group, wherein the controllable desktop cards comprise texts and controls.
Optionally, the apparatus further comprises:
the relative loading factor determining module is used for determining a relative loading factor based on the current card integration proportion and the card display proportion, wherein the current card integration proportion is used for representing the current integration degree of the desktop card, and the card display proportion is used for representing the screen proportion of the currently displayed desktop card;
the integrating module 1703 is configured to:
and integrating the desktop cards in the same card cluster to obtain the desktop integration component under the condition that the relative loading factor is larger than a threshold value.
Optionally, the display module 1704 is configured to:
determining a target display area to which the desktop integration component belongs based on the card content of the desktop card in the desktop integration component, wherein different display areas correspond to different display priorities, and the display priorities are related to the card content;
determining a target display position of the desktop integration component in the target display area based on the historical use frequency of the desktop card in the desktop integration component, wherein different display positions correspond to different display priorities, and the display priorities and the historical use frequency are in a positive correlation;
displaying the desktop integration component at the target display location of the target display area.
Optionally, the apparatus further comprises:
and the self-adaptive module is used for self-adaptively adjusting the display layout and the display content of the desktop card based on the screen display state, wherein the screen display state comprises the screen display size and/or the available display area of the screen.
Optionally, the apparatus further comprises:
the card screening module is used for screening target desktop cards from the desktop cards;
the card pushing module is used for pushing the target desktop card to a cross-end device so that the cross-end device can display the target desktop card, the cross-end device supports display of card contents in the target desktop card, and the cross-end device has control authority of the target desktop card.
Optionally, the card screening module is configured to:
screening out the target desktop card from the desktop cards based on a control relation between current equipment and the cross-end equipment, wherein the target desktop card does not contain the desktop card used for controlling the current equipment under the condition that the cross-end equipment is controlled by the current equipment;
determining a target card type which is supported and displayed by the cross-terminal equipment; and screening the target desktop cards from the desktop cards based on the target card type.
Fig. 18 is a block diagram illustrating a structure of a terminal according to an exemplary embodiment of the present application. The terminal 1800 may be implemented as the terminal in the various embodiments described above. The terminal 1800 may include one or more of the following components: a processor 1810, and memory 1820.
Processor 1810 may include one or more processing cores. The processor 1810, using various interfaces and lines to connect various parts throughout the terminal 1800, performs various functions of the terminal 1800 and processes data by executing or executing instructions, programs, code sets or instruction sets stored in the memory 1820 and invoking data stored in the memory 1820. Alternatively, the processor 1810 may be implemented in hardware using at least one of Digital Signal Processing (DSP), field-Programmable Gate Array (FPGA), and Programmable Logic Array (PLA). The processor 1810 may integrate one or a combination of a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), a Neural-Network Processing Unit (NPU), a modem, and the like. Wherein, the CPU mainly processes an operating system, a user interface, an application program and the like; the GPU is used for rendering and drawing contents required to be displayed by the touch display screen; the NPU is used to implement an Artificial Intelligence (AI) function; the modem is used to handle wireless communications. It is to be understood that the modem may not be integrated into the processor 1810, and may be implemented by a single chip.
The Memory 1820 may include a Random Access Memory (RAM) or a Read-Only Memory (ROM). Optionally, the memory 1820 includes a non-transitory computer-readable medium. The memory 1820 may be used to store instructions, programs, code, sets of codes, or sets of instructions. The memory 1820 may include a stored program area and a stored data area, wherein the stored program area may store instructions for implementing an operating system, instructions for at least one function (e.g., a touch function, a sound playing function, an image playing function, etc.), instructions for implementing various method embodiments described below, and the like; the storage data area may store data (such as audio data, a phonebook) created according to the use of the terminal 1800, and the like.
In addition, those skilled in the art will appreciate that the structure of the terminal 1800 as illustrated in the above-described figures is not meant to be limiting, and that the terminal may include more or fewer components than illustrated, or some of the components may be combined, or a different arrangement of components. For example, the terminal 1800 further includes a display screen, a camera, a microphone, a speaker, a radio frequency circuit, an input unit, a sensor (such as an acceleration sensor, an angular velocity sensor, a light sensor, etc.), an audio circuit, a WiFi module, a power supply, a bluetooth module, etc., which are not described herein again.
The embodiment of the present application further provides a computer-readable storage medium, where at least one program code is stored, and the program code is loaded and executed by a processor to implement the method for displaying a desktop card according to the above embodiments.
Embodiments of the present application provide a computer program product comprising computer instructions stored in a computer readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions to cause the computer device to perform the display method of the desktop card provided in the various alternative implementations of the above aspect.
It should be understood that reference to "a plurality" herein means two or more. "and/or" describes the association relationship of the associated object, indicating that there may be three relationships, for example, a and/or B, which may indicate: a exists alone, A and B exist simultaneously, and B exists alone. The character "/" generally indicates that the former and latter associated objects are in an "or" relationship. In addition, the step numbers described herein only show an exemplary possible execution sequence among the steps, and in some other embodiments, the steps may also be executed out of the numbering sequence, for example, two steps with different numbers are executed simultaneously, or two steps with different numbers are executed in a reverse order to the illustrated sequence, which is not limited in this application.
The above description is only exemplary of the present application and should not be taken as limiting, as any modification, equivalent replacement, or improvement made within the spirit and principle of the present application should be included in the protection scope of the present application.

Claims (31)

1. A method for displaying a desktop card, the method comprising:
the method comprises the steps of obtaining a desktop card, wherein the desktop card is a card-type component used for information display on a desktop;
performing card clustering on the desktop cards based on the card content of the desktop cards to obtain card clustering results, wherein the card clustering results comprise a plurality of card clusters;
integrating the desktop cards in the same card cluster to obtain a desktop integration component, wherein the desktop integration component is an independent desktop card or a desktop card group consisting of a plurality of desktop cards;
and displaying the desktop integration component.
2. The method of claim 1, wherein the card clustering of the desktop cards based on the card content of the desktop cards comprises:
obtaining a card feature vector of the desktop card, wherein the card feature vector is generated based on a card content vector corresponding to the card content of at least one content dimension, and the content dimension comprises a text and a control;
and clustering the desktop cards based on the vector distance between the card feature vectors to obtain the card clustering result.
3. The method of claim 2, wherein the obtaining card feature vectors of the desktop card comprises:
under the condition that the desktop card is generated based on an application interface, obtaining the card feature vector of the desktop card in the card generation process;
under the condition that the desktop card is obtained by filling content based on a card template, vectorizing the card content of at least one content dimension in the desktop card to obtain a card content vector; generating the card feature vector based on the card content vector.
4. The method of claim 3, wherein, in the case that the desktop card is generated based on the application interface, the method further comprises:
extracting key interface content of at least one content dimension from the interface content of the application interface;
taking the extracted key interface content as the card content, and generating the desktop card;
generating the card feature vector of the desktop card based on the key interface content.
5. The method of claim 4, wherein extracting the key interface content of at least one content dimension from the interface content of the application interface comprises:
performing text abstract processing on the text to obtain a key text;
and under the condition that the application interface contains the control, extracting the key control from the control.
6. The method of claim 5, wherein extracting key controls from the controls comprises:
acquiring a control tree of the application interface, wherein the control tree is used for representing the topological structure of each control in the application interface;
determining a first key control and a second key control associated with the first key control from the control tree;
and extracting the first key control and the second key control.
7. The method of claim 6, wherein determining a first key control from the control tree and a second key control associated with the first key control comprises:
acquiring control information of each control from the control tree, wherein the control information comprises at least one of text information, image information and structural information, and the structural information is used for representing the hierarchical structure of the control in the control;
coding the control information to obtain a control original coding vector of the control;
carrying out self-attention coding on the control original coding vector to obtain a control self-attention coding vector;
determining the first key control based on the control self-attention coding vector of the control corresponding to the key text and the control self-attention coding vectors of other controls;
determining the second key control associated with the first key control based on the control self-attention encoding vector of the first key control and the control self-attention encoding vectors of other controls.
8. The method of claim 7, wherein generating the card feature vector for the desktop card based on the key interface content comprises:
vectorizing the key text to obtain a text feature vector;
generating the card feature vector of the desktop card based on the text feature vector and the control self-attention coding vector of the key control.
9. The method of claim 1, wherein the integrating the desktop cards in the same card-like cluster to obtain a desktop integration component comprises:
integrating the plain text desktop cards based on texts contained in the plain text desktop cards in the same card cluster to obtain integrated desktop cards, wherein the integrated desktop cards are independent desktop cards and the plain text desktop cards do not contain controls;
and integrating the controllable desktop cards in the same card cluster into the desktop card group, wherein the controllable desktop cards comprise texts and controls.
10. The method of claim 1, further comprising:
determining a relative loading factor based on a current card integration proportion and a card display proportion, wherein the current card integration proportion is used for representing the current integration degree of the desktop card, and the card display proportion is used for representing the screen proportion of the currently displayed desktop card;
the desktop cards in the same card cluster are integrated to obtain a desktop integration component, which includes:
and integrating the desktop cards in the same card cluster to obtain the desktop integration component under the condition that the relative loading factor is larger than a threshold value.
11. The method of claim 1, wherein the displaying the desktop integration component comprises:
determining a target display area to which the desktop integration component belongs based on the card content of the desktop card in the desktop integration component, wherein different display areas correspond to different display priorities, and the display priorities are related to the card content;
determining a target display position of the desktop integration component in the target display area based on the historical use frequency of the desktop card in the desktop integration component, wherein different display positions correspond to different display priorities, and the display priorities and the historical use frequency are in a positive correlation;
displaying the desktop integration component at the target display location of the target display area.
12. The method of claim 1, further comprising:
and performing self-adaptive adjustment on the display layout and the display content of the desktop card based on a screen display state, wherein the screen display state comprises a screen display size and/or a screen available display area.
13. The method of claim 1, further comprising:
screening out target desktop cards from the desktop cards;
pushing the target desktop card to a cross-end device so that the cross-end device can display the target desktop card, wherein the cross-end device supports display of card contents in the target desktop card, and the cross-end device has control authority of the target desktop card.
14. The method of claim 13, wherein the filtering out the target desktop card from the desktop cards comprises:
screening the target desktop card from the desktop cards based on a control relation between current equipment and the cross-end equipment, wherein the target desktop card does not contain the desktop card for controlling the current equipment under the condition that the cross-end equipment is controlled by the current equipment;
determining the target card type supported and displayed by the cross-terminal equipment; and screening the target desktop cards from the desktop cards based on the target card type.
15. A desktop card display apparatus, the apparatus comprising:
the desktop card display system comprises an acquisition module, a display module and a display module, wherein the acquisition module is used for acquiring a desktop card, and the desktop card is a card-based component used for displaying information on a desktop;
the desktop card clustering module is used for clustering cards of the desktop card based on the card content of the desktop card to obtain a card clustering result, and the card clustering result comprises a plurality of card clusters;
the integration module is used for integrating the desktop cards in the same card cluster to obtain a desktop integration component, and the desktop integration component is an independent desktop card or a desktop card group consisting of a plurality of desktop cards;
and the display module is used for displaying the desktop integration component.
16. The apparatus of claim 15, wherein the clustering module comprises:
the system comprises a vector acquisition unit, a display unit and a control unit, wherein the vector acquisition unit is used for acquiring a card feature vector of the desktop card, the card feature vector is generated based on a card content vector corresponding to the card content of at least one content dimension, and the content dimension comprises a text and a control;
and the card clustering unit is used for clustering the desktop cards based on the vector distance between the card feature vectors to obtain the card clustering result.
17. The apparatus of claim 16, wherein the vector obtaining unit is configured to:
under the condition that the desktop card is generated based on an application interface, obtaining the card feature vector of the desktop card in the card generation process;
under the condition that the desktop card is obtained by filling content based on a card template, vectorizing the card content of at least one content dimension in the desktop card to obtain a card content vector; generating the card feature vector based on the card content vector.
18. The apparatus of claim 17, wherein if the desktop card is generated based on the application interface, the apparatus further comprises:
the content extraction module is used for extracting key interface content of at least one content dimension from the interface content of the application interface;
the card generating module is used for taking the extracted key interface content as the card content and generating the desktop card;
and the vector generation module is used for generating the card feature vector of the desktop card based on the key interface content.
19. The apparatus of claim 18, wherein the content extraction module is configured to:
performing text abstract processing on the text to obtain a key text;
and under the condition that the application interface contains the control, extracting the key control from the control.
20. The apparatus of claim 19, wherein the content extraction module is configured to:
acquiring a control tree of the application interface, wherein the control tree is used for representing the topological structure of each control in the application interface;
determining a first key control and a second key control associated with the first key control from the control tree;
and extracting the first key control and the second key control.
21. The apparatus of claim 20, wherein the content extraction module is configured to:
acquiring control information of each control from the control tree, wherein the control information comprises at least one of text information, image information and structural information, and the structural information is used for representing the hierarchical structure of the control in the control;
coding the control information to obtain a control original coding vector of the control;
carrying out self-attention coding on the original coding vector of the control to obtain a self-attention coding vector of the control;
determining the first key control based on the control self-attention coding vector of the control corresponding to the key text and the control self-attention coding vectors of other controls;
determining the second key control associated with the first key control based on the control self-attention encoding vector of the first key control and the control self-attention encoding vectors of other controls.
22. The apparatus of claim 21, wherein the vector generation module is configured to:
vectorizing the key text to obtain a text feature vector;
generating the card feature vector of the desktop card based on the text feature vector and the control self-attention coding vector of the key control.
23. The apparatus of claim 15, wherein the integration module is configured to:
integrating the plain text desktop cards based on texts contained in the plain text desktop cards in the same card cluster to obtain integrated desktop cards, wherein the integrated desktop cards are independent desktop cards and the plain text desktop cards do not contain controls;
and integrating the controllable desktop cards in the same card cluster into the desktop card group, wherein the controllable desktop cards comprise texts and controls.
24. The apparatus of claim 15, further comprising:
the relative loading factor determining module is used for determining a relative loading factor based on a current card integration proportion and a card display proportion, wherein the current card integration proportion is used for representing the current integration degree of the desktop card, and the card display proportion is used for representing the screen proportion of the currently displayed desktop card;
the integration module is configured to:
and integrating the desktop cards in the same card cluster to obtain the desktop integration component under the condition that the relative loading factor is larger than a threshold value.
25. The apparatus of claim 15, wherein the display module is configured to:
determining a target display area to which the desktop integration component belongs based on the card content of the desktop card in the desktop integration component, wherein different display areas correspond to different display priorities, and the display priorities are related to the card content;
determining a target display position of the desktop integration component in the target display area based on the historical use frequency of the desktop card in the desktop integration component, wherein different display positions correspond to different display priorities, and the display priorities and the historical use frequency are in a positive correlation;
displaying the desktop integration component at the target display location of the target display area.
26. The apparatus of claim 15, further comprising:
and the self-adaptive module is used for self-adaptively adjusting the display layout and the display content of the desktop card based on the screen display state, wherein the screen display state comprises the screen display size and/or the available display area of the screen.
27. The apparatus of claim 15, further comprising:
the card screening module is used for screening target desktop cards from the desktop cards;
the card pushing module is used for pushing the target desktop card to cross-end equipment so that the cross-end equipment can display the target desktop card, the cross-end equipment supports display of card contents in the target desktop card, and the cross-end equipment has control authority of the target desktop card.
28. The apparatus of claim 27, wherein the card filter module is configured to:
screening the target desktop card from the desktop cards based on a control relation between current equipment and the cross-end equipment, wherein the target desktop card does not contain the desktop card for controlling the current equipment under the condition that the cross-end equipment is controlled by the current equipment;
determining a target card type which is supported and displayed by the cross-terminal equipment; and screening the target desktop cards from the desktop cards based on the target card type.
29. A terminal, characterized in that the terminal comprises a processor and a memory; the memory stores at least one instruction for execution by the processor to implement a method of displaying a desktop card according to any one of claims 1 to 14.
30. A computer-readable storage medium having at least one program code stored therein, the program code being loaded and executed by a processor to implement the method of displaying a desktop card according to any one of claims 1 to 14.
31. A computer program product, characterized in that the computer program product comprises computer instructions, the computer instructions being stored in a computer readable storage medium; a processor of a computer device reads the computer instructions from the computer readable storage medium, the processor executing the computer instructions to cause the computer device to perform the method of displaying a desktop card according to any one of claims 1 to 14.
CN202211117854.9A 2022-09-14 2022-09-14 Desktop card display method, device, terminal, storage medium and program product Active CN115309299B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202211117854.9A CN115309299B (en) 2022-09-14 2022-09-14 Desktop card display method, device, terminal, storage medium and program product

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202211117854.9A CN115309299B (en) 2022-09-14 2022-09-14 Desktop card display method, device, terminal, storage medium and program product

Publications (2)

Publication Number Publication Date
CN115309299A true CN115309299A (en) 2022-11-08
CN115309299B CN115309299B (en) 2024-02-23

Family

ID=83866806

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202211117854.9A Active CN115309299B (en) 2022-09-14 2022-09-14 Desktop card display method, device, terminal, storage medium and program product

Country Status (1)

Country Link
CN (1) CN115309299B (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116420989A (en) * 2023-06-09 2023-07-14 广州美术学院 Intelligent office workstation

Citations (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20080307385A1 (en) * 2007-06-11 2008-12-11 Sap Ag Enhanced Widget Composition Platform
US20120240068A1 (en) * 2011-03-17 2012-09-20 Mccann William Jon Assigning labels to desktop items
CN107733988A (en) * 2017-08-18 2018-02-23 上海掌门科技有限公司 A kind of method and apparatus for being used to provide desktop
CN108804491A (en) * 2018-03-27 2018-11-13 优视科技新加坡有限公司 item recommendation method, device, computing device and storage medium
CN108803973A (en) * 2018-06-01 2018-11-13 西安领讯卓越信息技术有限公司 A kind of video information component groups display methods for mobile terminal
CN111176766A (en) * 2019-12-16 2020-05-19 青岛海信移动通信技术股份有限公司 Communication terminal and component display method
CN111752647A (en) * 2019-03-27 2020-10-09 西安欧思奇软件有限公司 Card information display method and device, computer equipment and storage medium
CN112068744A (en) * 2020-08-26 2020-12-11 深圳传音控股股份有限公司 Interaction method, mobile terminal and storage medium
CN113434235A (en) * 2021-06-30 2021-09-24 大众问问(北京)信息科技有限公司 Vehicle-mounted desktop display method and device, computer equipment and storage medium
CN113778574A (en) * 2020-06-08 2021-12-10 荣耀终端有限公司 Card sharing method, electronic equipment and communication system
CN114510170A (en) * 2021-03-04 2022-05-17 聚好看科技股份有限公司 Component display method and display device
CN114816610A (en) * 2021-01-29 2022-07-29 华为技术有限公司 Page classification method, page classification device and terminal equipment
CN114842270A (en) * 2022-05-31 2022-08-02 中冶赛迪技术研究中心有限公司 Target image classification method and device, electronic equipment and medium
CN115033331A (en) * 2022-06-28 2022-09-09 Oppo广东移动通信有限公司 Card display method and related product

Patent Citations (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20080307385A1 (en) * 2007-06-11 2008-12-11 Sap Ag Enhanced Widget Composition Platform
US20120240068A1 (en) * 2011-03-17 2012-09-20 Mccann William Jon Assigning labels to desktop items
CN107733988A (en) * 2017-08-18 2018-02-23 上海掌门科技有限公司 A kind of method and apparatus for being used to provide desktop
CN108804491A (en) * 2018-03-27 2018-11-13 优视科技新加坡有限公司 item recommendation method, device, computing device and storage medium
CN108803973A (en) * 2018-06-01 2018-11-13 西安领讯卓越信息技术有限公司 A kind of video information component groups display methods for mobile terminal
CN111752647A (en) * 2019-03-27 2020-10-09 西安欧思奇软件有限公司 Card information display method and device, computer equipment and storage medium
CN111176766A (en) * 2019-12-16 2020-05-19 青岛海信移动通信技术股份有限公司 Communication terminal and component display method
CN113778574A (en) * 2020-06-08 2021-12-10 荣耀终端有限公司 Card sharing method, electronic equipment and communication system
CN112068744A (en) * 2020-08-26 2020-12-11 深圳传音控股股份有限公司 Interaction method, mobile terminal and storage medium
CN114816610A (en) * 2021-01-29 2022-07-29 华为技术有限公司 Page classification method, page classification device and terminal equipment
CN114510170A (en) * 2021-03-04 2022-05-17 聚好看科技股份有限公司 Component display method and display device
CN113434235A (en) * 2021-06-30 2021-09-24 大众问问(北京)信息科技有限公司 Vehicle-mounted desktop display method and device, computer equipment and storage medium
CN114842270A (en) * 2022-05-31 2022-08-02 中冶赛迪技术研究中心有限公司 Target image classification method and device, electronic equipment and medium
CN115033331A (en) * 2022-06-28 2022-09-09 Oppo广东移动通信有限公司 Card display method and related product

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116420989A (en) * 2023-06-09 2023-07-14 广州美术学院 Intelligent office workstation
CN116420989B (en) * 2023-06-09 2023-08-22 广州美术学院 Intelligent office workstation

Also Published As

Publication number Publication date
CN115309299B (en) 2024-02-23

Similar Documents

Publication Publication Date Title
US7290223B2 (en) Interface for displaying and exploring hierarchical information
CN109977956A (en) A kind of image processing method, device, electronic equipment and storage medium
US20170221242A1 (en) Automatic overdraw reduction before rendering
US20140123179A1 (en) Zoomable content recommendation system
CN112235520B (en) Image processing method and device, electronic equipment and storage medium
US20110193867A1 (en) Method and apparatus for producing dynamic effect of character capable of interacting with image
WO2012169135A1 (en) Information processing device, information processing method and computer program product
CN111353392A (en) Face change detection method, device, equipment and storage medium
CN103384896A (en) Digital comic editing device and method therefor
CN115309299B (en) Desktop card display method, device, terminal, storage medium and program product
WO2011081193A1 (en) Image display system
CN112784110A (en) Key frame determination method and device, electronic equipment and readable storage medium
CN114706987B (en) Text category prediction method, device, equipment, storage medium and program product
CN102982054A (en) Information processor, information processing method and program
CN114529635A (en) Image generation method, device, storage medium and equipment
CN114548276A (en) Method and device for clustering data, electronic equipment and storage medium
CN116860114B (en) Augmented reality interaction method and related device based on artificial intelligence
CN113313066A (en) Image recognition method, image recognition device, storage medium and terminal
CN103544263B (en) Rendering method and rendering device for mobile terminal
CN111915701A (en) Button image generation method and device based on artificial intelligence
CN109886301A (en) A kind of characteristics of image arrangement judgment method and its system, terminal device
KR102530657B1 (en) Method, system, and computer program for layering recognized text in image
WO2024098878A1 (en) Information output channel selection method, information output method, apparatus, and device
CN108536510A (en) Implementation method based on human-computer interaction application program and device
KR102606878B1 (en) Method of providing personal virtual space deployment service and server performing thereof

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant