WO2022177144A1 - Procédé et dispositif électronique pour la personnalisation d'applications - Google Patents

Procédé et dispositif électronique pour la personnalisation d'applications Download PDF

Info

Publication number
WO2022177144A1
WO2022177144A1 PCT/KR2021/020339 KR2021020339W WO2022177144A1 WO 2022177144 A1 WO2022177144 A1 WO 2022177144A1 KR 2021020339 W KR2021020339 W KR 2021020339W WO 2022177144 A1 WO2022177144 A1 WO 2022177144A1
Authority
WO
WIPO (PCT)
Prior art keywords
application
electronic device
user
metadata
content
Prior art date
Application number
PCT/KR2021/020339
Other languages
English (en)
Inventor
Ankit Jain
Siba Prasad Samal
Original Assignee
Samsung Electronics Co., Ltd.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Samsung Electronics Co., Ltd. filed Critical Samsung Electronics Co., Ltd.
Publication of WO2022177144A1 publication Critical patent/WO2022177144A1/fr

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0487—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
    • G06F3/0488—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures
    • G06F3/04883—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures for inputting data by handwriting, e.g. gesture or text
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0481—Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
    • G06F3/0482—Interaction with lists of selectable items, e.g. menus
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0484—Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
    • G06F3/04842—Selection of displayed objects or displayed text elements
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0484—Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
    • G06F3/04847—Interaction techniques to control parameter settings, e.g. interaction with sliders or dials
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04M—TELEPHONIC COMMUNICATION
    • H04M1/00—Substation equipment, e.g. for use by subscribers
    • H04M1/72—Mobile telephones; Cordless telephones, i.e. devices for establishing wireless links to base stations without route selection
    • H04M1/724—User interfaces specially adapted for cordless or mobile telephones
    • H04M1/72469—User interfaces specially adapted for cordless or mobile telephones for operating the device by selecting functions from two or more displayed items, e.g. menus or icons
    • H04M1/72472—User interfaces specially adapted for cordless or mobile telephones for operating the device by selecting functions from two or more displayed items, e.g. menus or icons wherein the items are sorted according to specific criteria, e.g. frequency of use
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N5/00—Computing arrangements using knowledge-based models
    • G06N5/02—Knowledge representation; Symbolic representation
    • G06N5/022—Knowledge engineering; Knowledge acquisition
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04M—TELEPHONIC COMMUNICATION
    • H04M1/00—Substation equipment, e.g. for use by subscribers
    • H04M1/72—Mobile telephones; Cordless telephones, i.e. devices for establishing wireless links to base stations without route selection
    • H04M1/724—User interfaces specially adapted for cordless or mobile telephones
    • H04M1/72448—User interfaces specially adapted for cordless or mobile telephones with means for adapting the functionality of the device according to specific conditions

Definitions

  • the present disclosure relates to creation and sharing of intelligent on-device Artificial intelligence (AI) models across applications and more specifically related to a method and electronic device for personalization of applications for better user experience.
  • AI Artificial intelligence
  • certain information e.g. profile information
  • the principal object of the embodiments herein is to obtain user’s functional preferences from user interactions which represent application-level functionalities of a first application of the electronic device and personalizing run-time behavior of a second non-personalized application of the electronic device using knowledge of obtained from the functional preferences of the user.
  • Another object of the embodiment herein is to enable on-device AI models which are capable of processing the functional preferences across applications of the electronic device while maintaining privacy and applications proprietaries.
  • inventions herein disclose a method for personalization of applications.
  • the method includes displaying, by the electronic device, content of at least one page of a first application of the electronic device, where the content of the at least one page of the first application is non-personalized to a user of the electronic device. Further, the method includes extracting, by the electronic device, a plurality of content metadata (e.g. keywords, filters/sort options of application with keywords, at least one symbol to indicate a functionality of application, etc.) from the displayed content of at least one page of the first application, where the extraction of the plurality of content metadata is based on previous user interactions with other applications of the electronic device.
  • a plurality of content metadata e.g. keywords, filters/sort options of application with keywords, at least one symbol to indicate a functionality of application, etc.
  • the method includes applying, by the electronic device, a knowledge-based model on the extracted plurality of content metadata to obtain functional preferences of the user of the electronic device.
  • the knowledge-based model is associated with at least one of at least one-second application of the electronic device and/or at least one edge device (e.g. IoT device).
  • the method includes personalizing, by the electronic device, the content (e.g. navigation flow) of the at least one page of the first application of the electronic device based on the obtained functional preferences of the user of the electronic device.
  • the functional preferences of an application are the configuration values used in input components of the application which perform a function in the application, consequently introducing a behavior, navigation, an application flow, or operation change aiding the user in consuming application data efficiently.
  • a price range e.g. functional preference
  • 345 ? 567 will change items displayed on the listing page helping the user to navigate efficiently.
  • the functional preferences of the user can change an application’s component values like sorting value (e.g. price, latest, relevance), filters (e.g. color, ratings, offers, availability, category, size), brand preferences, etc.
  • the method further includes determining, by the electronic device, a plurality of keywords of the at least one page associated with the first application of the electronic device. Further, the method includes determining, by the electronic device, a plurality of filters associated with the first application of the electronic device. Further, the method includes determining, by the electronic device, at least one application-level functionality (e.g. scrolling, menu selection, search bar, option selection, etc.) associated with the first application of the electronic device.
  • application-level functionality e.g. scrolling, menu selection, search bar, option selection, etc.
  • the method further includes applying, by the electronic device, a machine learning model (e.g. recurrent neural network (RNN) model, Convolution Neural Network (CNN), Bidirectional Long short-term memory (Bi-LSTM) based Embedding) on the extracted plurality of content metadata to generate a plurality of queries.
  • a machine learning model e.g. recurrent neural network (RNN) model, Convolution Neural Network (CNN), Bidirectional Long short-term memory (Bi-LSTM) based Embedding
  • RNN recurrent neural network
  • CNN Convolution Neural Network
  • Bi-LSTM Bidirectional Long short-term memory
  • the method includes determining, by the electronic device, that at least one query from the plurality of the queries is matched with at least one metadata from the plurality of metadata. Further, the method includes mapping, by the electronic device, the at least one query with the at least one metadata to obtain the functional preferences of the user of the electronic device in response to determining that the at least query from the plurality of the queries is matched with the at least one metadata from the plurality of metadata.
  • the method also includes determining which on-device AI model from the plurality of AI models is best suited for a specific query from the plurality of queries. Also, the method includes the use of multiple on-device AI models together to answer the specific query or to answer various specificities of a single query using multiple on-device AI models.
  • an on-device personalization method for applications in an electronic device includes monitoring over time, by the electronic device, a plurality of interactions (e.g. click, menu selection, etc.) carried out by a user of the electronic device 100 on a first application of the electronic device. Further, the method includes identifying, by the electronic device, at least one user functional preference (e.g. color, price, etc.) and at least one pattern of usage relating to the first application from the monitored plurality of interactions. Further, the method includes building, by the electronic device, an on-device knowledge base based on the identified at least one user functional preferences and the identified at least one pattern of usage. Further, the method includes providing, by the electronic device, access of the on-device knowledge base to a second application of the electronic device.
  • a plurality of interactions e.g. click, menu selection, etc.
  • the method includes identifying, by the electronic device, at least one user functional preference (e.g. color, price, etc.) and at least one pattern of usage relating to the first application from the monitored
  • the embodiments herein provide the electronic device for personalization of applications.
  • the electronic device includes a processor and a memory.
  • the processor is configured to display content of at least one page of the first application of the electronic device, where the content of the at least one page of the first application is non-personalized to the user of the electronic device.
  • the processor is configured to extract the plurality of content metadata from the displayed content of at least one page of the first application, where the extraction of the plurality of content metadata is based on previous user interactions with other applications of the electronic device.
  • the processor is configured to apply the knowledge-based model on the extracted plurality of content metadata to obtain functional preferences of the user of the electronic device, where the knowledge-based model is associated with at least one of at least one-second application of the electronic device or/and at least one edge device. Further, the processor is configured to personalize the content of the at least one page of the first application of the electronic device based on the obtained preferences (i.e. functional preferences) of the user of the electronic device.
  • FIGS. 1A-1C are examples illustration in which system provides a manually customized user interface for an application of an electronic device, according to prior art
  • FIG. 2 illustrates a block diagram of an electronic device for personalized application, according to an embodiment as disclosed herein;
  • FIG. 3 is a flow diagram illustrating various operations for personalized application, according to an embodiment as disclosed herein;
  • FIG. 4 is a functional diagram illustrating various modules and modules’ operations for personalized application, according to an embodiment as disclosed herein;
  • FIGS. 5A-5H illustrating examples of various operations associated with an AI broker engine to personalize application of the electronic device, according to an embodiment as disclosed herein;
  • FIGS. 6A-6F are examples illustrating a difference between the existing system providing manually customized user interface and a proposed personalized application, according to an embodiment as disclosed herein;
  • FIGS. 7A-7C are examples illustrating various invocation methods for the AI broker engine to personalize applications of the electronic device, according to an embodiment as disclosed herein;
  • FIGS. 8A-8H are examples scenario in which the AI broker engine builds a knowledge base in the electronic device, according to an embodiment as disclosed herein;
  • FIGS. 9A-9B are examples scenario in which the AI broker engine personalizes an application based on the build knowledge base of the electronic device, according to an embodiment as disclosed herein.
  • circuits may, for example, be embodied in one or more semiconductor chips, or on substrate supports such as printed circuit boards and the like.
  • circuits constituting a block may be implemented by dedicated hardware, or by a processor (e.g., one or more programmed microprocessors and associated circuitry), or by a combination of dedicated hardware to perform some functions of the block and a processor to perform other functions of the block.
  • a processor e.g., one or more programmed microprocessors and associated circuitry
  • Each block of the embodiments may be physically separated into two or more interacting and discrete blocks without departing from the scope of the invention.
  • the blocks of the embodiments may be physically combined into more complex blocks without departing from the scope of the invention
  • the AI model (on-device AI model) is limited to/ associated with a single application of the electronic device and there is no mechanism in the existing systems to enable the AI model usage across various applications of the electronic device.
  • the AI model often requires a large amount of space, with each application having the AI model that represents the same user personalization feature, and thus ends up occupying more resources.
  • some AI models associated with single application are better than other models associated with other applications but deriving the same predictions and so some apps perform better than the other and there is no consistency of user experience across the device.
  • the AI model is unable to retrain itself by adjusting weights and parameters based on recent user interactions with the applications and the device and so eventually the AI model becomes stagnant and doesn’t represent the user anymore.
  • each on-device AI model can respond to different inputs when the inputs are well-formed and meet the needs of the model, there is no mechanism for sharing inputs between applications and on-device AI models.
  • the each on-device AI model can take requests from various applications of the electronic device and produce some results based on various application platforms which requires integration and tight coupling between the application of the on-device AI model and the external applications trying to use the on-device AI model. This required all the applications to be aware about all the other applications that have the AI model and not every application supports such integrations, there is no standardization of model data transfer methodologies in the existing system.
  • inventions herein disclose a method for personalization of electronic device applications.
  • the method includes displaying, by the electronic device, content of at least one page of a first application of the electronic device, where the content of the at least one page of the first application is non-personalized to a user of the electronic device. Further, the method includes extracting, by the electronic device, a plurality of content metadata from the displayed content of at least one page of the first application, where the extraction of the plurality of content metadata is based on previous user interactions with other applications of the electronic device.
  • the method includes applying, by the electronic device, a knowledge-based model on the extracted plurality of content metadata to obtain functional preferences of the user of the electronic device, wherein the knowledge-based model is associated with at least one of at least one-second application of the electronic device or/and at least one edge device. Further, the method includes personalizing, by the electronic device, the content of the at least one page of the first application of the electronic device based on the obtained functional preferences of the user of the electronic device.
  • the embodiments herein provide the electronic device for personalization of applications.
  • the electronic device includes a processor and a memory.
  • the processor is configured to display content of at least one page of the first application of the electronic device, where the content of the at least one page of the first application is non-personalized to the user of the electronic device.
  • the processor is configured to extract the plurality of content metadata from the displayed content of at least one page of the first application, where the extraction of the plurality of content metadata is based on previous user interactions with other applications of the electronic device.
  • the processor is configured to apply the knowledge-based model on the extracted plurality of content metadata to obtain functional preferences of the user of the electronic device, where the knowledge-based model is associated with at least one of at least one-second application of the electronic device or/and at least one edge device. Further, the processor is configured to personalize the content of the at least one page of the first application of the electronic device based on the obtained functional preferences of the user of the electronic device.
  • FIGS. 2 through 9 there are shown preferred embodiments.
  • FIGS. 1A-1C are examples illustration in which system provides a manually customized user interface for the application of an electronic device 100, according to prior art.
  • the electronic device 100 can be, for example, but not limited to a smartphone, a laptop, a desktop, a smart watch, a smart TV or a like.
  • a user of the electronic device 100 wants to order food using one of food applications of the electronic device 100.
  • the user of the electronic device 100 opens the food application 1, the food application displays a search option for the restaurant with various filter options (e.g. sort by, cuisine, rating, and cost per person, etc.) associated with the food application.
  • various filter options personalized (e.g. most used and recently used, etc.) application and provides better user experience to the user, the user selects cuisine (e.g. Chinese, burger ) from the various filter options, so that the food application 2 is personalized accordingly.
  • the user 3 of the electronic device 100 selects cost from the various filter options, so that the food application is personalized accordingly.
  • FIG. 2 illustrates a block diagram of the electronic device 100 for personalized application, according to an embodiment as disclosed herein.
  • the electronic device 100 includes a memory 110, a processor 120, a communicator 130, and a display 140.
  • the memory 110 stores a plurality of content metadata is based on previous user interactions with other applications of the electronic device 100, and meta data associated with a knowledge-based model (e.g. on device AI models).
  • the memory 110 stores instructions to be executed by the processor 120.
  • the memory 110 may include non-volatile storage elements. Examples of such non-volatile storage elements may include magnetic hard discs, optical discs, floppy discs, flash memories, or forms of electrically programmable memories (EPROM) or electrically erasable and programmable (EEPROM) memories.
  • EPROM electrically programmable memories
  • EEPROM electrically erasable and programmable
  • the memory 110 may, in some examples, be considered a non-transitory storage medium.
  • the term “non-transitory” may indicate that the storage medium is not embodied in a carrier wave or a propagated signal.
  • non-transitory should not be interpreted that the memory 110 is non-movable.
  • a non-transitory storage medium may store data that can, over time, change (e.g., in Random Access Memory (RAM) or cache).
  • the memory 110 can be an internal storage unit or it can be an external storage unit of the electronic device 100, a cloud storage, or any other type of external storage.
  • the memory 110 includes an application repository 110a.
  • the application repository 110a stores metadata information associated with the application 110aa to 110an of the electronic device 100.
  • the application 110aa to 110an can be, for example, but not limited to shopping e-commerce application, travel application, food application, home surveillance application, grocery application, car rental application or a like.
  • the processor 120 controls the memory 110, the communicator 130, and the display 140.
  • the processor 120 executes instructions stored in the memory 110 and performs various processes.
  • the processor may include one or a plurality of processors, may be a general purpose processor, such as a central processing unit (CPU), an application processor (AP), or the like, a graphics-only processing unit such as a graphics processing unit (GPU), a visual processing unit (VPU), and/or an Artificial intelligence (AI) dedicated processor such as a neural processing unit (NPU).
  • the processor includes an AI broker engine 121, and a knowledge base 122.
  • the AI broker engine 121 and knowledge base 122 are implemented by processing circuitry such as logic gates, integrated circuits, microprocessors, microcontrollers, memory circuits, passive electronic components, active electronic components, optical components, hardwired circuits, or the like, and may optionally be driven by firmware.
  • the circuits may, for example, be embodied in one or more semiconductor chips, or on substrate supports such as printed circuit boards and the like.
  • the AI broker engine 121 includes a keyword extraction engine 121a, a query generator 121b, a data aggregator 121c, and a personalize engine 121d.
  • the keyword extraction engine 121a extracts a plurality of content metadata from the displayed content of at least one page of the first application of the electronic device 100. Further, the keyword extraction engine 121a determines content of the at least one page associated with the first application of the electronic device 100. Further, the keyword extraction engine 121a determines a plurality of filters associated with the first application of the electronic device 100. Further, the keyword extraction engine 121a determines at least one application-level functionality associated with the first application of the electronic device 100. Further details of the keyword extraction engine 121a are provided in the FIG. 4.
  • the query generator 121b applies a machine learning model (e.g. RNN model, CNN, Bi-LSTM based Embedding) on the extracted plurality of content metadata to generate a plurality of queries. Further, the query generator 121) passes the plurality of queries to the knowledge base 122 to obtain the functional preferences of the user of the electronic device 100.
  • the knowledgebase 122 includes a query model discovery 122a, an input formulator 122b, and an output formulator 122c.
  • the knowledgebase 122 is associated with on-device AI models present inside different applications of the electronic device 100, AI models present inside edge device, and created knowledge graph/base of previous interactions of the user of the electronic device 100 on different applications, and using that input and output for intelligence.
  • the query model discovery 122a obtains a plurality of metadata associated with at least one of at least one-second application of the electronic device 100 or/and at least one edge device . Further, the query model discovery 122a determines that at least one query from the plurality of the queries is matched with at least one metadata from the plurality of metadata. Further, the query model discovery 122a maps the at least one query with the at least one metadata to obtain the functional preferences of the user of the electronic device 100 in response to determining that the at least query from the plurality of the queries is matched with the at least one metadata from the plurality of metadata. Further, the query model discovery 122a determines which on-device AI model from the plurality of AI models is best suited for a specific query or maps the specific query with a specific on-device AI model (mapped on-device AI model).
  • the input formulator 122b takes the specific query and determines an appropriate input from the query which can be handed over to the mapped AI model to obtain user functional preferences, based on the specific query. Further, the inputs are run through the on-device AI model to get outputs.
  • the output formulator 122c receives output from the on-device models and transforms it into an output format that can be used by the AI broker engine 121 to change various filters and other functional capabilities of the first application and provide the functional preferences of the user of the electronic device 100.
  • the data aggregator 121c aggregates different queries of the on-device AI models into a parsable single format and removes redundancies.
  • the personalize engine 121d personalizes the content of the at least one page of the first application of the electronic device 100 based on the obtained functional preferences of the user of the electronic device 100.
  • the communicator 130 communicates internally between internal hardware components and with external devices via one or more networks.
  • the display 140 displays content of the at least one page of the first application of the electronic device (00, where the display content of the first application is either non-personalized to the user of the electronic device 100 or personalized to the user of the electronic device 100.
  • At least one of the plurality of modules may be implemented through an AI model.
  • a function associated with AI may be performed through non-volatile memory, the volatile memory, and the processor.
  • the processor 120 may include one or a plurality of processors.
  • one or a plurality of processors may be a general-purpose processor, such as a central processing unit (CPU), an application processor (AP), or the like, a graphics-only processing unit such as a graphics processing unit (GPU), a visual processing unit (VPU), and/or an AI-dedicated processor such as a neural processing unit (NPU).
  • CPU central processing unit
  • AP application processor
  • GPU graphics-only processing unit
  • VPU visual processing unit
  • NPU neural processing unit
  • the one or a plurality of processors control the processing of the input data in accordance with a predefined operating rule or artificial intelligence (AI) model stored in the non-volatile memory and the volatile memory.
  • the predefined operating rule or artificial intelligence model is provided through training or learning.
  • learning means that, by applying a learning process to a plurality of learning data, a predefined operating rule or AI model of a desired characteristic is made.
  • the learning may be performed in a device itself in which AI according to an embodiment is performed, and/o may be implemented through a separate server/system.
  • the AI model may consist of a plurality of neural network layers. Each layer has a plurality of weight values and performs a layer operation through calculation of a previous layer and an operation of a plurality of weights.
  • Examples of neural networks include, but are not limited to, convolutional neural network (CNN), deep neural network (DNN), recurrent neural network (RNN), restricted Boltzmann Machine (RBM), deep belief network (DBN), bidirectional recurrent deep neural network (BRDNN), generative adversarial networks (GAN), and deep Q-networks.
  • the learning process is a method for training a predetermined target device (for example, a robot) using a plurality of learning data to cause, allow, or control the target device to make a determination or prediction.
  • Examples of learning process include, but are not limited to, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning.
  • FIG. 2 shows various hardware components of the electronic device 100 but it is to be understood that other embodiments are not limited thereon.
  • the electronic device 100 may include less or more number of components.
  • the labels or names of the components are used only for illustrative purpose and does not limit the scope of the invention.
  • One or more components can be combined together to perform same or substantially similar function to personalize the content of the at least one page of the first application of the electronic device 100.
  • FIG. 3 is a flow diagram S300 illustrating various operations for personalized application, according to an embodiment as disclosed herein.
  • the method includes displaying, by the electronic device 100, content of the at least one page of the first application of the electronic device 100, the content of the at least one page of the first application is non-personalized to the user of the electronic device 100.
  • the method includes extracting, by the electronic device 100, the plurality of content metadata from the displayed content of at least one page of the first application.
  • the method includes applying, by the electronic device 100, the knowledge-based model on the extracted plurality of content metadata to obtain functional preferences of the user of the electronic device 100, the knowledge-based model is associated with at least one of at least one-second application of the electronic device 100 and the at least one edge device.
  • the method includes personalizing, by the electronic device 100, the content (e.g. or changing the navigation flow) of the at least one page of the first application of the electronic device 100 based on the obtained functional preferences of the user of the electronic device 100.
  • FIG. 4 is a functional diagram illustrating various modules and modules operations for personalized application, according to an embodiment as disclosed herein.
  • the user of the electronic device 100 opens/displays the first application of the electronic device 100, wherein the content of the at least one page of the first application of the electronic device 100 is non-personalized to the user of the electronic device 100.
  • the keyword extraction engine 121a extracts the plurality of content metadata from the displayed content of the at least one page of the first application of the electronic device 100, the keyword extraction engine 121a includes an image processor 121aa, an optical character recognition (OCR) engine 121ab, a classifier 121ac, a layout analyzer 121ad, and a keyword enricher 121ae.
  • OCR optical character recognition
  • the image processor 121aa removes irrelevant data (e.g. advertisement, images) from the displayed content of the at least one page of the first application of the electronic device 100.
  • the OCR engine 121ab analyses meta-data/text information (e.g. cloth type, price, description text, food, spice, cuisine, rating, discount, categories, vocabulary, etc.) of displayed content of the at least one page of the first application of the electronic device 100. Further, the OCR engine 121ab retrieves the text information from the displayed content according to the layout shown in the display(or screen) 140.
  • the classifier 121ac e.g. reinforcement learning-based deep neural network (DNN) classifier uses the text information to obtain user personalization parameters (i.e. keywords) and determines the value of each parameter.
  • DNN deep neural network
  • the layout analyzer 121ad uses layout patterns and elements arrangement around the user personalization parameters to determine the generic input values that can be used for them.
  • the keyword enricher 121ae adds more data (relevant content) to the user personalization parameters from the first application ⁇ s meta-data and similar installed applications’ meta-data to segregate parameters shown in the Table 1.
  • the query generator 121b includes a machine learning model 121ba and a query relevancy scorer 121bb.
  • the machine learning model 121ba is used to form the plurality of queries from the user personalization parameters (i.e. keywords) and the enriched keywords associated with the user personalization parameters.
  • Example of the plurality of queries such as first query (does the user want toppings on a pizza), second query (does the user like tomato toppings), third query (what price does the user prefer for the toppings).
  • the plurality of queries passes to the query relevancy scorer 121bb (e.g. second-classifier).
  • the query relevancy scorer 121bb calculates a relevancy score for each query from the plurality of queries to know which are relevant queries to pass to the knowledge base 122.
  • the knowledge base 122 is associated with at least one of on-device AI models present inside different applications of the electronic device 100, AI models present inside edge device, and knowledge graph/base of previous interactions of the user of the electronic device 100 on different applications, and using that input and output for intelligence.
  • the details of the knowledge base 122 are already explained in the FIG. 2.
  • Example (categories) of the on-device AI models is given in the Table 2.
  • the data aggregator 121c aggregates different queries of the on-device AI models into the parsable single format and removes redundancies.
  • the personalize engine 121d includes a result analyzer 121da, a layout mapper 121db, and a value fitter 121dc.
  • the result analyzer 121da analyzes the output from output formulator 122c and determines the personalization parameters which can be used to adjust the first application for personalization.
  • the layout mapper 121db finds out the exact components that needs to be changed or the timed actions that need to be taken (like quick scrolling through items) depending on the results of the result analyzer 121da and maps these first application components to the personalization parameters.
  • the value fitter 121dc then pulls the values from the result analyzer 121da and changes the layouts components extracted by the layout mapper 121db which can result in the personalization of the first application.
  • the personalize engine 121d personalizing the content of the at least one page of the first application of the electronic device 100 based on the obtained functional preferences of the user of the electronic device 100.
  • FIGS. 5A-5H illustrating examples of various operations associated with the AI broker engine 121 to personalize the application of the electronic device 100, according to an embodiment as disclosed herein.
  • the user of the electronic device 100 wants to order food using one of the food applications of the electronic device 100.
  • the user opens the food application, the food application displays the search option for the restaurant with various filter options associated with the food application and list of various restaurants.
  • the image processor 121aa removes irrelevant data (i.e. images).
  • the OCR engine 121ab analyses the text information associated with the food application.
  • Example of the text information such as food cuisine type (Drink, Alcohol, Shake, North Indian, Chinese, Fast Food), price, discount code, rating, and spice level.
  • the classifier 121ac uses the text information to obtain user personalization parameters (i.e.
  • keywords and determines the value of each parameter (e.g. spice level (0.89), food cuisine type (0.85), and price (0.86)).
  • the layout analyzer 121ad uses layout patterns and elements arrangement around the user personalization parameters to determine the generic input values (e.g. spice level (incremental class), food cuisine type (word class), and price (range class)).
  • keyword enricher 121ae adds more data (relevant content) to the user personalization parameters from the first application ⁇ s meta-data and similar installed applications’ meta-data to segregate parameters. For example, Food, ingredients, black pepper are the relevant content for the spice level. Delivery, culture are the relevant content for the food cuisine type. Food, delivery, discounts are the relevant content for the price.
  • the query generator 121b generates the plurality of queries 516 from the user personalization parameters.
  • Example of the plurality of queries such as first query (What food cuisine user likes?), second query (What is user’s spice level and ingredients), third query (What price range of food user prefer?), fourth query (What spice level user can buy?), and fifth query (What is the range of food cuisines?).
  • the plurality of queries 516 passes to the query relevancy scorer 121bb.
  • the query relevancy scorer 121bb calculates the relevancy score for each query from the plurality of queries to know which are relevant queries to pass to the knowledge base 122 (i.e. pass only first query, second query and third query).
  • the query model discovery 122a obtains the plurality of metadata associated with at least one of at least one-second application (e.g. MD love, E-pay) of the electronic device 100 or/and at least one edge device (e.g. IoT (family hub)), if the at least one edge device is available.
  • at least one-second application e.g. MD love, E-pay
  • at least one edge device e.g. IoT (family hub)
  • the plurality of metadata such as user food cuisine, user food type, food by time, food payment, bill payment retail payment, credit card, ingredients, food flavor, food category, etc.
  • the query model discovery 122a determines that at least one query from the plurality of the queries 516 is matched with at least one metadata from the plurality of metadata.
  • the query model discovery 122a maps the at least one query with the at least one metadata to obtain the functional preferences of the user of the electronic device 100 in response to determining that the at least query from the plurality of the queries 516 is matched with the at least one metadata from the plurality of metadata, as shown in Table 3.
  • the query model discovery 122a determines which on-device AI model from the plurality of AI models (e.g. favorite food model, payment range model, user IoT device food model) is best suited for a specific query or maps the specific query with specific on-device AI model. As per Table. 3, for the first query (What food cuisine user likes?) best suited AI model is the favourite food model.
  • on-device AI model e.g. favorite food model, payment range model, user IoT device food model
  • the input formulator 122b creates an input from the mapped on-device AI model and the specific query, give the created input to the mapped on-device AI model, based on the specific query. Further, the inputs are run through the on-device AI model to get outputs, as shown in the Table 4.
  • the output formulator 122c receives output from the on-device models and provides preferences 526 of the user of the electronic device 100.
  • the query model discovery 122a receives the plurality of the queries 516.
  • the query model discovery 122a maps the at least one query to appropriate AI model using meta-data formed in part by using previous interactions of the user of the electronic device 100 on different applications and using that input and output for intelligence.
  • the output formulator 122c receives output from the on-device models and provides preferences 526 of the user of the electronic device (100).
  • the example is given in the FIG. 5A-5H related to the food application of the electronic device 100, can be extended/applicable to other applications of the electronic device 100.
  • the other applications can be, for example, but not limited to shopping e-commerce application, travel application, food application, home surveillance application, grocery application, car rental application, or a like.
  • FIGS. 6A-6F is an example illustrating a difference between the existing system providing manually customized user interface and the proposed personalized application, according to an embodiment as disclosed herein.
  • the user wants to order food using one of the food applications of the electronic device 100.
  • the user opens the food application, the food application displays a search option for the restaurant with various filter options 602 (e.g. sort by, cuisine, rating, and cost per person, etc.) associated with the food application.
  • various filter options personalized food application and provide a better user experience.
  • the user selects cost 604, and cuisine 606 from the various filter options so that the food application is personalized accordingly. So, in the existing system application setting, filters and preferences are not personalized for the user. Whenever the user opens the food application, the user has to set all the filters associated with the food application, the process is manual and time-consuming.
  • the content of the food application is automatically personalized 608, the user does not need to select the filters (i.e. cost 610, and cuisine 612) manually.
  • the knowledge from at least one of the other applications of the electronic device 100, and edge device is pre-apply to settings/filters of the food application, provides a better user experience.
  • FIGS. 7A-7C are examples illustrating various invocation methods for the AI broker engine 121 to personalize applications of the electronic device 100, according to an embodiment as disclosed herein.
  • the AI broker engine 121 determines one or more personalization requirements of the application of the electronic device 100 based on at least one of a feature request received from the application of the electronic device 100.
  • a feature request received from the application of the electronic device 100.
  • the feature request (user personalization parameters) is used to form user preference queries which can give some insight to user preferences to the application, which are passed to the on-device model knowledge base (i.e. knowledge base 122).
  • the on-device model knowledge base resolves the queries to the on-device /edge models that can process that query and get the results.
  • the result from on-device models is used to reconfigure the application.
  • application can itself create the feature request (don’t just rely on OCR for processing) and send the feature request to the AI broker engine 121. Further, for certain categories of application, the AI broker engine 121 invokes feature requests based on common data. For example, food ordering application, the AI broker engine 121 induces common feature requests such as what food type user like, price range of the user, favorite restaurant of the user.
  • FIGS. 8A-8H are examples scenario in which the AI broker engine 121 builds the knowledge base 122 in the electronic device 100, according to an embodiment as disclosed herein.
  • the user of the electronic device 100 wants to order food using one of the food application (i.e. UbrEt food application) of the electronic device 100, where the food application have a list of various restaurant with various filter options.
  • the user of the electronic device 100 selects a delivery option from the various filter options.
  • the AI broker engine 121 builds the knowledge base 122 (i.e. food and delivery) in the electronic device 100.
  • the user of the electronic device 100 selects a MJ pizza restaurant, a home address, a choice of crust, a veg toppings, and a dessert/drink.
  • the AI broker engine 121 builds/updates the knowledge base 122 (e.g. food, delivery, pizza, etc.) in the electronic device 100.
  • the AI broker engine 121 collects a user clicked position (e.g. using android API), data 814 from the layout analyzer 121ad and passes the collected data to on device AI models (AI model association engine) 816 when the user of the electronic device 100 selects a double cheese option 810.
  • the AI model association engine matches the collected data with meta-information of the AI model list 818.
  • the collected data passes to the input formulator 122b if the collected data matches with the meta-information of the AI model list 820.
  • the input formulator 122b connects 822 with the model through API 824 and loads the model and forwards input to the API 826.
  • the collected data passes to the knowledge base 122 if the collected data does not match with the meta-information of the AI model list 820.
  • One API is chosen 828 based on whether a previous neighbour node reference. I.e. if previously user selected something else in the application.
  • the node data is directly given to knowledge graph through API and stores in the knowledge base 122.
  • FIGS. 9A-9B are examples scenario in which the AI broker engine 121 personalizes an application based on the build knowledge base 122 of the electronic device 100, according to an embodiment as disclosed herein.
  • the user of the electronic device 100 wants to order food using one of the food application (i.e. Do-pizza application) of the electronic device 100.
  • the user of the electronic device 100 selects a double cheese pizza.
  • the AI broker engine 121 personalizes the application based on the build knowledge base 122 (Refer FIG.8).
  • the AI broker engine 121 personalizes current application (i.e. Do-pizza application) based on the build knowledge base 122 of first application (i.e. UbrEt food application).
  • information regarding the address, the choice of crust, the veg toppings, and the dessert/drink automatically fills (904a and 904b) during ordering food from the current application. So, the proposed method provides a better user interface/ navigation flow of the application, so the user can navigate faster.
  • the AI broker engine 121 obtains the user’s functional preferences from user interactions which represent application-level functionalities of the second application (i.e. UbrEt food application) of the electronic device 100 and personalizing run-time behavior of the first non-personalized application (i.e. Do-pizza application) of the electronic device 100 using knowledge of obtained from the functional preferences of the user of the electronic device 100.
  • application-level functionalities of the second application i.e. UbrEt food application
  • Do-pizza application personalizing run-time behavior of the first non-personalized application
  • the proposed method personalizes other components of the food application (e.g. price range in food application).
  • the price range was a result of personalization space transfer from the payment application to the food application.

Landscapes

  • Engineering & Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

L'invention concerne un procédé qui comprend l'affichage d'un contenu d'au moins une page d'une première application du dispositif électronique, le contenu de ladite page de la première application étant non personnalisé pour un utilisateur du dispositif électronique, l'extraction d'une pluralité de métadonnées de contenu à partir du contenu affiché d'au moins une page de la première application, l'application d'un modèle basé sur des connaissances sur la pluralité extraite de métadonnées de contenu pour obtenir des préférences fonctionnelles de l'utilisateur du dispositif électronique. Le modèle à base de connaissances est associé à au moins un élément parmi au moins une deuxième application du dispositif électronique et au moins un dispositif périphérique. De plus, le procédé comprend la personnalisation du contenu de ladite page de la première application du dispositif électronique sur la base des préférences fonctionnelles obtenues de l'utilisateur du dispositif électronique.
PCT/KR2021/020339 2021-02-18 2021-12-31 Procédé et dispositif électronique pour la personnalisation d'applications WO2022177144A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
IN202141006767 2021-02-18
IN202141006767 2021-02-18

Publications (1)

Publication Number Publication Date
WO2022177144A1 true WO2022177144A1 (fr) 2022-08-25

Family

ID=82931409

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/KR2021/020339 WO2022177144A1 (fr) 2021-02-18 2021-12-31 Procédé et dispositif électronique pour la personnalisation d'applications

Country Status (1)

Country Link
WO (1) WO2022177144A1 (fr)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2016173623A (ja) * 2015-03-16 2016-09-29 エヌ・ティ・ティ・コミュニケーションズ株式会社 コンテンツ提供装置、コンテンツ提供方法及びコンテンツ提供プログラム
US9785978B1 (en) * 2014-03-26 2017-10-10 EMC IP Holding Company LLC Dynamic content control in an information processing system
US9953011B1 (en) * 2013-09-26 2018-04-24 Amazon Technologies, Inc. Dynamically paginated user interface
US20190073412A1 (en) * 2017-09-05 2019-03-07 Samsung Electronics Co., Ltd. Personalized automatic tagging
KR20210015524A (ko) * 2019-08-02 2021-02-10 삼성전자주식회사 사용자 관심도의 산출 방법 및 이를 지원하는 전자 장치

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9953011B1 (en) * 2013-09-26 2018-04-24 Amazon Technologies, Inc. Dynamically paginated user interface
US9785978B1 (en) * 2014-03-26 2017-10-10 EMC IP Holding Company LLC Dynamic content control in an information processing system
JP2016173623A (ja) * 2015-03-16 2016-09-29 エヌ・ティ・ティ・コミュニケーションズ株式会社 コンテンツ提供装置、コンテンツ提供方法及びコンテンツ提供プログラム
US20190073412A1 (en) * 2017-09-05 2019-03-07 Samsung Electronics Co., Ltd. Personalized automatic tagging
KR20210015524A (ko) * 2019-08-02 2021-02-10 삼성전자주식회사 사용자 관심도의 산출 방법 및 이를 지원하는 전자 장치

Similar Documents

Publication Publication Date Title
US9779179B2 (en) Referent based search suggestions
US11003678B2 (en) Method and system for presenting a search result in a search result card
TWI604325B (zh) 搜尋輔助系統與方法
US9916363B2 (en) Method and apparatus for flexible diversification of recommendation results
US20230162005A1 (en) Neural network distillation method and apparatus
US20140280093A1 (en) Social entity previews in query formulation
US8878785B1 (en) Intent determination using geometric shape input
US20150212695A1 (en) Suggested action feedback
CN113850393A (zh) 基于图学习的特征工程的系统和方法
US9032316B1 (en) Value-based presentation of user-selectable computing actions
US11176626B1 (en) Personalizing food discovery and search based on inferred taste preference
KR101981136B1 (ko) 추천 컨텐츠 제공 시스템, 방법 및 프로그램
US8825671B1 (en) Referent determination from selected content
US20160335697A1 (en) Computer implemented method for ranking data & removing dependencies on menus or multiple fields and controls
US20230259575A1 (en) Systems and methods for implementing session cookies for content selection
CN111859147B (zh) 对象推荐方法、对象推荐装置和电子设备
US20180240023A1 (en) Methods and systems for selecting targeted content by machine learning algorithm
WO2022177144A1 (fr) Procédé et dispositif électronique pour la personnalisation d'applications
CN115062216A (zh) 用于搜索的方法、装置、设备和存储介质
US10990601B1 (en) Dynamic optimization of variant recommendations
US11797632B2 (en) Image reranking and presentation for visual exploration
LU102575B1 (en) Providing recent event information for web search queries
US11714867B2 (en) Discovering alternate online service providers
JP7249222B2 (ja) 情報処理装置、情報処理方法および情報処理プログラム
JP7080198B2 (ja) 推定装置、推定方法および推定プログラム

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21926942

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21926942

Country of ref document: EP

Kind code of ref document: A1