CN117406887A - Intelligent mirror cabinet control method and system based on human body induction - Google Patents
Intelligent mirror cabinet control method and system based on human body induction Download PDFInfo
- Publication number
- CN117406887A CN117406887A CN202311561732.3A CN202311561732A CN117406887A CN 117406887 A CN117406887 A CN 117406887A CN 202311561732 A CN202311561732 A CN 202311561732A CN 117406887 A CN117406887 A CN 117406887A
- Authority
- CN
- China
- Prior art keywords
- user
- application
- display area
- image
- intelligent mirror
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 29
- 230000006698 induction Effects 0.000 title claims abstract description 14
- 238000003062 neural network model Methods 0.000 claims abstract description 28
- 238000013459 approach Methods 0.000 claims abstract description 18
- 238000013527 convolutional neural network Methods 0.000 claims description 14
- 230000004438 eyesight Effects 0.000 claims description 7
- 239000013598 vector Substances 0.000 description 16
- 230000006399 behavior Effects 0.000 description 11
- 238000013528 artificial neural network Methods 0.000 description 7
- 230000011218 segmentation Effects 0.000 description 6
- 230000006870 function Effects 0.000 description 5
- 238000000547 structure data Methods 0.000 description 5
- 238000010586 diagram Methods 0.000 description 4
- 238000004364 calculation method Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000013136 deep learning model Methods 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 238000011478 gradient descent method Methods 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 238000011176 pooling Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0481—Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
- G06F3/04817—Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance using icons
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0484—Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
- G06F3/04845—Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range for image manipulation, e.g. dragging, rotation, expansion or change of colour
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/042—Knowledge-based neural networks; Logical representations of neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/0464—Convolutional networks [CNN, ConvNet]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T11/00—2D [Two Dimensional] image generation
- G06T11/001—Texturing; Colouring; Generation of texture or colour
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02B—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO BUILDINGS, e.g. HOUSING, HOUSE APPLIANCES OR RELATED END-USER APPLICATIONS
- Y02B20/00—Energy efficient lighting technologies, e.g. halogen lamps or gas discharge lamps
- Y02B20/40—Control techniques providing energy savings, e.g. smart controller or presence detection
Abstract
The invention provides an intelligent mirror cabinet control method and system based on human body induction, and the intelligent mirror cabinet control method and system based on human body induction, wherein the method comprises the steps of determining an image of a user in an intelligent mirror by using an image generation model based on the image when the user approaches the intelligent mirror; determining the name of a user, the brightness of ambient light and the display range of the human body of the user in the intelligent mirror based on the image of the user in the intelligent mirror; determining a left display area of the application, a right display area of the application and an upper display area of the application by using a display area determining model based on user physiological information, ambient light brightness and a display range of a user human body in the smart mirror corresponding to the user name; a display position of the plurality of target applications and each of the plurality of target applications in the smart mirror is determined based on the graph neural network model. The method can improve the comfortableness of displaying the application icons in the using process of the user and improve the user experience.
Description
Technical Field
The invention relates to the technical field of intelligent mirrors, in particular to an intelligent mirror cabinet control method and system based on human body induction.
Background
The smart mirror is a mirror integrating various intelligent functions, such as displaying weather information, time information, news information, applications, etc. These functions make smart mirrors an integral part of modern home and office environments.
However, when a user looks at the mirror, the smart mirror needs to display the application icon at the same time, and the image of the user in the mirror is often blocked by the displayed application icon, so that the image display of the person in the mirror is unclear or blocked. Existing solutions typically display application icons at the edges or corners of the mirror, but since the application icons are displayed at the corners, the corner locations are typically not the area of primary interest to the user, and are difficult for the user to find, and the corner locations of the mirror are not easily accessible, which may result in difficult user operation and poor user experience.
Therefore, how to improve the comfort of displaying application icons in the use process of a user and improve the user experience are the problems to be solved urgently.
Disclosure of Invention
The invention mainly solves the technical problem of how to improve the comfortableness of displaying the application icons of the user in the using process and improve the user experience.
According to a first aspect, the invention provides an intelligent mirror cabinet control method based on human body induction, comprising the following steps: acquiring an image when a user approaches the intelligent mirror; determining an image of the user in the smart mirror using an image generation model based on the image of the user when the user is near the smart mirror; determining the name of the user, the brightness of ambient light and the display range of the human body of the user in the intelligent mirror based on the image of the user in the intelligent mirror; acquiring user physiological information corresponding to a user name; determining a left display area of an application, a right display area of the application and an upper display area of the application by using a display area determining model based on user physiological information corresponding to the user name, ambient light brightness and a display range of the user human body in an intelligent mirror; determining the display size of an application icon based on the left display area of the application, the right display area of the application, the upper display area of the application, the user physiological information corresponding to the user name and the brightness of ambient light; determining an application display number based on the application icon display size, the left display area of the application, the right display area of the application, and the upper display area of the application; acquiring a plurality of pieces of application information in an intelligent mirror, wherein each piece of application information in the plurality of pieces of application information in the intelligent mirror comprises historical behavior data of a user in an application, category information, use frequency, user scores, update time and application size; constructing a plurality of nodes and a plurality of edges between the nodes based on a plurality of application information in the intelligent mirror, wherein each node in the plurality of nodes represents an application, each node comprises a plurality of node characteristics, the node characteristics of each node comprise the display quantity of the application, the left display area of the application, the right display area of the application, the upper display area of the application, category information, the use frequency, the user score, the update time and the application size, and the edge characteristics of each edge in the plurality of edges comprise the co-occurrence times of two applications in user operation and the similarity between the two applications; processing a plurality of nodes and a plurality of edges between the plurality of nodes based on the graph neural network model to determine a plurality of target applications and display positions of each target application in the plurality of target applications in the smart mirror.
Further, the physiological information of the user corresponding to the user name comprises arm length, eyesight, age and gender.
Still further, the image generation model is a convolutional neural network model, the input of the image generation model is an image when the user approaches the smart mirror, and the output of the image generation model is an image of the user in the smart mirror.
Still further, the method further comprises: and if the brightness of the ambient light is smaller than the threshold value, reminding a user to turn on a lamp.
Still further, the input of the graph neural network model is a plurality of nodes and a plurality of edges between the plurality of nodes, and the output of the graph neural network model is a display position of the plurality of target applications and each of the plurality of target applications in the smart mirror.
According to a second aspect, the present invention provides an intelligent mirror cabinet control system based on human body sensing, comprising: the first acquisition module is used for acquiring an image when a user approaches the intelligent mirror;
an image generation module for determining an image of the user in the smart mirror using an image generation model based on the image of the user when the user is near the smart mirror;
the display range determining module is used for determining the name of the user, the brightness of the ambient light and the display range of the human body of the user in the intelligent mirror based on the image of the user in the intelligent mirror;
the second acquisition module is used for acquiring the physiological information of the user corresponding to the user name;
the display area determining module is used for determining a left display area of an application, a right display area of the application and an upper display area of the application by using a display area determining model based on user physiological information corresponding to the user name, ambient light brightness and a display range of the user human body in the smart mirror;
the icon determining module is used for determining the display size of the application icon based on the left display area of the application, the right display area of the application, the upper display area of the application, the user physiological information corresponding to the user name and the ambient light brightness;
a display quantity determining module, configured to determine an application display quantity based on the application icon display size, the left display area of the application, the right display area of the application, and the upper display area of the application;
the third acquisition module is used for acquiring a plurality of pieces of application information in the intelligent mirror, wherein each piece of application information in the plurality of pieces of application information in the intelligent mirror comprises historical behavior data, category information, use frequency, user scores, update time and application size of a user in an application;
a graph structure module, configured to construct a plurality of nodes and a plurality of edges between the plurality of nodes based on a plurality of application information in the smart mirror, where each node in the plurality of nodes represents an application, each node includes a plurality of node features, and the node features of each node include an application display number, a left display area of the application, a right display area of the application, an upper display area of the application, category information, a frequency of use, a user score, an update time, and an application size, and the edge feature of each edge in the plurality of edges includes a number of co-occurrences of two applications in a user operation, and a similarity between the two applications;
and the graph neural network model processing module is used for processing the plurality of nodes and the plurality of edges between the plurality of nodes based on the graph neural network model to determine a plurality of target applications and display positions of each target application in the plurality of target applications in the intelligent mirror.
Further, the physiological information of the user corresponding to the user name comprises arm length, eyesight, age and gender.
Still further, the image generation model is a convolutional neural network model, the input of the image generation model is an image when the user approaches the smart mirror, and the output of the image generation model is an image of the user in the smart mirror.
Still further, the system is further configured to: and if the brightness of the ambient light is smaller than the threshold value, reminding a user to turn on a lamp.
Still further, the input of the graph neural network model is a plurality of nodes and a plurality of edges between the plurality of nodes, and the output of the graph neural network model is a display position of the plurality of target applications and each of the plurality of target applications in the smart mirror.
The invention provides an intelligent mirror cabinet control method and system based on human body induction, wherein the method comprises the steps of obtaining an image when a user approaches an intelligent mirror; determining an image of the user in the smart mirror using an image generation model based on the image of the user when the user is near the smart mirror; determining the name of the user, the brightness of ambient light and the display range of the human body of the user in the intelligent mirror based on the image of the user in the intelligent mirror; acquiring user physiological information corresponding to a user name; determining a left display area of an application, a right display area of the application and an upper display area of the application by using a display area determining model based on user physiological information corresponding to the user name, ambient light brightness and a display range of the user human body in an intelligent mirror; determining the display size of an application icon based on the left display area of the application, the right display area of the application, the upper display area of the application, the user physiological information corresponding to the user name and the brightness of ambient light; determining an application display number based on the application icon display size, the left display area of the application, the right display area of the application, and the upper display area of the application; acquiring a plurality of pieces of application information in an intelligent mirror, wherein each piece of application information in the plurality of pieces of application information in the intelligent mirror comprises historical behavior data of a user in an application, category information, use frequency, user scores, update time and application size; constructing a plurality of nodes and a plurality of edges between the nodes based on a plurality of application information in the intelligent mirror, wherein each node in the plurality of nodes represents an application, each node comprises a plurality of node characteristics, the node characteristics of each node comprise the display quantity of the application, the left display area of the application, the right display area of the application, the upper display area of the application, category information, the use frequency, the user score, the update time and the application size, and the edge characteristics of each edge in the plurality of edges comprise the co-occurrence times of two applications in user operation and the similarity between the two applications; processing a plurality of nodes and a plurality of edges between the plurality of nodes based on the graph neural network model to determine a plurality of target applications and display positions of each target application in the plurality of target applications in the smart mirror. The method can improve the comfortableness of displaying the application icons in the using process of the user and improve the user experience.
Drawings
Fig. 1 is a schematic flow chart of a method for controlling an intelligent mirror cabinet based on human body induction according to an embodiment of the present invention;
fig. 2 is a schematic diagram of a display area of an application icon according to an embodiment of the present invention;
fig. 3 is a schematic diagram of an intelligent mirror cabinet control system based on human body induction according to an embodiment of the present invention.
Detailed Description
In the embodiment of the invention, an intelligent mirror cabinet control method based on human body induction is provided as shown in fig. 1, and the intelligent mirror cabinet control method based on human body induction comprises the following steps of S1 to S10:
step S1, an image when a user approaches the intelligent mirror is acquired.
When a user approaches the intelligent mirror, a camera installed in the intelligent mirror can automatically trigger a photographing function to acquire a user image. For example, a user walking near the smart mirror and standing in front of it, the camera may take a picture of the user as an image of the user approaching the smart mirror. As an example, the smart mirror may be equipped with an infrared sensor that detects the approach of the user when the user approaches the smart mirror, thereby triggering an automatic trigger photographing function.
And S2, determining an image of the user in the smart mirror by using an image generation model based on the image of the user when the user approaches the smart mirror.
The image of the user in the intelligent mirror is an image of the user in the mirror, which is generated by the image generation model according to the shot image when the user approaches the intelligent mirror, and the image is generated by simulating the view angle of the user. I.e. the image of the user when approaching the smart mirror is a truly existing image, while the image of the user in the smart mirror is an image of the user in the smart mirror that simulates what the user sees from the perspective of the user. The image of the user in the smart mirror is a reflected image that the user observes when using the smart mirror.
The image generation model is a convolutional neural network model. The Convolutional Neural Network (CNN) may be a multi-layer neural network (e.g., comprising at least two layers). The at least two layers may include at least one of a convolutional layer (CONV), a modified linear unit (ReLU) layer, a pooling layer (POOL), or a fully-connected layer (FC). Convolutional neural networks are capable of extracting useful features from an image and progressively understanding and learning the contextual information of the image. The image generation model is a convolutional neural network model, the input of the image generation model is an image when the user approaches the intelligent mirror, and the output of the image generation model is an image of the user in the intelligent mirror. The convolutional neural network model learns statistical rules and characteristic representation of the image through a training process, and rebuilds the image. Through a large number of training samples and optimization algorithms, the model can learn more accurate and real image generation capability so as to generate images of users in the intelligent mirrors.
And step S3, determining the name of the user, the brightness of the ambient light and the display range of the human body of the user in the intelligent mirror based on the image of the user in the intelligent mirror.
In some embodiments, the user name, ambient light intensity, and the range of display of the user's human body in the smart mirror may be determined based on the user identification model. The input of the user identification model is the image of the user in the intelligent mirror, and the output of the user identification model is the name of the user, the brightness of the ambient light and the display range of the human body of the user in the intelligent mirror. The user identification model is a convolutional neural network model. The user identification model is a convolutional neural network model. The image of the user in the intelligent mirror comprises ambient light brightness information and display range information of the human body of the user in the intelligent mirror, and the image of the user in the intelligent mirror can be identified through a user identification model, so that the display range of the human body of the user in the intelligent mirror is obtained.
In some embodiments, the user identification model includes a background segmentation layer, a brightness determination layer, a user information determination. The background segmentation layer, the brightness determination layer and the user information determination are all convolutional neural networks. The input of the background segmentation layer is the image of the user in the intelligent mirror, the output of the background segmentation layer is the segmented background image and the segmented user human body image, the input of the brightness determination layer is the segmented background image, the output of the brightness determination layer is the ambient light brightness, the input of the user information determination layer is the segmented user human body image, and the output of the user information determination layer is the user name and the display range of the user human body in the intelligent mirror.
The display range of the user's human body in the smart mirror means that the user can see the region of the user's human body in the smart mirror. The display range of the user body in the intelligent mirror is determined, so that the display area of the application can be accurately distributed beside the display range of the user body in the intelligent mirror when the display area of the application is determined later.
In some embodiments, the method further comprises: and if the brightness of the ambient light is smaller than the threshold value, reminding a user to turn on a lamp.
And S4, acquiring user physiological information corresponding to the user name.
The user physiological information corresponding to the user name comprises arm length, eyesight, age and gender.
The arm length may help determine the display range on the smart mirror, as an example, the longer the arm length of the user, the wider the display range may be, enabling the user to touch more applications while operating.
The vision condition of the user has an important influence on the display range of the smart mirror, and as an example, the worse the vision of the user is, the smaller the display range of the smart mirror is, the smaller the number of displayed applications is, and the larger the display size of the application icons can be.
Age and gender can also affect the scope and content of the smart mirror to some extent. For example, for young users, a larger display range may be employed to display more content; while for elderly users a smaller display range and less content may be employed.
And S5, determining a left display area of an application, a right display area of the application and an upper display area of the application by using a display area determination model based on the user physiological information corresponding to the user name, the ambient light brightness and the display range of the user human body in the smart mirror.
The left display area of the application refers to a display area on the screen of the smart mirror, which is positioned at the left side of the display range of the human body of the user in the smart mirror, and is used for displaying the application icons.
The right display area of the application refers to a display area which is positioned on the screen of the intelligent mirror and is positioned on the right side of the display range of the human body of the user in the intelligent mirror, and the display area is used for displaying application icons.
The upper display area of the application refers to a display area which is positioned above the display range of the human body of the user in the intelligent mirror on the screen of the intelligent mirror and is used for displaying the application icons.
The display area determination model is a convolutional neural network model. The input of the display area determining model is the user physiological information, the ambient light brightness and the display range of the user human body in the intelligent mirror corresponding to the user name, and the output of the display area determining model is the left display area of the application, the right display area of the application and the upper display area of the application.
The brightness of the ambient light may also affect the display area of the application, e.g., if the brightness of the ambient light is greater, the display area may be larger to display more content, and, for example, if the brightness of the ambient light is lower, the display area may be smaller and the displayed content may be less, so that the user can accurately identify the content in the display area.
As an example, fig. 2 is a schematic diagram of a display area of an application icon according to an embodiment of the present invention. As shown in fig. 2, fig. 2 shows an intelligent mirror, and through fig. 2, a left display area of an application, a right display area of the application and an upper display area of the application are respectively displayed beside a user, so that the display effect is good, and the shielding of application icons to the user is avoided.
The display area determining model can comprehensively consider the user physiological information corresponding to the user name, the ambient light brightness and the display range of the user human body in the intelligent mirror, effectively process image data and finally help to determine the display area of the application through learning and extracting features. In some embodiments, the display area determination model may be trained by a gradient descent method to obtain the display area determination model.
And S6, determining the display size of the application icon based on the left display area of the application, the right display area of the application, the upper display area of the application, the user physiological information corresponding to the user name and the ambient light brightness.
In some embodiments, a left display area of an application, a right display area of the application, an upper display area of the application, user physiological information corresponding to a user name, and ambient light brightness may be constructed as a vector to be matched, and an application icon display size corresponding to a reference vector with a distance smaller than a threshold value may be determined as an application icon display size by calculating a distance between the vector to be matched and each reference vector in the database. The database is pre-constructed, the database comprises reference vectors and application icon display sizes corresponding to the reference vectors, the reference vectors are obtained by constructing the reference vectors based on a left display area of an application, a right display area of the application, an upper display area of the application, user physiological information corresponding to a user name and ambient light brightness in historical data, and the application icon display sizes corresponding to the reference vectors are determined application icon display sizes in the historical data.
In some embodiments, the display size of the application icon may be determined using a deep neural network model, where the input of the deep neural network model is a left display area of the application, a right display area of the application, an upper display area of the application, user physiological information corresponding to the user name, and ambient light brightness, and the output of the deep neural network model is the display size of the application icon.
Step S6 enables the display size of the application icon to be adjusted according to the physiological characteristics and environmental conditions of the user, thereby providing a more personalized and comfortable user experience.
And S7, determining the display quantity of the application based on the display size of the application icon, the left display area of the application, the right display area of the application and the upper display area of the application.
In some embodiments, the display area of the left display area of the application, the display area of the right display area of the application, and the display area of the upper display area of the application may be added and divided by the display size of the application icon to obtain the number of application displays. The application display number is a sum of the number of applications displayed in the display area of the application.
In some embodiments, the display areas of the left display area of the application, the right display area of the application, and the upper display area of the application may be added to obtain a display total area. The number of applications displayed may be determined by querying a preset table of the number of applications displayed. The application display quantity preset table comprises each display total area and application display quantity corresponding to each display total area, and the application display quantity preset table can be artificially constructed based on historical data.
Step S8, acquiring a plurality of pieces of application information in the intelligent mirror, wherein each piece of application information in the plurality of pieces of application information in the intelligent mirror comprises historical behavior data, category information, use frequency, user scores, update time and application size of a user in an application.
Historical behavior data of a user in an application refers to operational records and behavior data of the user in a particular application, such as browsing history, clicking behavior, purchasing records, and the like.
The category information represents a category or classification to which the application belongs, such as social, entertainment, tools, etc.
Usage frequency refers to how frequently a user uses a particular application, typically measured in times or periods of time.
The user score reflects the user's satisfaction and rating of the application, typically presented in a star rating or fractional form.
The update time refers to the time stamp or date the application was last updated.
The application size represents the amount of storage space occupied by an application program, typically in units of MB or GB.
By collecting this information, the smart mirror can better understand the preferences and behavior habits of the user, while also being able to provide more personalized and efficient application recommendation and management services for the user.
In some embodiments, relevant information for each application may be extracted from an application management system or database of the smart mirror.
Step S9, constructing a plurality of nodes and a plurality of edges between the plurality of nodes based on the plurality of application information in the smart mirror, wherein each node in the plurality of nodes represents an application, each node comprises a plurality of node features, each node feature of each node comprises an application display number, a left display area of the application, a right display area of the application, an upper display area of the application, category information, a use frequency, a user score, an update time and an application size, and the edge feature of each edge in the plurality of edges comprises a co-occurrence number of two applications in a user operation and a similarity between the two applications.
The number of co-occurrences of two applications in a user operation indicates the number of times the two applications are simultaneously opened and operated during the use of the smart mirror by the user. As an example, assuming that the user performs 10 operations in total in the past week, in which both the music player is turned on and the weather forecast application is switched to among 3 operations, the number of co-occurrences of the two applications is 3.
In some embodiments, the number of co-occurrences of two applications in a user operation may be determined by: user operation data is collected, including applications and number of uses used by the user. For each user, the number of simultaneous occurrences of both applications is counted. If two applications are present simultaneously in the same user operation, the count is incremented by one. And finally obtaining the co-occurrence times of the two applications in the user operation.
The degree of similarity between two applications indicates the degree of similarity between the two applications. In some embodiments, simHash values of two application information may be calculated separately, and then similarity of SimHash values of the two application information may be calculated by a hamming distance. The smaller the hamming distance, the higher the similarity. The application information includes historical behavior data of the user in the application, category information, frequency of use, user score, update time, and application size.
The step of calculating the applied SimHash value may include the steps of word segmentation, hash calculation, weighting, merging, dimension reduction, and the like. For example, word segmentation: firstly, the application information is segmented, and feature vectors are extracted. And setting weight (weight) for the feature vector; hash calculation: calculating a hash value of each feature vector through a hash function, wherein the hash value is an n-bit signature consisting of binary numbers 01; weighting: on the basis of the Hash value, weighting all feature vectors, namely W=hash weight, wherein when the weight is 1, the Hash value and the weight are multiplied positively, and when the weight is 0, the Hash value and the weight are multiplied negatively; combining: accumulating the weighted results of the feature vectors to form a serial string; dimension reduction: and setting 1 if the accumulated result is larger than 0, otherwise setting 0, and thus obtaining the simhash value of the statement.
The degree of association and the degree of similarity between two applications can be quantified by calculating the number of co-occurrences of the two applications and the degree of similarity of the application feature vectors. These metrics can be used as inputs to the neural network model to recommend applications that fit the user, providing better experience and decision support for the user.
The aim of performing this step is to achieve a deep understanding of the relationship between user behavior and applications by building a correlation network between applications, thereby providing a more accurate application recommendation for smart mirrors.
And step S10, processing a plurality of nodes and a plurality of edges between the nodes based on the graph neural network model to determine a plurality of target applications and display positions of each target application in the plurality of target applications in the intelligent mirror.
In the graph structure data, nodes represent entities or concepts, each node containing a set of features describing attributes or states of the entity. Nodes refer herein to an application in a smart mirror, each node including various characteristic information of the application. Edges in the graph structure data, edges represent relationships between nodes.
The graphic neural network model comprises a graphic neural network (Graph Neural Network, GNN) and a full-connection layer, wherein the graphic neural network is a neural network directly acting on graphic structure data, and the graphic structure data is a data structure consisting of nodes and edges. The graph neural network is a deep learning model designed for graph structure data, and can effectively capture the characteristics of nodes and edges in the graph and the relation between the nodes and edges. The plurality of application nodes and the plurality of edges in the smart mirror may form a graph structure upon which the graph neural network is applied for processing.
The input of the graph neural network model is a plurality of nodes and a plurality of edges between the nodes, and the output of the graph neural network model is the display positions of the plurality of target applications and each target application in the plurality of target applications in the smart mirror.
Each node represents an application and contains a plurality of node features such as the number of applications displayed, left display area, right display area, upper display area, category information, frequency of use, user score, update time, and application size, etc. These node features provide detailed information about the application that can help the model understand the properties and features of the application. Each edge represents a relationship between two applications, and the edge feature includes the number of co-occurrences of the two applications in user operation and the similarity between the two applications. These edge features provide interaction and similarity information between applications, helping the model understand the relevance and manner of connection between applications. Through the learning and reasoning process of the graph neural network model, a plurality of target applications and display positions of the target applications in the intelligent mirror can be determined according to the node embedded vector and the weight information of the edge. The graph neural network model can sort, screen and position a plurality of applications by considering the relation, the similarity and other context information among the nodes, so as to determine a target application, and finally achieve the aim of optimizing the display effect.
Based on the same inventive concept, fig. 3 is a schematic diagram of an intelligent mirror cabinet control system based on human body induction according to an embodiment of the present invention, where the intelligent mirror cabinet control system based on human body induction includes:
a first acquiring module 31, configured to acquire an image when a user approaches the smart mirror;
an image generation module 32 for determining an image of the user in the smart mirror using an image generation model based on the image of the user when the user is near the smart mirror;
a display range determining module 33, configured to determine a user name, ambient light brightness, and a display range of a user human body in the smart mirror based on the image of the user in the smart mirror;
a second obtaining module 34, configured to obtain physiological information of the user corresponding to the user name;
a display area determining module 35, configured to determine a left display area of an application, a right display area of the application, and an upper display area of the application using a display area determining model based on user physiological information corresponding to the user name, ambient light brightness, and a display range of the user's human body in a smart mirror;
an icon determining module 36, configured to determine an application icon display size based on the left display area of the application, the right display area of the application, the upper display area of the application, the user physiological information corresponding to the user name, and the ambient light brightness;
a display number determining module 37 for determining an application display number based on the application icon display size, the left display area of the application, the right display area of the application, and the upper display area of the application;
a third obtaining module 38, configured to obtain a plurality of application information in the smart mirror, where each of the plurality of application information in the smart mirror includes historical behavior data of a user in an application, category information, frequency of use, user score, update time, and application size;
a graph structure module 39, configured to construct a plurality of nodes and a plurality of edges between the plurality of nodes based on a plurality of application information in the smart mirror, where each node in the plurality of nodes represents an application, each node includes a plurality of node features, and the node features of each node include an application display number, a left display area of the application, a right display area of the application, an upper display area of the application, category information, a frequency of use, a user score, an update time, and an application size, and the edge feature of each edge in the plurality of edges includes a number of co-occurrences of two applications in a user operation, and a similarity between the two applications;
the graph neural network model processing module 40 is configured to determine a plurality of target applications and display positions of each target application in the plurality of target applications in the smart mirror based on the graph neural network model processing the plurality of nodes and the plurality of edges between the plurality of nodes.
Claims (10)
1. The intelligent mirror cabinet control method based on human body induction is characterized by comprising the following steps of:
acquiring an image when a user approaches the intelligent mirror;
determining an image of the user in the smart mirror using an image generation model based on the image of the user when the user is near the smart mirror;
determining the name of the user, the brightness of ambient light and the display range of the human body of the user in the intelligent mirror based on the image of the user in the intelligent mirror;
acquiring user physiological information corresponding to a user name;
determining a left display area of an application, a right display area of the application and an upper display area of the application by using a display area determining model based on user physiological information corresponding to the user name, ambient light brightness and a display range of the user human body in an intelligent mirror;
determining the display size of an application icon based on the left display area of the application, the right display area of the application, the upper display area of the application, the user physiological information corresponding to the user name and the brightness of ambient light;
determining an application display number based on the application icon display size, the left display area of the application, the right display area of the application, and the upper display area of the application;
acquiring a plurality of pieces of application information in an intelligent mirror, wherein each piece of application information in the plurality of pieces of application information in the intelligent mirror comprises historical behavior data of a user in an application, category information, use frequency, user scores, update time and application size;
constructing a plurality of nodes and a plurality of edges between the nodes based on a plurality of application information in the intelligent mirror, wherein each node in the plurality of nodes represents an application, each node comprises a plurality of node characteristics, the node characteristics of each node comprise the display quantity of the application, the left display area of the application, the right display area of the application, the upper display area of the application, category information, the use frequency, the user score, the update time and the application size, and the edge characteristics of each edge in the plurality of edges comprise the co-occurrence times of two applications in user operation and the similarity between the two applications;
processing a plurality of nodes and a plurality of edges between the plurality of nodes based on the graph neural network model to determine a plurality of target applications and display positions of each target application in the plurality of target applications in the smart mirror.
2. The intelligent mirror cabinet control method based on human body sensing according to claim 1, wherein the user physiological information corresponding to the user name comprises arm length, eyesight, age and gender.
3. The intelligent mirror cabinet control method based on human body sensing according to claim 1, wherein the image generation model is a convolutional neural network model, the input of the image generation model is an image when the user approaches the intelligent mirror, and the output of the image generation model is an image of the user in the intelligent mirror.
4. The intelligent mirror cabinet control method based on human body sensing according to claim 1, wherein the method further comprises: and if the brightness of the ambient light is smaller than the threshold value, reminding a user to turn on a lamp.
5. The human body sensing-based intelligent mirror cabinet control method of claim 1, wherein the input of the graph neural network model is a plurality of nodes and a plurality of edges between the plurality of nodes, and the output of the graph neural network model is a display position of the plurality of target applications and each of the plurality of target applications in the intelligent mirror.
6. Intelligent mirror cabinet control system based on human response, characterized by comprising:
the first acquisition module is used for acquiring an image when a user approaches the intelligent mirror;
an image generation module for determining an image of the user in the smart mirror using an image generation model based on the image of the user when the user is near the smart mirror;
the display range determining module is used for determining the name of the user, the brightness of the ambient light and the display range of the human body of the user in the intelligent mirror based on the image of the user in the intelligent mirror;
the second acquisition module is used for acquiring the physiological information of the user corresponding to the user name;
the display area determining module is used for determining a left display area of an application, a right display area of the application and an upper display area of the application by using a display area determining model based on user physiological information corresponding to the user name, ambient light brightness and a display range of the user human body in the smart mirror;
the icon determining module is used for determining the display size of the application icon based on the left display area of the application, the right display area of the application, the upper display area of the application, the user physiological information corresponding to the user name and the ambient light brightness;
a display quantity determining module, configured to determine an application display quantity based on the application icon display size, the left display area of the application, the right display area of the application, and the upper display area of the application;
the third acquisition module is used for acquiring a plurality of pieces of application information in the intelligent mirror, wherein each piece of application information in the plurality of pieces of application information in the intelligent mirror comprises historical behavior data, category information, use frequency, user scores, update time and application size of a user in an application;
a graph structure module, configured to construct a plurality of nodes and a plurality of edges between the plurality of nodes based on a plurality of application information in the smart mirror, where each node in the plurality of nodes represents an application, each node includes a plurality of node features, and the node features of each node include an application display number, a left display area of the application, a right display area of the application, an upper display area of the application, category information, a frequency of use, a user score, an update time, and an application size, and the edge feature of each edge in the plurality of edges includes a number of co-occurrences of two applications in a user operation, and a similarity between the two applications;
and the graph neural network model processing module is used for processing the plurality of nodes and the plurality of edges between the plurality of nodes based on the graph neural network model to determine a plurality of target applications and display positions of each target application in the plurality of target applications in the intelligent mirror.
7. The intelligent mirror cabinet control system based on human body sensing as claimed in claim 6, wherein the user physiological information corresponding to the user name includes arm length, eyesight, age, sex.
8. The smart mirror cabinet control system based on human body sensing as claimed in claim 6, wherein the image generation model is a convolutional neural network model, an input of the image generation model is an image of the user when the user approaches the smart mirror, and an output of the image generation model is an image of the user in the smart mirror.
9. The smart mirror cabinet control system based on human sensing of claim 6, further comprising: and if the brightness of the ambient light is smaller than the threshold value, reminding a user to turn on a lamp.
10. The smart mirror cabinet control system based on human sensing of claim 6, wherein the input of the graph neural network model is the plurality of nodes and a plurality of edges between the plurality of nodes, and the output of the graph neural network model is the plurality of target applications and a display position of each of the plurality of target applications in the smart mirror.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311561732.3A CN117406887B (en) | 2023-11-21 | 2023-11-21 | Intelligent mirror cabinet control method and system based on human body induction |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311561732.3A CN117406887B (en) | 2023-11-21 | 2023-11-21 | Intelligent mirror cabinet control method and system based on human body induction |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN117406887A true CN117406887A (en) | 2024-01-16 |
| CN117406887B CN117406887B (en) | 2024-04-09 |
Family
ID=89494501
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311561732.3A Active CN117406887B (en) | 2023-11-21 | 2023-11-21 | Intelligent mirror cabinet control method and system based on human body induction |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117406887B (en) |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105573627A (en) * | 2015-04-28 | 2016-05-11 | 宇龙计算机通信科技(深圳)有限公司 | Method and device for prompting user to protect eyes through intelligent glasses |
| CN110109553A (en) * | 2019-05-24 | 2019-08-09 | 京东方科技集团股份有限公司 | A kind of Intelligent mirror and display methods |
| CN111401330A (en) * | 2020-04-26 | 2020-07-10 | 四川自由健信息科技有限公司 | Teaching system and intelligent mirror adopting same |
| CN113628239A (en) * | 2021-08-16 | 2021-11-09 | 百度在线网络技术（北京）有限公司 | Display optimization method, related device and computer program product |
| CN114431684A (en) * | 2022-01-11 | 2022-05-06 | 茹柚智能科技(上海)有限公司 | Intelligent mirror |
| CN116048363A (en) * | 2023-04-03 | 2023-05-02 | 成都苏扶软件开发有限公司 | Display method, system, equipment and medium of software interface based on artificial intelligence |
| CN116909408A (en) * | 2023-09-13 | 2023-10-20 | 中物联讯(北京)科技有限公司 | Content interaction method based on MR intelligent glasses |
| CN117031971A (en) * | 2023-07-18 | 2023-11-10 | 东莞莱姆森科技建材有限公司 | Intelligent furniture equipment adjusting method, device, equipment and medium based on intelligent mirror |
-
2023
- 2023-11-21 CN CN202311561732.3A patent/CN117406887B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105573627A (en) * | 2015-04-28 | 2016-05-11 | 宇龙计算机通信科技(深圳)有限公司 | Method and device for prompting user to protect eyes through intelligent glasses |
| CN110109553A (en) * | 2019-05-24 | 2019-08-09 | 京东方科技集团股份有限公司 | A kind of Intelligent mirror and display methods |
| CN111401330A (en) * | 2020-04-26 | 2020-07-10 | 四川自由健信息科技有限公司 | Teaching system and intelligent mirror adopting same |
| CN113628239A (en) * | 2021-08-16 | 2021-11-09 | 百度在线网络技术（北京）有限公司 | Display optimization method, related device and computer program product |
| CN114431684A (en) * | 2022-01-11 | 2022-05-06 | 茹柚智能科技(上海)有限公司 | Intelligent mirror |
| CN116048363A (en) * | 2023-04-03 | 2023-05-02 | 成都苏扶软件开发有限公司 | Display method, system, equipment and medium of software interface based on artificial intelligence |
| CN117031971A (en) * | 2023-07-18 | 2023-11-10 | 东莞莱姆森科技建材有限公司 | Intelligent furniture equipment adjusting method, device, equipment and medium based on intelligent mirror |
| CN116909408A (en) * | 2023-09-13 | 2023-10-20 | 中物联讯(北京)科技有限公司 | Content interaction method based on MR intelligent glasses |
Also Published As
| Publication number | Publication date |
|---|---|
| CN117406887B (en) | 2024-04-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11567989B2 (en) | Media unit retrieval and related processes | |
| US11334628B2 (en) | Dressing recommendation method and dressing recommendation apparatus | |
| CN108898579A (en) | A kind of image definition recognition methods, device and storage medium | |
| CN110532479A (en) | A kind of information recommendation method, device and equipment | |
| CN111095293A (en) | Image aesthetic processing method and electronic equipment | |
| CN112632403B (en) | Training method, recommendation method, device, equipment and medium for recommendation model | |
| CN110555481A (en) | Portrait style identification method and device and computer readable storage medium | |
| WO2021042841A1 (en) | Method and device for intelligently adjusting push screen, computer apparatus, and storage medium | |
| KR102649211B1 (en) | Device for visual equipment selection based on machine learning | |
| KR20190140597A (en) | Social network-based smart wardrobe and costume recommendation method | |
| CN116048363B (en) | Display method, system, equipment and medium of software interface based on artificial intelligence | |
| CN112084825A (en) | Cooking evaluation method, cooking recommendation method, computer device and storage medium | |
| Luo et al. | A computational approach to determination of main subject regions in photographic images | |
| CN117406887B (en) | Intelligent mirror cabinet control method and system based on human body induction | |
| CN114330482A (en) | Data processing method and device and computer readable storage medium | |
| CN109407941A (en) | A kind of display processing system and display methods | |
| JP7188566B2 (en) | Information processing device, information processing method and program | |
| CN112489815A (en) | Depression emotion monitoring method and device and readable storage medium | |
| CN116629925A (en) | Digital exhibition method and system | |
| CN113591550B (en) | Method, device, equipment and medium for constructing personal preference automatic detection model | |
| CN115905688A (en) | Reference information generation method based on artificial intelligence and smart home | |
| CN115408611A (en) | Menu recommendation method and device, computer equipment and storage medium | |
| CN113095157A (en) | Image shooting method and device based on artificial intelligence and related products | |
| Gottimukkala et al. | Semi-automatic Annotation of Images Using Eye Gaze Data (SAIGA) | |
| CN112313563B (en) | Method for determining an ophthalmic part |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant |