KR20210001798A - smart mirror chatbot system of high-level context awareness by use of adaptive multiple biometrics - Google Patents

Abstract

The present invention relates to a technology for generally performing high-level context awareness based on image and sound information for a person who holds a smart mirror up to himself or herself, and providing a chatbot service on the smart mirror based thereon. In particular, the present invention relates to a technology for providing a high-level context awareness based chatbot service with the same level on a plurality of smart mirrors by performing high-level context awareness by adaptive multi-biometric information processing based on the weighting factor of image and sound information for a person who holds a smart mirror up to himself or herself, providing a chatbot service on the smart mirror based thereon, adjusting biometric information weighting factor information by reflecting satisfaction evaluation, and inputting biometric information weighting factor change information in a blockchain to perform distributed management. The present invention can gradually improve high-level context awareness performance by reflecting the satisfaction evaluation for the chatbot service to adjust a weighting factor table. At this time, the present invention is smoothly operated even if a network environment is insufficient by allowing edge artificial intelligence to apply the weighting factor table in a local environment, and distributing and managing update information of the weighting factor table with the blockchain.

Description

{Smart mirror chatbot system of high-level context awareness by use of adaptive multiple biometrics}

The present invention generally relates to a technology for performing high-level context recognition based on image and sound information for a person who is projecting itself on a smart mirror and providing a chatbot service on the smart mirror based on this.

In particular, the present invention performs high-level context recognition by processing adaptive multi-biometric information based on the weight of image and sound information for a person who is reflecting itself on a smart mirror, and based on this, provides a chatbot service on the smart mirror and evaluates satisfaction. It relates to a technology that can provide a chatbot service based on recognition of the same level of high-level situations in multiple smart mirrors by adjusting biometric weight information by reflecting and managing biometric weight change information in a blockchain.

Recently, artificial intelligence assistants have been implemented, and services that interact with users by text or voice are being actively tried around the world. Representative examples of these are artificial intelligence (AI) speakers and smart home services.

For example, starting with Apple Siri, there have been attempts to use a smartphone as a digital assistant, which is a method of linking notifications, weather, stock information, and messages built into the smartphone with the AI assistant function. Was. In addition, there have been attempts to combine the AI assistant function with a standalone speaker device, examples of such attempts are Google Assistant and Amazon Alexa. Line Friends and Kakao Mini are representative. In addition, attempts to control indoor lighting with voice or strengthen home security by linking the artificial intelligence assistant function with the Internet of Things (IoT) have also been attempted by communication service companies.

However, the conventional AI secretary's'answering questions' and'following orders' methods have a limited service level, so the man-machine interaction method needs to evolve one step further. Even if there is no explicit question or command from the user, it is desirable for the AI assistant to operate in a way that watches, hears, and thinks the situation in which the user is facing and serves as an active action. For example, after an AI assistant observes the user, it is desirable to work with "You are cooking. You can remove particulate matter and toxic gases by turning on the hood. Would you like to turn on the hood?" This method of operation is a new type of service that conventional AI assistants cannot provide.

However, in order for such a service to be possible, an AI assistant must have a high-level situational awareness function. It is known that humans can intuitively perform such a situational awareness function, but it is very difficult for an artificial intelligence assistant implemented with computer software to observe the user's situation and perform high-level situational awareness. In addition, even if it is possible to recognize such a high level situation, it remains a very difficult task to implement a service useful to users by utilizing it.

It is an object of the present invention to provide a technology for performing high-level context recognition based on video and sound information for a person who is reflecting themselves on a smart mirror in general, and providing a chatbot service on the smart mirror based on this.

In particular, it is an object of the present invention to perform high-level context recognition by processing adaptive multi-biometric information based on the weight of image and sound information for a person reflecting themselves on a smart mirror, and to provide a chatbot service on the smart mirror based on this. It is to provide a technology that can provide a chatbot service based on the same level of high-level situational awareness in multiple smart mirrors by adjusting the biometric weight information by reflecting the satisfaction evaluation and distributing the biometric weight change information by putting it in the blockchain. .

In order to achieve the above object, the smart mirror chatbot system based on high-level context awareness using adaptive multi-biometric information according to the present invention is based on image and sound information for a user who illuminates itself from the front of the smart mirror 100. As a smart mirror chatbot system based on high-level context awareness using adaptive multi-biometric information that performs context awareness and provides a chatbot service through a mirror interface in response to the perceived context, it is installed in a specific location in front of the smart mirror 100 A mirror camera 111 that photographs the front of the mirror to generate image data related to a user; A mirror microphone 112 installed in the smart mirror 100 and collecting ambient sound to generate sound data related to a user; An environment sensor module 113 for acquiring surrounding environment data of the smart mirror 100; A weight table storage unit 130 that stores a biometric information weight table that changes and defines a weight of image data and sound data for a high level situation in response to surrounding environment data; A reflection weight obtained by installing and operating in the internal memory space of the smart mirror 100, performing image-based context recognition and sound-based context recognition from image data and sound data, respectively, and applying surrounding environment data to the biometric information weight table According to the data coding process of the image-based context recognition result and the sound-based context recognition result, an individual context or complex code is obtained, and the individual context or complex code is processed with artificial intelligence-based data to perform complex context recognition. Edge artificial intelligence unit 140 to obtain; And a chatbot processing unit 150 that provides a text or voice-based real-time chat service through a user interface of the smart mirror 100 in response to a high-level context recognition result.

In the present invention, the edge artificial intelligence unit 140 includes an image-based context recognition unit 141 that processes image data with artificial intelligence-based data to obtain image-based context recognition information related to a user; A sound-based context recognition unit 142 for acquiring acoustic-based context recognition information related to a user by processing acoustic data based on artificial intelligence; An environmental condition recognition unit 143 that obtains environmental condition information related to a user from the surrounding environment data; By applying environmental condition information to the biometric information weight table, a weighted calculation is obtained by acquiring the reflection weight of image data and sound data for high-level context recognition, and applying the above-acquired reflection weight to image-based context recognition information and sound-based context recognition information And a biometric information weighting coding unit 144 for obtaining a composite code indicating an individual situation related to a user by data coding the weighting operation result according to a preset format; It may include a complex context recognition unit 145 that obtains a high-level context recognition result related to a user by performing complex context recognition by processing individual context recognition complex codes with artificial intelligence-based data.

In addition, in the present invention, the mirror camera 111 is provided with a 3D camera module to generate 3D image data on the face and expression of the user in front of the mirror, and the mirror microphone 112 is provided with an array microphone module of 3D sound. Thus, three-dimensional sound data on the surrounding sound is generated, and the environment sensor module 113 may be configured to include a sensor interface module for acquiring ambient environment data of at least one of illumination, temperature, humidity, and fine dust from an external IoT sensor. have.

In addition, the smart mirror chatbot system based on high-level context awareness according to the present invention includes: a feedback receiving unit 160 receiving feedback for a satisfaction evaluation for a real-time chat service through a user interface of the smart mirror 100; When a positive satisfaction evaluation is provided from the feedback receiving unit 160, the difference between the reflection weight of the image data corresponding to the current environmental condition information and the acoustic data is increased with respect to the biometric information weight table, and the negative satisfaction level from the feedback receiving unit 160 A weight adjustment unit 170 for reducing a difference between a reflection weight of image data and sound data corresponding to current environmental condition information with respect to the biometric information weight table when receiving evaluation; A block chain processing unit 180 that generates block data by including the biometric information weight table and performs block chain update according to a preset schedule to participate in sharing distributed information on the weight distribution ledger 200; to be further included. I can.

According to the present invention, there is an advantage in that it is possible to provide a chatbot service based on a high-level situation for a person who is projecting himself or herself in a smart mirror. At this time, by applying a weight table linked to an environmental factor for multiple biometric information, there is an advantage of maintaining high-level context recognition performance even when environmental conditions change.

In addition, according to the present invention, there is an advantage in that the performance of high-level context recognition can be gradually improved by adjusting the weight table by reflecting the satisfaction evaluation for the chatbot service. At this time, the edge AI has the advantage of applying a weight table in a local environment and distributing and managing the update information of the weight table through a blockchain, so that it can operate smoothly even if the network environment is insufficient.

[Fig. 1] is a diagram conceptually showing the overall configuration of a smart mirror chatbot system according to the present invention.
[Fig. 2] is a diagram conceptually showing a process of performing whether a smart mirror is in a high-level situation in the present invention.
3 is a diagram showing an example of a biometric information weight table in the present invention.
[Fig. 4] is a diagram showing the concept of adjusting a biometric information weight table and applying a block chain in the present invention.
[Fig. 5] is a diagram showing an example of a general emotional expression.
[Fig. 6] is a diagram showing an example of an emotion classification system that can be favorably used in the present invention.
[Fig. 7] is a diagram showing an example of an emotion code encoding that can be preferably used in the present invention.

Hereinafter, the present invention will be described in detail with reference to the drawings.

[Fig. 1] is a diagram conceptually showing the overall configuration of a smart mirror chatbot system according to the present invention.

The smart mirror 100 is very similar in appearance to a general mirror, but is a device in which various electronic equipment and displays are provided.

In the present invention, the smart mirror 100 includes a mirror camera 111 and a mirror microphone 112, through which image data and sound data are obtained in relation to a user who illuminates itself from the front of the mirror. In addition, the smart mirror 100 includes an environmental sensor module 113, through which it acquires environmental data related to the surrounding of the mirror, for example, data related to one or more of illumination, temperature, humidity, and fine dust.

The smart mirror 100 is a high-level situation in which the edge artificial intelligence unit 140 identifies the current situation or status of the user by artificial intelligence technology based on image data and sound data in relation to the user who illuminates itself from the front. -level context awareness). In this case, the edge artificial intelligence unit 140 is preferably implemented inside the smart mirror 100 in consideration of the response speed. In addition, in the present invention, the environmental data is used to determine how the edge artificial intelligence unit 140 will combine image data and sound data, and reference information on the data combination is stored in the biometric information weight table. In this case, the biometric information weight table is stored in the weight table storage unit 130 implemented in the internal storage space of the smart mirror 100.

The smart mirror 100 provides a real-time chat service to the user through its own mirror interface, for example, a built-in LCD screen or a built-in speaker, based on the result of recognizing the high-level situation derived by the edge artificial intelligence unit 140.

[Fig. 2] is a diagram conceptually showing a process of performing whether the smart mirror 100 is in a high-level situation in the present invention.

First, a specific front position of the smart mirror 100, for example, a mirror camera 111 installed on the top of the mirror or inside the windshield, photographs the front of the mirror to generate image data 121. As a natural result of the general use attribute of the smart mirror 100, the mirror camera 111 generates image data related to a user who is currently looking at itself (eg, a face) from the front of the mirror. In this case, the mirror camera 111 is advantageous in whether it is a high-level situation of the present invention to have a 3D camera module to generate 3D image data for a face and facial expressions for a user in front of the mirror.

At the same time, the mirror microphone 112 installed in the smart mirror 100 collects ambient sound and generates sound data 122 related to the user. As a result, the mirror microphone 112 records what the user says, what other people say to the user, and sounds around the user. At this time, the mirror microphone 112 is advantageous in whether it is a high-level situation of the present invention to have a 3D sound array microphone module to generate 3D sound data for ambient sound.

In addition, the environment sensor module 113 acquires data related to one or more of the surrounding environment data 123 of the smart mirror 100, such as illumination, temperature, humidity, and fine dust. In this case, the environmental sensor module 113 may be configured to include an IoT sensor device that directly acquires these environmental data, or may include a sensor interface module for acquiring these environmental data from an external IoT sensor.

The weight table storage unit 130 is implemented in an internal storage space (e.g., flash memory) of the smart mirror 100, and in the present invention, reflecting weights of image data and sound data for high-level situations are applied to the surrounding environment data. It stores the biometric-weight table to be changed and defined.

In order to show the concept of the biometric information weight table, [Fig. 3] presents an example that can be favorably used in the present invention. 3 shows an example of utilizing voice recognition information, image recognition information, pulse recognition information, moving recognition (user's movement) information, and body temperature recognition information as the user's biometric information. 2 shows a configuration using images and sounds, but the present invention is not limited thereto, and multiple biometric information in various combinations may be used according to implementation examples.

Referring to FIG. 3, it can be seen that the weight for utilizing multiple biometric information for high-level context recognition processing is changed and defined by environmental factors. That is, a state in which a weight combination of multiple biometric information is changed and defined in response to a change in illumination, which is an environmental factor, is shown in [Fig. 3]. In this specification, the term'adaptive' refers to a configuration in which a combination of weights reflected in whether multiple biometric information is a high-level situation is changed according to a sensing result of an environmental element.

The edge-AI 140 is an artificial intelligence data processing module installed and operated in an internal memory space (eg, flash memory, DRAM) of the smart mirror 100. In general, artificial intelligence data processing is configured to use an external server. In the present invention, an edge-AI method in which an AI algorithm is built in advance in a smart mirror is preferable in order to secure real-time responsiveness of a chat service.

The edge artificial intelligence unit 140 is provided with image data 121, sound data 122, and environment data 123, and whether it is an image-based situation or sound-based situation from the image data 121 and sound data 122 Each is performed, and according to the reflected weight obtained by applying the environmental data 123 to the biometric information weight table, the image-based context recognition result and the sound-based context recognition result are combined and data-coded to obtain an individual context recognition composite code. . Then, by processing this individual context recognition complex code with artificial intelligence-based data, complex context recognition is performed, and through this, high-level context recognition results based on image and sound are obtained.

The chatbot processing unit 150 provides a text or voice-based real-time chat service through a user interface of the smart mirror 100, for example, a built-in LCD screen or a built-in speaker in response to a high-level context recognition result. The chatbot software itself is a commonly used technology, but the difference of the present invention is that the content of the chat is set by reflecting the result of whether the edge AI unit 140 has obtained a high-level situation.

Hereinafter, a preferred internal functional configuration of the edge artificial intelligence unit 140 in the present invention will be described in more detail. Referring to Fig. 2, the edge artificial intelligence unit 140 includes an image-based context recognition unit 141, a sound-based context recognition unit 142, an environmental condition recognition unit 143, and a biometric information weighted coding unit 144 , And a complex situation recognition unit 145.

First, the image-based context recognition unit 141 obtains image-based context recognition information related to a user by processing the image data 121 on the image data 121 according to an artificial intelligence-based data processing, for example, a Convolutional Neural Network (CNN) algorithm. In other words, image-based individual situation recognition is performed. Recognizing an image-based situation may include identifying a user's emotion based on facial expressions, for example.

[Fig. 5] is a diagram showing an example of a general emotion expression, and [Fig. 6] and [Fig. 7] show an example of an emotion classification system and an emotion code encoding that can be favorably used in the present invention. [Fig. 6] is a diagram showing the wheel of emotions of Robert Plutchik, which is known to be suitable for classification of emotions for English, and [Fig. 7] is suitable for classification of emotions for Korean (Korean). Although it is a known sentiment code rule, it can be well used in the present invention.

In the emotional classification embodiment of FIG. 6, for example, 3 bits, 3 bits, and 2 bits may be allocated to each of the three major classifications, sub classifications, and fine classifications in a direction away from the center of a circle to perform emotion code encoding. . According to these rules, each emotion classification result can be derived in a manner similar to that indicated by'emotional code encoding' in the right column of [Fig. 7]. In addition, the emotion code encoding of [Fig. 7] is designed to respond to human bright emotions as the value increases and to respond to dark emotions of humans as the value decreases. In the present invention, the image-based context recognition information is set to have a certain regularity (a tendency) as described above, and this tendency is advantageous for the AI operation or learning of the edge artificial intelligence unit 140.

The sound-based context recognition unit 142 obtains sound-based context recognition information related to a user by processing the sound data 122 based on artificial intelligence data, for example, according to a Deep Neural Network (DNN) algorithm. In other words, it performs sound-based individual situation recognition. In the present invention, the acoustic-based context perception information is set to have a certain regularity (a tendency), and this tendency is advantageous for the AI operation or learning of the edge AI unit 140.

The environmental condition recognition unit 143 acquires environmental condition information related to a user from the surrounding environment data 123.

The biometric information weighting coding unit 144 applies the environmental condition information acquired by the environmental condition recognition unit 143 to the biometric information weight table as shown in [Fig. 3], so that the image data and sound data corresponding to the environmental element are Acquire the reflection weight. The acquired reflection weight is a value indicating to what extent each of the image-based individual situation or the sound-based individual situation or the result is reflected in performing whether it is a high-level situation in the current environmental condition. The biometric information weighted coding unit 144 performs a weighted calculation by applying a reflection weight to the image-based context recognition information and the sound-based context recognition information. That is, the image-based context information and the sound-based context information are data values coded to have a certain tendency, and a weighting operation is performed by applying each reflection weight to them and adding or concatenating them. The result of this weighting operation is data coded according to a preset format, which is called a composite code whether it is an individual situation related to the user.

The complex context recognition unit 145 obtains a result of recognizing a high-level context related to a user by performing complex context recognition by processing the individual context or the complex code with artificial intelligence-based data. This corresponds to performing complex situation recognition by putting and combining a plurality of individual situation recognition information into an artificial intelligence processing module. It is preferable that the artificial intelligence algorithm used by the complex context recognition unit 145 is generated and learned in advance and stored in the internal memory space of the smart mirror 100, and the satisfaction level for the real-time chatbot service of the smart mirror 100 is evaluated. It is more desirable to configure to continuously update by performing artificial intelligence learning by receiving feedback.

[Fig. 4] is a diagram showing a concept in which the smart mirror 100 adjusts the biometric information weight table and applies a block chain in the present invention.

A number of smart mirrors 100 are connected through a block chain to distribute and share the weighted distributed ledger 200. In [Fig. 4], a plurality of smart mirrors 100 may be distributed in several areas or may be installed in several points in a single building. These smart mirrors 100 self-adjust the biometric information weight table, and share the weight change information accordingly through the blockchain.

As described above, in the present invention, the smart mirror 100 has its own edge artificial intelligence unit 140 installed and operated in the internal memory space without using an external artificial intelligence server in order to secure real-time responsiveness of the chat service. Operate as. In the present invention, while adopting a configuration to adjust the biometric information weight table to gradually improve the high-level context-aware performance, a block chain was adopted to show the performance of a plurality of smart mirrors 100 having the same style of high-level context recognition. . This is a unique configuration of the present invention for achieving performance improvement and uniformity in a system configuration environment employing the edge artificial intelligence unit 140 without using a single artificial intelligence server.

To this end, referring to [Fig. 2], a feedback receiving unit 160, a weight adjusting unit 170, and a block chain processing unit 180 are provided.

First, the feedback receiving unit 160 receives feedback on a satisfaction evaluation for a real-time chat service through the user interface of the smart mirror 100. The user interface of the smart mirror 100 that receives feedback from the satisfaction evaluation is not limited to the interface implemented in hardware in the smart mirror 100, and a channel input by the user through a smartphone application interlocking with the smart mirror 100 is also possible.

According to the satisfaction evaluation provided by the feedback receiving unit 160, the weight adjustment unit 170 converts the biometric information weight table into numerical values, in particular, to the current environmental condition information used to provide a chat service to the user who has entered the current satisfaction evaluation. The reflection weight value of the corresponding image data and sound data is adjusted.

First, when a positive satisfaction evaluation is provided from the feedback receiving unit 160, the difference between the reflection weight of the image data and the sound data is enlarged in the biometric information weight table. In this case, since the reflection weight values of the image data and the sound data are well applied to a certain extent, the difference in the reflection weights is widened to test the possibility of enhancing performance in a high-level situation.

Conversely, when a negative satisfaction evaluation is provided from the feedback receiver 160, the difference between the reflection weights of the image data and the sound data in the biometric information weight table is reduced. In this case, the possibility of improvement is tested by changing the reflection weight values of image data and sound data in opposite directions.

The block chain processing unit 180 generates block data by including the biometric information weight table operated by the corresponding smart mirror 100, and performs a block chain update according to a preset schedule (e.g., 24 hours) to perform a weight distribution ledger (200). ) For the smart mirror 100 to participate in distributed information sharing. Through this, a number of smart mirrors 100 participating in the blockchain can exhibit the same level of high-level situation recognition performance, and through this, a chatbot service can be uniformly implemented.

Blockchain generally has the concept of a public digital transaction ledger in which transaction information generated on a network is duplicated and stored distributedly as many as the number of nodes shared among network participants. Blockchain technology is actively applied around cryptocurrency.In the present invention, blockchain technology is used to propagate changes in the weight of biometric information generated in the smart mirror 100 to a plurality of smart mirrors 100. To this end, a number of smart mirrors 100 participate in the digital ledger management of the blockchain as a kind of peer. In this specification, the digital ledger of the blockchain is named weighted distributed ledger 200 to indicate that it is a distributed ledger that manages information related to the biometric information weight table.

The weighted distributed ledger 200 is a form in which a number of transaction data is configured like a block and connected to each other, and one block is peers that are linked to the blockchain in the corresponding blockchain update cycle (e.g., 24 hours), That is, the plurality of smart mirrors 100 correspond to newly generated block data in response to the biometric information weight adjustment event. Referring to [Fig. 4], biometric information weight change information is basically reflected in such a block. The biometric information weight table itself after weight adjustment can be included, or change information for a specific weight value can be extracted and included. have. Due to the nature of the blockchain, which is based on a distributed network, data once written in the weighted distributed ledger 200 cannot be modified without permission, so it is reliable.

On the other hand, the present invention can be implemented in the form of a computer-readable code on a nonvolatile computer-readable recording medium. Various types of storage devices exist as such non-volatile recording media, such as hard disks, SSDs, CD-ROMs, NAS, magnetic tapes, web disks, and cloud disks, and codes are distributed and stored in multiple storage devices connected through a network. It can be implemented and executed. In addition, the present invention may be implemented in the form of a computer program stored in a medium in order to execute a specific procedure in combination with hardware.

100: smart mirror
111: mirror camera 112: mirror microphone
113: environmental sensor module
130: weight table storage unit
140: edge artificial intelligence unit
141: image-based context recognition unit 142: sound-based context recognition unit
143: environmental condition recognition unit 144: biometric information weighted coding unit
145: Complex situation awareness department
150: chatbot processing unit 160: feedback receiving unit
170: weight adjustment unit 180: block chain processing unit
200: weighted distributed ledger

Claims (5)

High-level using adaptive multi-biometric information that performs high-level situation recognition based on image and sound information for the user who illuminates itself from the front of the smart mirror 100 and provides a chatbot service through the mirror interface in response to the recognized situation. As a context awareness-based smart mirror chatbot system,
A mirror camera 111 installed at a specific front position of the smart mirror 100 and photographing the front of the mirror to generate image data related to the user;
A mirror microphone 112 installed in the smart mirror 100 and collecting ambient sound to generate sound data related to the user;
An environmental sensor module 113 for acquiring environmental data of the smart mirror 100;
A weight table storage unit 130 for storing a biometric information weight table for changing and defining the weights of the image data and the sound data corresponding to the surrounding environment data for recognizing a high level situation;
It is installed and operated in the internal memory space of the smart mirror 100, performs image-based context or sound-based context, respectively, from the image data and the acoustic data, and applies the surrounding environment data to the biometric information weight table According to the acquired reflection weight, the image-based context or the sound-based context recognition result is data-coded to obtain an individual context or a composite code, and the individual context or the composite code is processed with artificial intelligence-based data to determine whether a composite context or not. An edge artificial intelligence unit 140 that acquires a high-level situation recognition result by performing it;
A chatbot processing unit 150 that provides a text- or voice-based real-time chat service through a user interface of the smart mirror 100 in response to the high level context recognition result;
A smart mirror chatbot system based on high-level context awareness using adaptive multi-biometric information comprising a.
The method according to claim 1,
The edge artificial intelligence unit 140,
An image-based context recognition unit 141 for obtaining image-based context recognition information related to a user by processing the image data based on artificial intelligence;
A sound-based context recognition unit (142) for acquiring acoustic-based context recognition information related to a user by processing the acoustic data based on artificial intelligence;
An environmental condition recognition unit 143 for obtaining environmental condition information related to a user from the surrounding environment data;
Applying the environmental condition information to the biometric information weight table to obtain a reflection weight of the image data and the sound data for high-level context recognition, and the acquired reflection in the image-based context recognition information and the acoustic context recognition information A biometric information weighting coding unit 144 that applies a weight to perform a weighting operation and obtains a composite code indicating an individual situation related to a user by data coding the weighting operation result according to a preset format;
A complex context recognition unit 145 for obtaining a high-level context recognition result related to a user by processing the individual context recognition complex code with artificial intelligence-based data to perform complex context recognition;
Smart mirror chatbot system based on high-level context perception using adaptive multi-biometric information, characterized in that it comprises a.
The method according to claim 1,
The mirror camera 111 is provided with a 3D camera module to generate three-dimensional image data of a face shape and expression for a user in front of the mirror,
The mirror microphone 112 is provided with an array microphone module of 3D sound to generate 3D sound data for ambient sound,
The environmental sensor module 113 includes a sensor interface module for acquiring ambient environment data of at least one of illuminance, temperature, humidity, and fine dust from an external IoT sensor. Smart mirror chatbot system based on context awareness.
The method according to claim 2,
A feedback receiving unit 160 for receiving feedback on a satisfaction evaluation for the real-time chat service through a user interface of the smart mirror 100;
When a positive satisfaction evaluation is provided from the feedback receiving unit 160, the difference between the reflection weight of the image data corresponding to the current environmental condition information and the acoustic data is increased with respect to the biometric information weight table, and the feedback receiving unit ( A weight adjustment unit 170 for reducing a difference between a reflection weight of the image data corresponding to the current environmental condition information and the sound data with respect to the biometric information weight table when a negative satisfaction evaluation is received from 160);
Smart mirror chatbot system based on high-level context awareness using adaptive multi-biometric information, characterized in that it further comprises.
The method of claim 4,
A block chain processing unit 180 that generates block data including the biometric information weight table and performs block chain update according to a preset schedule to participate in sharing distributed information on the weight distribution ledger 200;
Smart mirror chatbot system based on high-level context awareness using adaptive multi-biometric information, characterized in that it further comprises.

Images