CN113852524A - Intelligent household equipment control system and method based on emotional characteristic fusion - Google Patents

Intelligent household equipment control system and method based on emotional characteristic fusion Download PDF

Info

Publication number
CN113852524A
CN113852524A CN202110810721.9A CN202110810721A CN113852524A CN 113852524 A CN113852524 A CN 113852524A CN 202110810721 A CN202110810721 A CN 202110810721A CN 113852524 A CN113852524 A CN 113852524A
Authority
CN
China
Prior art keywords
information
features
instruction
user
intelligent household
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202110810721.9A
Other languages
Chinese (zh)
Inventor
李鸣秋
杭云
郭宁
施唯佳
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Tianyi Digital Life Technology Co Ltd
Original Assignee
Tianyi Smart Family Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Tianyi Smart Family Technology Co Ltd filed Critical Tianyi Smart Family Technology Co Ltd
Priority to CN202110810721.9A priority Critical patent/CN113852524A/en
Publication of CN113852524A publication Critical patent/CN113852524A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L12/00—Data switching networks
    • H04L12/28—Data switching networks characterised by path configuration, e.g. LAN [Local Area Networks] or WAN [Wide Area Networks]
    • H04L12/2803—Home automation networks
    • H04L12/2816—Controlling appliance services of a home automation network by calling their functionalities
    • H04L12/282—Controlling appliance services of a home automation network by calling their functionalities based on user interaction within the home
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/211—Selection of the most significant subset of features
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00—Speech recognition
    • G10L15/22—Procedures used during a speech recognition process, e.g. man-machine dialogue
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00—Speech recognition
    • G10L15/26—Speech to text systems
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L25/00—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
    • G10L25/03—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 characterised by the type of extracted parameters
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L25/00—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
    • G10L25/03—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 characterised by the type of extracted parameters
    • G10L25/24—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 characterised by the type of extracted parameters the extracted parameters being the cepstrum
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L25/00—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
    • G10L25/48—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use
    • G10L25/51—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use for comparison or discrimination
    • G10L25/60—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use for comparison or discrimination for measuring the quality of voice signals
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00—Speech recognition
    • G10L15/22—Procedures used during a speech recognition process, e.g. man-machine dialogue
    • G10L2015/223—Execution procedure of a spoken command

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Health & Medical Sciences (AREA)
  • Computational Linguistics (AREA)
  • Human Computer Interaction (AREA)
  • Theoretical Computer Science (AREA)
  • Multimedia (AREA)
  • Acoustics & Sound (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Signal Processing (AREA)
  • Data Mining & Analysis (AREA)
  • General Physics & Mathematics (AREA)
  • Evolutionary Computation (AREA)
  • General Engineering & Computer Science (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Molecular Biology (AREA)
  • Mathematical Physics (AREA)
  • General Health & Medical Sciences (AREA)
  • Computing Systems (AREA)
  • Software Systems (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Evolutionary Biology (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Automation & Control Theory (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Quality & Reliability (AREA)
  • User Interface Of Digital Computer (AREA)

Abstract

The application discloses an intelligent household equipment control system and method based on emotional characteristic fusion. The system comprises: the data processing module is used for collecting voice information input by a user and preprocessing the information; the data analysis module is used for analyzing the preprocessed information to acquire the emotional state of the user; the instruction generation and distribution module is used for generating an instruction based on the emotional state of the user and sending the instruction to the intelligent household equipment; and the execution module is used for controlling the intelligent household equipment based on the issued instruction. The application also discloses a corresponding method.

Description

Intelligent household equipment control system and method based on emotional characteristic fusion
Technical Field
The present application relates to the field of intelligent control and smart home devices, and more particularly, to a system and method for controlling smart home devices based on emotional feature fusion.
Background
Voice control technology has been widely used in today's society, and smart home devices have also been commonly used in our daily lives. At present, a voice control technology used in smart home devices mainly collects voice, analyzes and discriminates voice data, and then sends an instruction to the smart home devices to enable the smart home devices to execute related operations. Many intelligent home devices that exist in the current market, because the instruction received is comparatively single, more be that the user unilaterally communicates with the machine, consequently can't carry out individualized, humanized, diversified operation to every user, and then can't accomplish more intelligent interaction between people and machine. In the face of the rising smart home device market, how to make the smart home devices more fit to the actual situation of the user according to the actual requirements of the individual user is very important to cater to the personalized preferences of the individual user.
Human beings are the only animals that can carry out such complex speech communication, and speech is an important behavioral signal that can reflect their emotion. Different voices, tones, sizes and rhythms of the voices contain a lot of emotional information of people. For example, when a person is happy or the arousing state becomes high, the tone and the loudness are increased, and the rhythm is also increased; the trembling and duration of the sound can become relatively sluggish as the person is depressed.
The technology for carrying out voice control on intelligent household equipment in the prior art obviously cannot meet the requirements at all. Therefore, there is an urgent need in the art for a system and method capable of learning emotional characteristics of human beings through voice, and then performing intelligent control on smart home devices based on the fusion of the emotional characteristics.
Disclosure of Invention
The following presents a simplified summary of one or more aspects in order to provide a basic understanding of such aspects. This summary is not an extensive overview of all contemplated aspects, and is intended to neither identify key or critical elements of all aspects nor delineate the scope of any or all aspects. Its sole purpose is to present some concepts of one or more aspects in a simplified form as a prelude to the more detailed description that is presented later.
Aiming at the problems in the prior art, the application provides an intelligent household equipment control system and method based on deep learning emotional characteristic fusion. The system and the method can acquire acoustic feature and text feature data information from voice of a user, perform feature fusion to obtain mixed features, analyze the mixed features, and finally acquire information of the user in aspects of emotion, psychological change and the like, so that the control equipment sends out related instructions. After the intelligent device receives the instruction, the intelligent device is adjusted according to the current emotion and psychological state of the user, so that the user is in a suitable scene, comfortable product experience can be brought to the user, and the requirements of the user are better met.
According to the first aspect of the application, an intelligent home equipment control system based on emotional feature fusion is disclosed, including:
the data processing module is used for collecting voice information input by a user and preprocessing the information;
the data analysis module is used for analyzing the preprocessed information to acquire the emotional state of the user;
the instruction generation and distribution module is used for generating an instruction based on the emotional state of the user and sending the instruction to the intelligent household equipment; and
and the execution module is used for controlling the intelligent household equipment based on the issued instruction and reporting the current equipment information.
According to a preferred embodiment of the present application, the pre-processing comprises extracting features and feature fusion, wherein the extracting features comprises extracting both acoustic information features and text information features, and the feature fusion comprises adding or weighting the acoustic information features and the text information features to obtain fused features.
According to a preferred embodiment of the present application, the analysis of the preprocessed information includes training data and result verification, and the emotional state of the user is obtained based thereon.
The instruction generating and distributing module is also used for inquiring other intelligent household equipment and issuing the instruction to the intelligent household equipment after generating the instruction.
According to the preferred embodiment of the present application, the acoustic information features include tone, loudness, structure, and the text information features include part of speech, word frequency, keywords.
According to a second aspect of the application, an intelligent home device control method based on emotional feature fusion is disclosed, and comprises the following steps:
collecting voice information input by a user and preprocessing the information;
analyzing the preprocessed information to obtain the emotional state of the user;
generating an instruction based on the emotional state of the user and sending the instruction to the intelligent household equipment; and
and controlling the intelligent household equipment based on the issued instruction.
To the accomplishment of the foregoing and related ends, the one or more aspects comprise the features hereinafter fully described and particularly pointed out in the claims. The following description and the annexed drawings set forth in detail certain illustrative features of the one or more aspects. These features are indicative, however, of but a few of the various ways in which the principles of various aspects may be employed and the present description is intended to include all such aspects and their equivalents.
Drawings
So that the manner in which the above recited features of the present application can be understood in detail, a more particular description of the disclosure briefly summarized above may be had by reference to aspects, some of which are illustrated in the appended drawings. It is to be noted, however, that the appended drawings illustrate only certain typical aspects of this application and are therefore not to be considered limiting of its scope, for the description may admit to other equally effective aspects.
In the drawings:
fig. 1 is a block diagram illustrating a system for controlling smart home devices based on emotional feature fusion according to an embodiment of the application;
fig. 2 is a schematic diagram illustrating extracting acoustic information features according to an embodiment of the present application;
FIG. 3 is a schematic diagram illustrating extracting textual information features according to an embodiment of the application;
FIG. 4 is a functional block diagram illustrating a data processing module and a data analysis module according to an embodiment of the present application;
fig. 5 is a schematic diagram illustrating feature fusion of extracted acoustic information features and textual information features according to an embodiment of the application;
FIG. 6 is a functional block diagram illustrating an instruction generation dispatch module according to an embodiment of the present application;
FIG. 7 is a functional block diagram illustrating execution modules according to an embodiment of the present application; and
fig. 8 is a schematic diagram illustrating a data analysis module determining an emotional state of a user based on fused feature vectors according to an embodiment of the application.
Detailed Description
The detailed description set forth below in connection with the appended drawings is intended as a description of various configurations and is not intended to represent the only configurations in which the concepts described herein may be practiced. The detailed description includes specific details to provide a thorough understanding of the various concepts. It will be apparent, however, to one skilled in the art that these concepts may be practiced without these specific details. In some instances, well known components are shown in block diagram form in order to avoid obscuring such concepts.
It is to be understood that other embodiments will be evident based on the present disclosure, and that system, structural, process, or mechanical changes may be made without departing from the scope of the present disclosure.
Referring to the figures, one or more components and one or more methods that may perform the actions or functions described herein depict aspects. In an aspect, the term "component" or "module" as used herein may be one of the parts that make up a system, may be hardware or software or some combination thereof, and may be divided into other components. While the operations described below in the figures are presented in a particular order and/or as performed by example components, it should be understood that the order of the actions and the components performing the actions may vary depending on the implementation. Further, it should be understood that the following acts or functions may be performed by a specially programmed processor, a processor executing specially programmed software or computer readable media, or by any other combination of hardware components and/or software components capable of performing the described acts or functions.
Fig. 1 is a block diagram illustrating a system for controlling smart home devices based on emotional feature fusion according to an embodiment of the present application.
As shown in fig. 1, the system of the present application may be divided into four modules: a data processing module; a data analysis module; an instruction generation and distribution module; and an execution module. The functions of the various modules are described in detail below in conjunction with fig. 1.
The data processing module is used for collecting voice information input by a user and preprocessing the data of the information, and comprises: acoustic information features such as tone, loudness and rhythm and text information features such as words in the voice are extracted, and the features are fused to form a mixed feature vector. These emotion-related feature vectors are then used to train a classifier or regression system (see fig. 4).
Extracting features
The process of extracting the acoustic information features is to perform frame division after the sound is filtered and denoised (pre-emphasis). The voice signal after framing is convenient to process. After the speech signal is frequency domain transformed (FFT, filter, DCT), its acoustic information features are extracted using MFCC (see fig. 2). As is well known, the most commonly used Speech feature in Speech Recognition (Speech Recognition) and Speaker Recognition (Speaker Recognition) is the Mel-Frequency Cepstral Coefficients (MFCCs).
It is based on the physiological characteristics of human ear to change the waveform of each frame into a multidimensional vector, which can be simply understood as that the vector contains the content information of the frame of speech. The result of the whole process is that a frame of speech signal is simply represented by a 12-20-dimensional vector.
The extraction of text information features is to convert voice into characters, then perform word segmentation on the sentences, and train word vectors by using word2vec (a word embedding model proposed in google 2013, which is the most common text representation model at present) so as to obtain feature vectors of the sentences (see fig. 4).
The above processes for feature extraction are well known in the art and therefore will not be described in detail herein.
Feature fusion
Since the prior art often only focuses on the single feature of acoustic information or text information, the emotion capture is very limited. According to the technical scheme, two information features with different dimensionalities are fused, so that the features of the voice information can be extracted more comprehensively, and the recognition performance of the system is improved more effectively (see fig. 5).
In one embodiment, the feature fusion can be performed by using a bit-by-bit addition mode, and the existing feature vectors are expressed by using a mathematical expression of V1 and V2.
To fuse these two eigenvectors, the corresponding elements are added directly, i.e., V1+ V2. I.e. is V ═ xi|xi＝v1[i]+v2[i],i＝1,.....n}。
Where V represents the feature vector, x represents the value of each dimension of the feature vector, and n represents the dimension of the feature vector.
Of course, other methods exist in the art, such as a weight-based fusion method, in addition to feature fusion in a mode of adding bits by bits. All of which fall within the scope of the present application.
The data analysis module is used for analyzing the preprocessed voice data. Specifically, the data analysis module adopts a deep learning method, such as CNN + LSTM, to label the obtained fused feature vector, train and verify the obtained fused feature vector, and determine what emotional state the user is currently in based on the labeled fused feature vector, such as happy (happy), difficult (sadness), angry (anger), nausea (dispost), fear (fear), surprise (surrise), and the like (see fig. 8).
The instruction generation and distribution module is used for distributing instructions of different situations to the intelligent equipment by the control center according to the emotional state of the user after obtaining the emotional state information of the user. For example, the artificial intelligence voice interaction control device is used as a control center of the intelligent home device, and when a user sends voice to the device, the device receives and analyzes the information and then switches corresponding scenes, so that the voice tone of the artificial intelligence interacting with the user is changed. Meanwhile, according to the analyzed emotional state of the user, other intelligent home devices bound in the system are queried, and then corresponding instructions are sent to the other intelligent home devices (see fig. 6).
And the execution module is positioned in the corresponding intelligent household equipment. The intelligent household equipment in the same environment can check and receive the instruction sent by the instruction generating and distributing module, and judge whether the instruction is executed by the intelligent household equipment according to the instruction. And after the result is obtained, inquiring and matching the corresponding operation to be performed and starting to execute, namely adjusting the parameters of the device according to the instruction and reporting the device information to a control center (see fig. 7).
Specific implementations of systems and methods based on emotional feature fusion according to the present application are described in detail below with reference to specific embodiments.
When the user A returns home, the mood is happy, the tone is slightly high, and the rhythm is light. The device control center (e.g., smart speaker) is woken up by voice. When the sound box is awakened after sensing the voice, the voice of the user is read at the same time, the emotion of the current user is analyzed and judged through the built-in data analysis processing module, after the data that the user is happy at the moment is obtained, the intelligent sound box dialogues with the user by using correspondingly cheerful voice, and the emotion data is used as emotion factors and is transmitted to the instruction generating and distributing module. And the instruction generating and distributing module generates an instruction according to the obtained emotional factor, inquires other indoor intelligent equipment and then sends the instruction to each intelligent household equipment. The execution module is responsible for regulating and controlling intelligent equipment such as doors and windows, temperature, light, music and the like of a room, carrying out corresponding parameter regulation and control and reporting equipment information to the control center. Therefore, the setting of the intelligent household equipment is more suitable for the actual situation of the user, the individual preference of the individual user is met, and the individual experience of the user is stronger.
According to the system and the method, the emotional information contained in the user voice is analyzed through the combination of acoustic information features such as tone, loudness and rhythm and text information features such as part of speech and word frequency by adopting a mode of fusing the emotional features and the voice features.
Meanwhile, the analyzed emotion information is used for controlling the intelligent equipment, so that a brand new control method of the intelligent household equipment is further developed, and intelligent interaction between people and machines is further deepened. Moreover, the application is not limited to specific products and fields, and thus the life cycle can be long and can last 10 years or even longer.
As will be appreciated by those skilled in the art, although the present application is described above in connection with smart home devices, the system and method is not limited to the field of smart home, but may be generalized to any other field of smart device control as understood by those skilled in the art.
The automatic control of the intelligent household equipment in the current market has no pertinence, most voice control is only a surface paraphrasing instruction of a simple execution voice, and the experience of a user is not good. The present application clearly has significant technical advantages over many control systems and methods that exist in the prior art.
Firstly, by adopting the system and the method for fusing the emotional characteristics, after the acoustic information characteristics such as tone, loudness and rhythm and the text information characteristics such as part of speech and word frequency are fused, the information contained in the user speech is comprehensively analyzed, so that the use of the intelligent equipment is closer to the actual condition of the user, and the experience of the user on the intelligent household equipment is enhanced.
Secondly, the control method tends to be customized in a personalized way, and the flexibility and the sensitivity are higher, so that the equipment is more intelligent.
And thirdly, through continuously optimizing the instruction, the linkage of the intelligent household equipment is faster, so that the interaction between the user and the machine is deepened.
It is to be understood that the specific order or hierarchy of steps in the methods disclosed is an illustration of exemplary processes. Based upon design preferences, it is understood that the specific order or hierarchy of steps in the methods or methodologies described herein may be rearranged. The accompanying method claims present elements of the various steps in a sample order, and are not meant to be limited to the specific order or hierarchy presented unless specifically recited herein.
The previous description is provided to enable any person skilled in the art to practice the various aspects described herein. Various modifications to these aspects will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other aspects. Thus, the claims are not intended to be limited to the aspects shown herein, but is to be accorded the full scope consistent with the language claims, wherein reference to an element in the singular is not intended to mean "one and only one" (unless specifically so stated) but rather "one or more". The term "some" means one or more unless specifically stated otherwise. A phrase referring to "at least one of a list of items refers to any combination of those items, including a single member. By way of example, "at least one of a, b, or c" is intended to encompass: at least one a; at least one b; at least one c; at least one a and at least one b; at least one a and at least one c; at least one b and at least one c; and at least one a, at least one b, and at least one c. All structural and functional equivalents to the elements of the various aspects described throughout this disclosure that are known or later come to be known to those of ordinary skill in the art are expressly incorporated herein by reference and are intended to be encompassed by the claims. Moreover, nothing disclosed herein is intended to be dedicated to the public regardless of whether such disclosure is explicitly recited in the claims.

Claims (10)

1. The utility model provides an intelligent household equipment control system based on emotional characteristic fuses, the system includes:
the data processing module is used for collecting voice information input by a user and preprocessing the information;
the data analysis module is used for analyzing the preprocessed information to acquire the emotional state of the user;
the instruction generation and distribution module is used for generating an instruction based on the emotional state of the user and sending the instruction to the intelligent household equipment; and
and the execution module is used for controlling the intelligent household equipment based on the issued instruction and reporting the current equipment information.
2. The system of claim 1, wherein the pre-processing comprises extracting features and feature fusion, wherein the extracting features comprises extracting both acoustic information features and textual information features, and the feature fusion comprises adding or weighting the acoustic information features and the textual information features to obtain fused features.
3. The system of claim 1, wherein analyzing the preprocessed information includes training data and result validation and obtaining the emotional state of the user based thereon.
4. The system of claim 1, wherein the instruction generation and distribution module is further configured to query other smart home devices and issue the instructions to the other smart home devices after generating the instructions.
5. The system of claim 2, wherein the acoustic information features include pitch, loudness, structure, and the text information features include part of speech, word frequency, keywords.
6. An intelligent household equipment control method based on emotional feature fusion comprises the following steps:
collecting voice information input by a user, and preprocessing the information;
analyzing the preprocessed information to obtain the emotional state of the user;
generating an instruction based on the emotional state of the user and sending the instruction to the intelligent household equipment; and
and controlling the intelligent household equipment based on the issued instruction and reporting the current equipment information.
7. The method of claim 6, wherein the pre-processing comprises extracting features and feature fusion, wherein the extracting features comprises extracting both acoustic information features and textual information features, and the feature fusion comprises adding or weighting the acoustic information features and the textual information features to obtain fused features.
8. The method of claim 6, wherein analyzing the preprocessed information includes training data and result validation, and obtaining the emotional state of the user based thereon.
9. The method of claim 6, wherein the instruction generation and distribution module further queries other smart home devices and issues the instruction to the other smart home devices after generating the instruction.
10. The method of claim 7, wherein the acoustic information features include pitch, loudness, structure, and the text information features include part of speech, word frequency, keywords.
CN202110810721.9A 2021-07-16 2021-07-16 Intelligent household equipment control system and method based on emotional characteristic fusion Pending CN113852524A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110810721.9A CN113852524A (en) 2021-07-16 2021-07-16 Intelligent household equipment control system and method based on emotional characteristic fusion

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110810721.9A CN113852524A (en) 2021-07-16 2021-07-16 Intelligent household equipment control system and method based on emotional characteristic fusion

Publications (1)

Publication Number Publication Date
CN113852524A true CN113852524A (en) 2021-12-28

Family

ID=78975193

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110810721.9A Pending CN113852524A (en) 2021-07-16 2021-07-16 Intelligent household equipment control system and method based on emotional characteristic fusion

Country Status (1)

Country Link
CN (1) CN113852524A (en)

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105334743A (en) * 2015-11-18 2016-02-17 深圳创维－Rgb电子有限公司 Intelligent home control method and system based on emotion recognition
CN111368609A (en) * 2018-12-26 2020-07-03 深圳Tcl新技术有限公司 Voice interaction method based on emotion engine technology, intelligent terminal and storage medium
CN112489688A (en) * 2020-11-09 2021-03-12 浪潮通用软件有限公司 Neural network-based emotion recognition method, device and medium
CN112631137A (en) * 2020-04-02 2021-04-09 张瑞华 Intelligent household control method and intelligent control equipment applied to biological feature recognition

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105334743A (en) * 2015-11-18 2016-02-17 深圳创维－Rgb电子有限公司 Intelligent home control method and system based on emotion recognition
CN111368609A (en) * 2018-12-26 2020-07-03 深圳Tcl新技术有限公司 Voice interaction method based on emotion engine technology, intelligent terminal and storage medium
CN112631137A (en) * 2020-04-02 2021-04-09 张瑞华 Intelligent household control method and intelligent control equipment applied to biological feature recognition
CN112489688A (en) * 2020-11-09 2021-03-12 浪潮通用软件有限公司 Neural network-based emotion recognition method, device and medium

Similar Documents

Publication Publication Date Title
US10706873B2 (en) Real-time speaker state analytics platform
JP6684391B2 (en) Multi-domain real-time question and answer system
CN111415677B (en) Method, apparatus, device and medium for generating video
US9501743B2 (en) Method and apparatus for tailoring the output of an intelligent automated assistant to a user
EP3642833B1 (en) Dynamic and/or context-specific hot words to invoke automated assistant
US9396180B1 (en) System and method for analyzing video content and presenting information corresponding to video content to users
US20200075024A1 (en) Response method and apparatus thereof
CN109410927A (en) Offline order word parses the audio recognition method combined, device and system with cloud
US10224030B1 (en) Dynamic gazetteers for personalized entity recognition
CN113010138B (en) Article voice playing method, device and equipment and computer readable storage medium
CN112418011A (en) Method, device and equipment for identifying integrity of video content and storage medium
US11450306B2 (en) Systems and methods for generating synthesized speech responses to voice inputs by training a neural network model based on the voice input prosodic metrics and training voice inputs
CN108847222B (en) Speech recognition model generation method and device, storage medium and electronic equipment
CN109994106B (en) Voice processing method and equipment
CN110136726A (en) A kind of estimation method, device, system and the storage medium of voice gender
CN110516083B (en) Album management method, storage medium and electronic device
Shah et al. Speech emotion recognition based on SVM using MATLAB
CN110910898B (en) Voice information processing method and device
CN110889008B (en) Music recommendation method and device, computing device and storage medium
CN109074809A (en) Information processing equipment, information processing method and program
Schuller et al. Ten recent trends in computational paralinguistics
CN113948076A (en) Voice interaction method, device and system
CN113852524A (en) Intelligent household equipment control system and method based on emotional characteristic fusion
Johar Paralinguistic profiling using speech recognition
CN113516963B (en) Audio data generation method and device, server and intelligent sound box

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
TA01 Transfer of patent application right

Effective date of registration: 20220129

Address after: Room 1423, No. 1256 and 1258, Wanrong Road, Jing'an District, Shanghai 200072

Applicant after: Tianyi Digital Life Technology Co.,Ltd.

Address before: 201702 3rd floor, 158 Shuanglian Road, Qingpu District, Shanghai

Applicant before: Tianyi Smart Family Technology Co.,Ltd.

TA01 Transfer of patent application right