CN105045122A - Intelligent household natural interaction system based on audios and videos - Google Patents

Intelligent household natural interaction system based on audios and videos Download PDF

Info

Publication number
CN105045122A
CN105045122A CN201510355845.7A CN201510355845A CN105045122A CN 105045122 A CN105045122 A CN 105045122A CN 201510355845 A CN201510355845 A CN 201510355845A CN 105045122 A CN105045122 A CN 105045122A
Authority
CN
China
Prior art keywords
module
information
system
cloud server
signal
Prior art date
Application number
CN201510355845.7A
Other languages
Chinese (zh)
Inventor
张子兴
陈宇翔
黄力
林子楠
Original Assignee
张子兴
陈宇翔
黄力
林子楠
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 张子兴, 陈宇翔, 黄力, 林子楠 filed Critical 张子兴
Priority to CN201510355845.7A priority Critical patent/CN105045122A/en
Publication of CN105045122A publication Critical patent/CN105045122A/en

Links

Classifications

    • GPHYSICS
    • G05CONTROLLING; REGULATING
    • G05BCONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
    • G05B15/00Systems controlled by a computer
    • G05B15/02Systems controlled by a computer electric
    • GPHYSICS
    • G05CONTROLLING; REGULATING
    • G05BCONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
    • G05B19/00Programme-control systems
    • G05B19/02Programme-control systems electric
    • G05B19/418Total factory control, i.e. centrally controlling a plurality of machines, e.g. direct or distributed numerical control [DNC], flexible manufacturing systems [FMS], integrated manufacturing systems [IMS], computer integrated manufacturing [CIM]
    • GPHYSICS
    • G05CONTROLLING; REGULATING
    • G05BCONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
    • G05B2219/00Program-control systems
    • G05B2219/20Pc systems
    • G05B2219/26Pc applications
    • G05B2219/2642Domotique, domestic, home control, automation, smart house

Abstract

The invention discloses an intelligent household natural interaction system based on audios and videos. The system mainly comprises four parts, i.e.,a front end, a central processing unit, a back end and a cloud end. The front end comprises a microphone system, a camera system, a third party sensor interface and a feedback module. The front end is used to collect sound and picture associated information and display system feedback. The central processing unit comprises an audio signal processing and information retrieving module, a video signal processing and information retrieving module, a third party signal processing and information retrieving module and an information integrating module. The central processing unit processes the acquired sounds and visual signals and utilizes a machine learning method to get useful commanding order. The back end comprises an indoor signal controlling and emitting module and a cloud server communication module. The back end is used to convert obtained commanding order into emitting signals. At the same time, the back end provides a communication channel for the system. The cloud end comprises the cloud server which provides computing resources, storing resources and communicating resources. The system is highly human-machine interactive; and the intelligent household natural interaction system greatly improves convenience for controlling household electric appliances and acquiring information.

Description

A kind of Smart Home natural interaction system based on Voice & Video

Technical field

The present invention relates to areas of information technology, be specifically related to a kind of wired home natural interaction system based on Voice & Video technology.

Background technology

Under the technology tide of Internet of Things and artificial intelligence, Smart Home technical development is very rapid, has occurred the hardware product that many wired homes are relevant, as intelligent thermostat and the smoke alarm of Nest, the Hue intelligent bulbs of Philip, the intelligent refrigerator of Haier, smart lock of August etc.These smart machines greatly meet the demand for control of people to household equipment.But these equipment lack unified control criterion and interface.In general, the control method that they have a set of independently system separately and match, such as mobile phone A pp.It is this that incompatible what bring to user is the control complexity such as repeatedly repetitive operation.Given this, Apple has issued oneself parametric controller Homekit, and Samsung develops SmartHome platform, and Quicky has Wink and Relay platform etc., and these platforms or equipment improve the convenience to smart machine manipulation to a certain extent.But these platforms existing or equipment all adopt more single Voice command, or smart mobile phone control etc.Under many circumstances, these single interaction modes all can not realize with household equipment naturally mutual.

Through inquiry, patent publication No. be the system of CN102298443 and control method have employed the method reading lip reading carry out auxiliary home environment under speech recognition system.But lip reading identification is greatly subject to the restriction such as angle, position, illumination of user, is difficult to reach higher discrimination, thus affects Consumer's Experience in practical application.Meanwhile, the interface that this system is not opened to the outside world and cloud service platform, this greatly limited to this system extendability and usable range.

Summary of the invention

In order to overcome the deficiency controlled existing intelligent home equipment, the invention provides a set of wired home interactive system based on Voice & Video.Compare existing household equipment to control and interactive system, the means that the present invention adopts voice and image to combine reach more natural, healthy and strong man-machine interaction experience; Provide unified information analysis and convergence platform, the product with other Smart Home manufacturer compatible can be expanded well, make user operation more natural and convenient.

The present invention for the adopted concrete technical scheme that solves the problem as follows:

Based on a Smart Home intersection control routine for Voice & Video, mainly comprise front end, CPU (central processing unit), rear end and high in the clouds.Front end includes the information search modules such as Voice & Video, as microphone system and camera system, third party's sensor interface and feedback display module.CPU (central processing unit) comprises Audio Signal Processing and information extraction modules, video frequency signal processing and information extraction modules, third party signalling process and information extraction interface module, information fusion module.Rear end include control signal transmitter module, with cloud server communication module.High in the clouds is cloud server.

Described microphone system is microphone array.Original sound signal by the acoustic information under specific sample frequency and coded system real-time collecting home environment, and is passed to audio signal analysis and information extraction modules by it.

Described audio signal analysis and information extraction modules, for carrying out noise reduction to the voice signal collected, fall the process in early stage such as echo, Sound seperation, and carry out auditory localization, Speaker Identification, voice wake up and the process such as speech recognition and command detection.

First, Kalman filter is carried out preliminary except making an uproar to the signal of each sound channel, and carries out end-point detection, cutoff signal; May be there is the situation of many sound sources mixing in the signal split, described module by nonnegative matrix algorithm by different sound source separately, extracts object sound source; Then, echo technology restraint speckle and echo fall in the noise reduction that signal carries out multichannel by GCCdelay-and-sumbeamforming algorithm.

While application multichannel noise and echo suppression technology, described sonic location system utilizes different sound channel and the signal time difference (TDOA) that receives to determine the position of sound source.After sound source is determined, system can, according to the automatic adjustment direction in speaker position, make system of the present invention and user be in relative suitable angle.

Then, fall the signal after echo process through noise reduction and can be input to described speaker verification's module.This module is for judging user's whether systematic right to use of tool.This module adopts i-vector algorithm, confirms speaker.The control authority that unauthorized user will not have system.

If user has rights of using, voice wake-up module can judge whether the sound detected comprises and wake key word up.If have, present system can enter activation interactive mode from sleep pattern.The voice signal that subsequent probes arrives directly can send into speech recognition and natural semantic understanding module.

Voice signal is converted into Word message by sound identification module, and by natural language understanding technology, analysis and resolution goes out to control or interactive instruction.

Described camera system comprises common camera and depth camera.It is responsible for action and the action message of collecting user.Specifically, it is for the face of detecting user, gesture and movable information.

First, Face datection is carried out to the RGB image that common camera obtains.Once detect and comprise face, recognition of face and authentication will be carried out to associated picture.Here, the face detected and the authorized user face prestored are compared (based on face characteristic and machine learning) in native system, if be proved to be successful, action recognition module will be activated.The depth image being input as depth camera acquisition of this module, first this image will be used to real-time skeleton tracking, obtain the information such as human synovial position.The information of skeleton tracking can also be used for user location, and native system can, according to the automatic adjustment direction of customer location, make system of the present invention and user be in relative suitable angle.

Then, human synovial information can compared with the action in maneuver library in native system.Mate action accordingly once find, the command information be associated with this action will be generated.

Described third party's sensor interface and third party signalling process and information extraction interface module, for Function Extension, for other developers following provide corresponding interface, to realize customization function.

Described feedback display module, for the communication of system and user and mutual.When instruction identification fuzzy or wrong time, user can be confirmed by feedback display module or be corrected.

Described information fusion module, for phonetic order, gesture instruction and other command informations that fusion detection arrives, utilize probability to differentiate the instruction of user, its mathematical description is: , wherein .Wherein, for instruction prediction probability value; , with be respectively voice, video and other sensor to instruction prediction probability; , with be respectively voice, video and other sensor signal weight.

Described control signal transmitter module, for steering order being converted into the actual signal that can control household electrical appliances, utilizes the communication such as infrared, RF radio frequency, bluetooth, wifi, Zigbee, Z-Wave to reach the object of manipulation household electrical appliances.

Described with cloud server communication module, for communicating of information fusion module and cloud server.Local side can send Gains resources instruction to high in the clouds, and respective resources turns back to local side by this module.High in the clouds also sends instruction by described module to local side, to realize the Long-distance Control of household electrical appliances, or by information transmission in family to high in the clouds.

Described cloud server, for a) for local side provides extra computational resource; B) for this locality provides extra storage space or data backup; C) for user terminal as mobile phone etc. provides information exchange platform; D) for user provides other information, as query search or music etc.

The invention has the beneficial effects as follows: 1) front end have employed voice and the mutual mode of gesture identification, improves mutual naturality; 2) interactive voice mode and visual interactive mode are independent and complementation, and they both can work alone, also can collaborative work, breach single interactive mode application limitation in the family, improve the robustness of man-machine interaction; 3) provide third-party interface, third party developer as required, can add signal transacting and the information extraction function of other sensors, for present system provides good expansion; 4) rear end provides various wireless communication mode, provides good compatibility; 5) local and remote two kinds of mode of operations are provided.Local mode ensure that safety and the privacy of custom system physically, and remote mode can be supplied to the extra information of user and more senior service.

Accompanying drawing explanation

Fig. 1 is the wired home natural interaction control system frame diagram that the present invention is based on Voice & Video.

Fig. 2 is Audio Signal Processing of the present invention and information extraction process flow diagram.

Fig. 3 is video frequency signal processing of the present invention and information extraction process flow diagram.

Fig. 4 is information fusion block flow diagram of the present invention.

Embodiment

For problems of the prior art, a kind of wired home interactive system is proposed in the present invention, this system, based on intelligent audio and video analysis treatment technology, can improve the accuracy of the convenience of man-machine interaction, comfort level and manipulation, have very high compatibility and extensibility simultaneously.

In order to make technical scheme of the present invention more clear, below in conjunction with accompanying drawing and example, the present invention program to be described in further details, and these describe and will be considered to exemplary.

As shown in Figure 1, this system comprises: front end, CPU (central processing unit), rear end and high in the clouds four part.Front end primary responsibility sound and picture signal and etc. the collection of information, and the feedback display of system; CPU (central processing unit) primary responsibility processes the sound collected and visual signal, utilizes the method for machine learning and pattern-recognition to obtain useful command information; Rear end primary responsibility transfers the instruction of acquisition to missile signal, controls electrical equipment etc. in family; Also can obtain and the information of exchange from the cloud server in high in the clouds simultaneously.

The present invention can voice signal in real time in explorer and picture signal when opening.

Wherein the detail flowchart of Audio Signal Processing of the present invention and information extraction as shown in Figure 2.Speak when user is in, such as, " turn on light ".This sound is detected (step 202) by microphone system, through multi-channel audio signal preliminary except make an uproar process after (step 202), carry out end-point detection and segmentation (step 203), extract the sound signal comprising " turning on light ".When having multi-acoustical while during sounding (such as multiple user speaks simultaneously, or has music when user speaks simultaneously), system can be separated (step 204) sound source, peels off background sound.Meanwhile, the present invention can analyze the source (step 205) of sound, comes the direction (step 206) of timely adjustment System.Such as, when user is positioned at the back side of system, system can rotate 180 degree with front in the face of user.At further noise reduction with after falling echo process (step 207), system can confirm user, if not the member with authority, will ignore; If so, the sound import of this user will be processed (step 208) further, and carries out system wake-up detection (step 209).If the sound of user can mate wake key word up as " turning on light ", system will switch to wake-up states from sleep state; Otherwise continue detection and wake instruction up.After system wake-up, speech recognition (step 210) will be carried out to the sound of subsequent user.Such as, when recognition result is " please turn on this electric light ", " heighten air-conditioner temperature ", " play the blue and white porcelain of Zhou Jielun ", " check my unread mail " etc., system extracts key word wherein, as " turning on ", " this electric light ", " heightening ", " air-conditioning ", " temperature ", " broadcasting ", " Zhou Jielun ", " blue and white porcelain ", " watching ", " I ", " unread mail " etc. by nature semantic understanding (step 211).These key words can be sent to information fusion module (module 15), do next step process.

The present invention, while detection sound signal, is also detecting vision signal in real time.Wherein the detailed process of video frequency signal processing and information extraction as shown in Figure 3.This module be input as vision signal, it comprises two kinds: common RGB picture signal (301) and depth image signal (302).First this module carries out Face datection (303) in real time in RGB image, this image is carried out recognition of face and identity validation (304) when face having been detected.Once identity is confirmed and this identity has corresponding rights of using, then permit further operation, otherwise turn back to Face datection step.Simultaneously, this module also will utilize depth image to carry out real-time skeleton tracking (305), and this tracked information may be used for consumer positioning (306), and the direction of real-time adjustment native system is to reach best Detection results (307).Once the identity of user is confirmed, the framework information of this user will be used to carry out action recognition (309), and these actions that can identify will be (308) that are stored among maneuver library.Finally, the action identified will be translated into the instruction (311) among instruction database (310).This instruction can be fed to information fusion module and be further processed.

When systems axiol-ogy is to sound or gesture instruction signal, information fusion module (as shown in Figure 4) of the present invention decides last instruction by by maximum probability.Some of them typical apply scene is exemplified below.

1) only audio system activates.Such as, when user cooks, both hands are in busy condition.If now user wants to listen song, then can wake native system up by voice, and select the song wanting broadcasting.

2) only video system activates.Such as, during family party, indoor are in height noisy environments, the manipulation that owner can realize household equipment by gesture instruction.

3) audio frequency and video activates simultaneously.Now, audio and video information supplements mutually, promotes the identification accuracy of instruction.Such as, while user says " closing this lamp ", with finger to specific electric light, the present invention can close specific electric light in conjunction with voice and gesture instruction.

As above-mentioned example of staging an uprising, audio system of the present invention, video system both can work alone, also can associated working.The height reaching man-machine interaction merges, and improves the robustness of instruction identification simultaneously.If the instruction maximum probability that information fusion module obtains occurs conflict lower than the threshold value of specifying or audio frequency and video instruction, when namely instruction identification is uncertain, system can pass through the confirmation that feedback display module (module 14) obtains user.The present invention's feedback has three kinds of modes: voice, image and word.Word feedback can directly show on feedback display module, and voice are needed by being play by user feedback module after phonetic synthesis.Such as, the present invention when indefinite whether will turning off the light, " you determine to close electric light? " can be fed back similarly, image also can export in feedback display module, improves the interactivity of system.User can utilize voice or gesture to confirm to native system, to avoid maloperation.

Next, information fusion module according to classes of instructions, will give control signal transmitter module (module 16), or give cloud server communication module (module 17) to process.

Wherein relate to the instruction of household electrical appliances, such as " turn on electric light ", control signal transmitter module can be given.This module handle " turning on electric light " changes into the signal specific that electric controller can receive, and sends.This signal can be infrared, RF radio frequency, bluetooth, Wifi, Zigbee, Z-Wave etc.Similarly, user also can use action command, and such as the gesture of hand left and right paddling switches the music of broadcasting, and the gesture of upper and lower paddling regulates volume.

Wherein relate to the instruction of internet, such as Query Information etc., will by being sent to high in the clouds with cloud server communication module.Such as " check my unread mail ", this instruction will be sent to cloud server, obtain the mail do not read and be back to local side; Again such as, " downloading the blue and white porcelain of Zhou Jielun ", this module downloads song by the music libraries of network attached server equally.

The above-mentioned cloud server mentioned is connected with local side.Its function is but is not limited to following example.

1) for local side provides extra computational resource.Speech recognition, recognition of face etc. of arriving involved in the present invention, by part or all of computation requirement being transferred to cloud server, to save local computational resource, can improve recognition correct rate simultaneously.

2) for local side provides the space of information back-up and storage.The data such as document, picture, video according to the needs of oneself, can be saved in high in the clouds by user.The advantage of this example is to make user whenever can obtain this data by internet anywhere.

3) for third party provides resources portal.Such as played songs, by the cloud server of native system, can be connected into third party's music libraries to obtain song and to return, to meet the entertainment requirements of user.Again such as, by cloud server, user can inquire about online commodity, for ecommerce is provided access.

4) for mobile terminal (as mobile phone, flat board etc.) provide the entrance of message exchange.User can be connected with cloud server by mobile phone A PP, and utilizes cloud server that control signal is transmitted to local side, reaches the object controlling electrical equipment in family.This example, can meet the demand of user's remote control domestic electrical equipment.Again such as, the situation in family can be inquired about in mobile terminal by cloud server, and the present invention can send by cloud server the request obtaining image or video to local side.

The two-way communication of cloud server and local side, the user both for being in provides Internet portal, obtains external information; The entrance of local side can be provided for user outside again, understand and monitor the situation in family.

In addition, cloud server of the present invention is the optional module of user.Namely when closing cloud server module, the present invention will be in local mode of operation, be cut off with the communicative channel of external information.Do the information security that can ensure user like this, but also can lose the function that cloud server provides.

To those skilled in the art, obvious the present invention is not limited to the details of above-mentioned one exemplary embodiment, and when not deviating from spirit of the present invention or essential characteristic, can realize the present invention in other specific forms.Therefore, no matter from which point, all should embodiment be regarded as exemplary, and be nonrestrictive, scope of the present invention has appended claims instead of above-mentioned explanation to limit, and all changes be therefore intended in the implication of the equivalency by dropping on claim and scope are included in the present invention.Any Reference numeral in claim should be considered as the claim involved by limiting.

Claims (10)

1. based on a Smart Home natural interaction system for Voice & Video, it is characterized in that, comprise front end, CPU (central processing unit), rear end and high in the clouds four part;
Wherein front end comprises:
Microphone system (111) is microphone permutation, sends acoustic information for Real-time Collection;
Camera system (121) is infrared depth camera and common camera, for the concurrent sending information image of Real-time Collection;
Third party's sensor interface (131), gathers other possible information for implementing and sends this information;
Feedback display module (14), for responding and showing the reaction made command information;
Its CPU (central processing unit) comprises:
Audio Signal Processing and information extraction modules (112), for the treatment of voice signal, and extract the information such as speaker, semanteme wherein;
Video frequency signal processing and information extraction modules (122), for the treatment of picture signal, and extract gesture, face, movable information wherein;
Third party signalling process and information extraction interface module (132), for the treatment of the signal of third party's sensor collection, and extract relevant information;
Information fusion module (15), for merging the information that above-mentioned module (112,122 and 132) is collected, generates final instruction;
Its rear end comprises:
Indoor control signal transmitter module (16), for the wireless signal becoming specifically can launch by concrete instruction transformation, controls household electrical appliances;
With cloud server communication module (17), for concrete instruction transformation is become concrete network operation, obtain and exchange the information on internet network;
Its high in the clouds comprises:
Cloud server (18), for providing necessary computational resource, storage resources, Internet resources and communication pipe for user.
2. Smart Home natural interaction system according to claim 1, it is characterized in that, its Audio Signal Processing and information extraction modules (112) also comprise except making an uproar, except the signal pre-processing module such as echo, Sound seperation, and Speaker Identification module, voice wake-up module, based on the sound identification module of degree of depth study and natural language understanding module.
3. Smart Home natural interaction system according to claim 1, is characterized in that, its video frequency signal processing and information extraction modules (122) also comprise the modules such as gesture identification, human face detection and tracing, motion detection.
4. Smart Home natural interaction system according to claim 1 and the AV signal process described in claim 3 and 4 and information extraction modules, is characterized in that,
If indoor environment is not suitable for audio system, video system can complete independently instruction identification and perform work;
If indoor environment is not suitable for video system, audio system can complete independently instruction identification and perform work;
If indoor environment is normal, audio-visual system can side information mutually, collaborative work;
When branch prediction probability is lower than predetermined value or when going out prediction appearance conflict, feedback information is by response and be shown to described feedback display module.
5. Smart Home natural interaction system according to claim 1 and the AV signal process described in claim 3 and 4 and information extraction modules, it is characterized in that, real-time consumer positioning can be carried out, in real time the orientation of the described system of adjustment according to auditory localization module, human detection and Face datection.
6. Smart Home natural interaction system according to claim 1 and the AV signal process described in claim 3 and 4 and information extraction modules, it is characterized in that, the access rights of user to described system can be judged according to Speaker Identification module and face recognition module.
7. Smart Home natural interaction system according to claim 1, is characterized in that, described third party's sensor interface (131) and third party signalling process and information extraction modules (132) are for the following possible function of expanding system.
8. Smart Home natural interaction system according to claim 1, is characterized in that, described indoor control signal transmitter module collection (16) becomes infrared, RF radio frequency, bluetooth, the communication such as wifi, Zigbee, Z-Wave;
Indoor control signal transmitter module (16), according to different household electrical appliance, selects specific radio communication and coded system; Simultaneously for uncommon household electrical appliance brand, described module can learn its radio communication coding.
9. Smart Home natural interaction system according to claim 1, is characterized in that, whether can access cloud server (18) according to the selection of user;
If do not access cloud server, described signal transacting and information extraction modules (112,122 and 132) can at processing locality; If access cloud server, all or part of computational resource of described signal transacting and information extraction modules (112,122 and 132) can transfer to cloud server process.
10. Smart Home interactive system according to claim 1 and cloud server according to claim 9, is characterized in that,
If described CPU (central processing unit) access cloud server, user instruction can also obtain corresponding storage resources, information resources by described cloud server; User also can connect cloud server by terminals such as mobile phones and controls and monitor indoor situations.
CN201510355845.7A 2015-06-24 2015-06-24 Intelligent household natural interaction system based on audios and videos CN105045122A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201510355845.7A CN105045122A (en) 2015-06-24 2015-06-24 Intelligent household natural interaction system based on audios and videos

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201510355845.7A CN105045122A (en) 2015-06-24 2015-06-24 Intelligent household natural interaction system based on audios and videos

Publications (1)

Publication Number Publication Date
CN105045122A true CN105045122A (en) 2015-11-11

Family

ID=54451742

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201510355845.7A CN105045122A (en) 2015-06-24 2015-06-24 Intelligent household natural interaction system based on audios and videos

Country Status (1)

Country Link
CN (1) CN105045122A (en)

Cited By (37)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105653709A (en) * 2015-12-30 2016-06-08 广东顺德中山大学卡内基梅隆大学国际联合研究院 Intelligent home voice text control method
CN105957535A (en) * 2016-04-15 2016-09-21 青岛克路德机器人有限公司 Robot voice signal detecting and identifying system
CN105955040A (en) * 2016-05-20 2016-09-21 深圳市大拿科技有限公司 Intelligent household system according to real-time video picture visual control and control method thereof
CN106019973A (en) * 2016-07-30 2016-10-12 杨超坤 Smart home with emotion recognition function
CN106200396A (en) * 2016-08-05 2016-12-07 易晓阳 A kind of appliance control method based on Motion Recognition
CN106200395A (en) * 2016-08-05 2016-12-07 易晓阳 A kind of multidimensional identification appliance control method
CN106254186A (en) * 2016-08-05 2016-12-21 易晓阳 A kind of interactive voice control system for identifying
CN106406119A (en) * 2016-11-15 2017-02-15 福州大学 Service robot based on voice interaction, cloud technology and integrated intelligent home monitoring
CN106444415A (en) * 2016-12-08 2017-02-22 湖北大学 Smart home control method and system
CN106445455A (en) * 2016-09-29 2017-02-22 深圳前海弘稼科技有限公司 Planting device and method for controlling planting device
CN106507047A (en) * 2016-11-15 2017-03-15 浙江工业大学 A kind of audio-video terminal system towards smart home
CN106653020A (en) * 2016-12-13 2017-05-10 中山大学 Multi-business control method and system for smart sound and video equipment based on deep learning
CN106647305A (en) * 2016-12-28 2017-05-10 重庆金鑫科技产业发展有限公司 Control method and terminal
CN106710594A (en) * 2016-11-17 2017-05-24 北京中科汇联科技股份有限公司 Intelligent speech interaction system based on cloud end
CN106782540A (en) * 2017-01-17 2017-05-31 联想(北京)有限公司 Speech ciphering equipment and the voice interactive system including the speech ciphering equipment
CN106896743A (en) * 2015-12-18 2017-06-27 北京奇虎科技有限公司 A kind of instruction responding device, the method for control terminal equipment, server and device
CN106899460A (en) * 2015-12-18 2017-06-27 北京奇虎科技有限公司 A kind of instruction responding device, the method for control terminal equipment, server and system
CN106910500A (en) * 2016-12-23 2017-06-30 北京第九实验室科技有限公司 The method and apparatus of Voice command is carried out to the equipment with microphone array
CN107065586A (en) * 2017-05-23 2017-08-18 中国科学院自动化研究所 Interactive intelligent home services system and method
CN107230476A (en) * 2017-05-05 2017-10-03 众安信息技术服务有限公司 A kind of natural man machine language's exchange method and system
CN107395746A (en) * 2017-08-21 2017-11-24 时瑞科技(深圳)有限公司 A kind of Internet of things system
CN107682240A (en) * 2017-09-27 2018-02-09 四川长虹电器股份有限公司 A kind of distributed sound interactive system for intelligent domestic
WO2018027504A1 (en) * 2016-08-09 2018-02-15 曹鸿鹏 Lighting control method
WO2018027505A1 (en) * 2016-08-09 2018-02-15 曹鸿鹏 Lighting control system
WO2018027507A1 (en) * 2016-08-09 2018-02-15 曹鸿鹏 Emotion recognition-based lighting control system
CN107734213A (en) * 2016-08-11 2018-02-23 漳州立达信光电子科技有限公司 Intelligent domestic electronic installation and system
CN107993660A (en) * 2017-12-26 2018-05-04 江苏可美智能科技股份有限公司 Speech control system for Internet of Things intelligence control system
CN108154140A (en) * 2018-01-22 2018-06-12 北京百度网讯科技有限公司 Voice awakening method, device, equipment and computer-readable medium based on lip reading
CN108229391A (en) * 2018-01-02 2018-06-29 京东方科技集团股份有限公司 Gesture identifying device and its server, gesture recognition system, gesture identification method
CN108563208A (en) * 2018-06-28 2018-09-21 马雷明 Intelligent domestic system and its control method
CN109036430A (en) * 2018-09-29 2018-12-18 芜湖星途机器人科技有限公司 Voice control terminal
CN109151393A (en) * 2018-10-09 2019-01-04 深圳市亿联智能有限公司 A kind of sound fixation and recognition method for detecting
CN109168110A (en) * 2018-09-29 2019-01-08 芜湖星途机器人科技有限公司 External hanging type speech packet
CN109473095A (en) * 2017-09-08 2019-03-15 北京君林科技股份有限公司 A kind of intelligent home control system and control method
WO2019071989A1 (en) * 2017-10-13 2019-04-18 歌尔股份有限公司 Smart device speech enhancement method and device and smart device
CN109803013A (en) * 2019-01-21 2019-05-24 浙江大学 A kind of weak interactive system and its control method based on artificial intelligence
CN109884908A (en) * 2019-03-14 2019-06-14 苏州宏裕千智能设备科技有限公司 Cloud platform, apparatus control method and system, readable storage medium storing program for executing

Cited By (43)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106899460A (en) * 2015-12-18 2017-06-27 北京奇虎科技有限公司 A kind of instruction responding device, the method for control terminal equipment, server and system
CN106896743A (en) * 2015-12-18 2017-06-27 北京奇虎科技有限公司 A kind of instruction responding device, the method for control terminal equipment, server and device
CN105653709A (en) * 2015-12-30 2016-06-08 广东顺德中山大学卡内基梅隆大学国际联合研究院 Intelligent home voice text control method
CN105957535A (en) * 2016-04-15 2016-09-21 青岛克路德机器人有限公司 Robot voice signal detecting and identifying system
CN105955040A (en) * 2016-05-20 2016-09-21 深圳市大拿科技有限公司 Intelligent household system according to real-time video picture visual control and control method thereof
CN106019973A (en) * 2016-07-30 2016-10-12 杨超坤 Smart home with emotion recognition function
CN106254186A (en) * 2016-08-05 2016-12-21 易晓阳 A kind of interactive voice control system for identifying
CN106200395A (en) * 2016-08-05 2016-12-07 易晓阳 A kind of multidimensional identification appliance control method
CN106200396A (en) * 2016-08-05 2016-12-07 易晓阳 A kind of appliance control method based on Motion Recognition
WO2018027505A1 (en) * 2016-08-09 2018-02-15 曹鸿鹏 Lighting control system
WO2018027507A1 (en) * 2016-08-09 2018-02-15 曹鸿鹏 Emotion recognition-based lighting control system
WO2018027504A1 (en) * 2016-08-09 2018-02-15 曹鸿鹏 Lighting control method
CN107734213A (en) * 2016-08-11 2018-02-23 漳州立达信光电子科技有限公司 Intelligent domestic electronic installation and system
CN106445455A (en) * 2016-09-29 2017-02-22 深圳前海弘稼科技有限公司 Planting device and method for controlling planting device
CN106406119B (en) * 2016-11-15 2019-05-10 福州大学 Service robot based on interactive voice, cloud and integrated intelligent Household monitor
CN106507047B (en) * 2016-11-15 2019-05-31 浙江工业大学 A kind of audio-video terminal system towards smart home
CN106507047A (en) * 2016-11-15 2017-03-15 浙江工业大学 A kind of audio-video terminal system towards smart home
CN106406119A (en) * 2016-11-15 2017-02-15 福州大学 Service robot based on voice interaction, cloud technology and integrated intelligent home monitoring
CN106710594A (en) * 2016-11-17 2017-05-24 北京中科汇联科技股份有限公司 Intelligent speech interaction system based on cloud end
CN106444415A (en) * 2016-12-08 2017-02-22 湖北大学 Smart home control method and system
CN106653020A (en) * 2016-12-13 2017-05-10 中山大学 Multi-business control method and system for smart sound and video equipment based on deep learning
CN106910500A (en) * 2016-12-23 2017-06-30 北京第九实验室科技有限公司 The method and apparatus of Voice command is carried out to the equipment with microphone array
US10453457B2 (en) 2016-12-23 2019-10-22 Beijing Xiaoniao Tingting Technology, Co., Ltd. Method for performing voice control on device with microphone array, and device thereof
CN106910500B (en) * 2016-12-23 2020-04-17 北京小鸟听听科技有限公司 Method and device for voice control of device with microphone array
CN106647305A (en) * 2016-12-28 2017-05-10 重庆金鑫科技产业发展有限公司 Control method and terminal
CN106782540A (en) * 2017-01-17 2017-05-31 联想(北京)有限公司 Speech ciphering equipment and the voice interactive system including the speech ciphering equipment
CN107230476A (en) * 2017-05-05 2017-10-03 众安信息技术服务有限公司 A kind of natural man machine language's exchange method and system
CN107065586A (en) * 2017-05-23 2017-08-18 中国科学院自动化研究所 Interactive intelligent home services system and method
CN107065586B (en) * 2017-05-23 2020-02-07 中国科学院自动化研究所 Interactive intelligent home service system and method
CN107395746A (en) * 2017-08-21 2017-11-24 时瑞科技(深圳)有限公司 A kind of Internet of things system
CN109473095B (en) * 2017-09-08 2020-01-10 北京君林科技股份有限公司 Intelligent household control system and control method
CN109473095A (en) * 2017-09-08 2019-03-15 北京君林科技股份有限公司 A kind of intelligent home control system and control method
CN107682240A (en) * 2017-09-27 2018-02-09 四川长虹电器股份有限公司 A kind of distributed sound interactive system for intelligent domestic
WO2019071989A1 (en) * 2017-10-13 2019-04-18 歌尔股份有限公司 Smart device speech enhancement method and device and smart device
CN107993660A (en) * 2017-12-26 2018-05-04 江苏可美智能科技股份有限公司 Speech control system for Internet of Things intelligence control system
CN108229391A (en) * 2018-01-02 2018-06-29 京东方科技集团股份有限公司 Gesture identifying device and its server, gesture recognition system, gesture identification method
CN108154140A (en) * 2018-01-22 2018-06-12 北京百度网讯科技有限公司 Voice awakening method, device, equipment and computer-readable medium based on lip reading
CN108563208A (en) * 2018-06-28 2018-09-21 马雷明 Intelligent domestic system and its control method
CN109168110A (en) * 2018-09-29 2019-01-08 芜湖星途机器人科技有限公司 External hanging type speech packet
CN109036430A (en) * 2018-09-29 2018-12-18 芜湖星途机器人科技有限公司 Voice control terminal
CN109151393A (en) * 2018-10-09 2019-01-04 深圳市亿联智能有限公司 A kind of sound fixation and recognition method for detecting
CN109803013A (en) * 2019-01-21 2019-05-24 浙江大学 A kind of weak interactive system and its control method based on artificial intelligence
CN109884908A (en) * 2019-03-14 2019-06-14 苏州宏裕千智能设备科技有限公司 Cloud platform, apparatus control method and system, readable storage medium storing program for executing

Similar Documents

Publication Publication Date Title
US9691378B1 (en) Methods and devices for selectively ignoring captured audio data
KR101915575B1 (en) Intelligent assistant for home automation
JP6522503B2 (en) Device control method, display control method and purchase settlement method
EP3077921B1 (en) Natural language control of secondary device
US10031721B2 (en) System and method for processing control commands in a voice interactive system
CN103730116B (en) Intelligent watch realizes the system and method that intelligent home device controls
US10628714B2 (en) Entity-tracking computing system
US10121473B2 (en) System and method for determining recipient of spoken command in a control system
US9685171B1 (en) Multiple-stage adaptive filtering of audio signals
CN106415719B (en) It is indicated using the steady endpoint of the voice signal of speaker identification
TWI544778B (en) Remote doorbell control system and its samrt doorbell device
US9576591B2 (en) Electronic apparatus and control method of the same
CN102903362B (en) Integrated this locality and the speech recognition based on cloud
TWI478049B (en) Intelligent switch with voice control function and intelligent control system using the same
EP2932371B1 (en) Response endpoint selection
JP2017076393A (en) Apparatus and method for processing control command based on voice agent, and agent device
KR101737191B1 (en) Method and apparatus for controlling smart terminal
US20170163435A1 (en) Smart home automation systems and methods
US8321885B2 (en) In-home system monitoring method and system
KR100413622B1 (en) Voice control system for operating home electrical appliances
JP6690031B2 (en) System control method, system, and program
CN104049721B (en) Information processing method and electronic equipment
CN105446162B (en) A kind of intelligent home furnishing control method of smart home system and robot
KR20140097365A (en) Audio pattern matching for device activation
US10056081B2 (en) Control method, controller, and non-transitory recording medium

Legal Events

Date Code Title Description
PB01 Publication
C06 Publication
WD01 Invention patent application deemed withdrawn after publication

Application publication date: 20151111

WD01 Invention patent application deemed withdrawn after publication