WO2003044774A1 - Audio-based method and apparatus for controlling operation of an appliance - Google Patents
Audio-based method and apparatus for controlling operation of an appliance Download PDFInfo
- Publication number
- WO2003044774A1 WO2003044774A1 PCT/IB2002/004586 IB0204586W WO03044774A1 WO 2003044774 A1 WO2003044774 A1 WO 2003044774A1 IB 0204586 W IB0204586 W IB 0204586W WO 03044774 A1 WO03044774 A1 WO 03044774A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- appliance
- audio characteristic
- adjustment
- predefined audio
- predefined
- Prior art date
Links
Classifications
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
- G10L15/00—Speech recognition
- G10L15/22—Procedures used during a speech recognition process, e.g. man-machine dialogue
Definitions
- the present invention relates to methods and apparatus for controlling appliances and other devices, such as food processors, mixers and microwave ovens, and more particularly, to an audio-based method and apparatus for automatically detecting when a task performed by an appliance has been completed based on specified sound patterns and to automatically trigger a subsequent step (if any) or terminate the operation of an appliance.
- the marketplace offers a number of consumer appliances, such as microwave ovens, food processors, mixers and blenders, that provide an ever-growing number of features intended to increase the convenience and capabilities of these appliances.
- Most appliances for example, have a one-touch feature that allows the appliance to be activated in a desired mode with the press of only a single button. Thereafter, the appliance can typically operate in the desired mode in an unattended manner until the human operator deactivates the appliance, allowing the human operator to perform other tasks while the appliance performs the desired primary task.
- a method and apparatus for monitoring audio activity associated with an appliance and automatically adjusting the appliance in response to predefined audio features.
- the disclosed appliance controller includes one or more microphones focused on one or more appliances.
- a number of appliance adjustment rules define various audio features that suggest when an appliance setting should be adjusted.
- Each rule contains one or more specified audio features and a corresponding appliance adjustment that should be performed when the rule is satisfied.
- the detected audio feature may be a static audio characteristic, such as a specified pitch or volume, or a time varying audio characteristic, such as a specified pitch or volume over an interval of time.
- the appliance adjustment rules may be predefined or dynamically determined by observing user behavior during a learning phase.
- a classifier learns distinguishing audio features hen a user makes a manual adjustment.
- the appliance is automatically adjusted when the observed audio features are again detected.
- FIG. 1 illustrates an appliance controller in accordance with the present invention
- Fig. 2 illustrates a sample table from the appliance adjustment rules database of Fig. 1 in accordance with the present invention
- Fig. 3 illustrates an exemplary appliance learning process embodying principles of the present invention
- Fig. 4 is a flow chart describing an exemplary appliance monitoring process embodying principles of the present invention.
- Fig. 1 illustrates an appliance controller 100 in accordance with the present invention.
- the appliance controller 100 includes one or more microphones 150-1 through 150-N (hereinafter, collectively referred to as microphones 150) that are focused on one or more appliances 160, such as a microwave oven, food processor, mixer or blender.
- the audio information generated by the microphones 150 is processed by the appliance controller 100, in a manner discussed below in conjunction with Fig. 4, to identify one or more predefined audio events suggesting that the appliance(s) should be adjusted.
- the present invention employs an appliance adjustment rules database 200, discussed further below in conjunction with Fig. 2, that identifies a number of audio characteristics that should initiate the adjustment of the appliance 160 in a specified manner.
- each appliance adjustment rule contains one or more audio features that must be observed in order for the rule to be triggered, and a corresponding appliance adjustment that should be performed by the appliance controller 100 when the predefined criteria for initiating the appliance adjustment rule is satisfied.
- At least one of the criteria for each rule is an audio feature detected in the audio information generated by the microphones 150.
- the detected audio feature may be a static audio characteristic, such as a specified pitch or volume, or a time varying audio characteristic, such as a specified pitch or volume over an interval of time.
- the appliance controller 100 Upon detection of a predefined audio feature, the appliance controller 100 automatically adjusts the appliance in a specified manner. As discussed below in conjunction with Figs.
- the exemplary appliance adjustment rales recorded in the audio adjustment rales database 200 may include a number of default rules that have been installed, for example, during a manufacturing or upgrade process, and optionally, additional rules that are dynamically learned by the appliance controller 100 based on user activity.
- the appliance controller 100 includes an appliance learning process 300 and an appliance monitoring process 400.
- the appliance learning process 300 observes the operation of one or more appliances 160 to learn a set of appliance adjustment rales that define when a given appliance 160 should be adjusted based on observed audio characteristics.
- the appliance monitoring process 400 processes the audio information obtained by the microphones 150 and detects one or more predefined actions that should trigger the adjustment of an appliance 160.
- the appliance controller 100 may be embodied as any computing device, such as a personal computer or workstation, that contains a processor 120, such as a central processing unit (CPU), and memory 110, such as RAM and/or ROM.
- a processor 120 such as a central processing unit (CPU)
- memory 110 such as RAM and/or ROM.
- the appliance controller 100 maybe embodied as an application specific integrated circuit (ASIC) (not shown) that is included, for example, in an appliance.
- ASIC application specific integrated circuit
- Fig. 2 illustrates an exemplary table of the audio adjustment rales database 200 that records various rales for one or more appliances.
- Each rale in the audio adjustment rales database 200 identifies the corresponding appliance and includes predefined criteria specifying the conditions under which the rale should be initiated, as well as a corresponding action item that should be triggered when the criteria associated with the rule is satisfied.
- the action item identifies the manner in which the appliance(s) 160 should be adjusted when the rule is triggered.
- the audio adjustment rales database 200 is comprised of a plurality of records, such as records 205-210, each associated with a different appliance adjustment rule. For each rule, the audio adjustment rules database 200 identifies the corresponding appliance in field 250 (and possibly a particular task performed by the appliance, such as mixing bread dough), the corresponding rale criteria in field 260 and the corresponding action in field 270.
- the exemplary appliance adjustment rule set forth in record 206 defines an audio characteristic (a specified sound pitch) that should trigger turning a mixer to a high speed for a specified interval.
- the exemplary appliance adjustment rale set forth in record 208 defines an audio characteristic (an absence of sound for a specified time interval when popcorn is popping) that should trigger turning off the microwave oven.
- the appliance adjustment rules recorded in the audio adjustment rales database 200 may include a number of default rales that have been pre- installed, for example, during the manufacturing process, and optionally, additional rales that are dynamically learned by the appliance controller 100 based on user activity.
- Fig. 3 illustrates an exemplary appliance learning process 300 that may be employed in an exemplary embodiment to generate appliance adjustment rules.
- the appliance learning process 300 observes the operation of one or more appliances 160 to learn a set of appliance adjustment rules that define when a given appliance 160 should be adjusted based on observed audio characteristics.
- the microphones 150 capture an audio signal that includes audio activity associated with the operation of an appliance 160. Thereafter, a window-based feature extraction is performed at stage 310 to generate audio feature intervals, such as 10 millisecond intervals.
- a classifier 330 processes the audio features to correlate audio activity with manual adjustments by a user during a learning phase. In this manner, the classifier 330 generates appliance adjustment rales that are recorded in the audio adjustment rales database 200. Generally, the classifier 330 learns distinguishing audio features when a user makes a manual adjustment that can later be applied to automatically make the same adjustment to an appliance 160 when the observed audio features are again detected during operation of the appliance 160.
- the classifier 330 maybe embodied, for example, as a Bayesian classifier or a decision tree (DT) classifier, such as those described in United States Patent Application Serial No. 10/219,412, filed August 15, 2002, entitled "CLASSIFIERS USING EIGEN
- Fig. 4 is a flow chart describing an exemplary appliance monitoring process 400.
- the appliance monitoring process 400 processes the audio information obtained by the microphones 150 and detects one or more predefined actions that should trigger the adjustment of an appliance 160.
- the exemplary appliance monitoring process 400 is a general process illustrating the broad concepts of the present invention.
- the appliance monitoring process 400 initially obtains one or more inputs from the microphones 150 during step 410. Thereafter, the appliance monitoring process 400 analyzes the audio information during step 420 using audio content analysis techniques.
- audio content analysis techniques see, for example, Silvia Pfeiffer et al., "Automatic Audio Content Analysis,” Proc. ACM Multimedia 96, 21-40, Boston, MA. (Nov. 1996), or Dongge Li et al., "Classification of General Audio Data for Content-Based Retrieval," Pattern Recognition Letters 22, 533-44 (2001), each incorporated by reference herein.
- the audio content analysis is employed to recognize various features in the audio signals obtained by the microphones 150.
- a test is performed during step 430 to determine if the audio content analysis detects a predefined event, as defined in the audio adjustment rales database 200. If it is determined during step 430 that the audio content analysis does not detect a predefined event, then program control returns to step 420 to continue monitoring the audio information in the manner discussed above.
- step 430 If, however, it is determined during step 430 that the audio content analysis does detect a predefined audio event, then the audio event is processed during step 440 as indicated in field 270 of the audio adjustment rales database 200 for the identified appliance. Program control then terminates (or returns to step 410 and continues monitoring user activities in the manner discussed above).
Abstract
Description
Claims
Priority Applications (3)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
EP02777724A EP1449202A1 (en) | 2001-11-19 | 2002-10-29 | Audio-based method and apparatus for controlling operation of an appliance |
AU2002339676A AU2002339676A1 (en) | 2001-11-19 | 2002-10-29 | Audio-based method and apparatus for controlling operation of an appliance |
JP2003546330A JP2005509988A (en) | 2001-11-19 | 2002-10-29 | Acoustic-based method and apparatus for controlling the operation of electrical products |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US09/988,942 | 2001-11-19 | ||
US09/988,942 US20030095673A1 (en) | 2001-11-19 | 2001-11-19 | Audio-based method and apparatus for controlling operation of an appliance |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2003044774A1 true WO2003044774A1 (en) | 2003-05-30 |
Family
ID=25534616
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/IB2002/004586 WO2003044774A1 (en) | 2001-11-19 | 2002-10-29 | Audio-based method and apparatus for controlling operation of an appliance |
Country Status (6)
Country | Link |
---|---|
US (1) | US20030095673A1 (en) |
EP (1) | EP1449202A1 (en) |
JP (1) | JP2005509988A (en) |
CN (1) | CN1589465A (en) |
AU (1) | AU2002339676A1 (en) |
WO (1) | WO2003044774A1 (en) |
Families Citing this family (8)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US7181400B2 (en) * | 2001-04-20 | 2007-02-20 | Intel Corporation | Method and apparatus to provision a network appliance |
CN102436813A (en) * | 2011-12-21 | 2012-05-02 | 成都众询科技有限公司 | Controllable voice control system |
WO2016073573A1 (en) | 2014-11-04 | 2016-05-12 | Board Of Regents, The University Of Texas System | Method and apparatus for controlling coffee bean roasting |
TWI672102B (en) * | 2017-11-16 | 2019-09-21 | 志勇無限創意有限公司 | Assisting appatatus for bean roasting and bean roasting appatatus |
US10249293B1 (en) * | 2018-06-11 | 2019-04-02 | Capital One Services, Llc | Listening devices for obtaining metrics from ambient noise |
US11382349B2 (en) * | 2019-07-18 | 2022-07-12 | Grand Mate Co., Ltd. | Coffee bean roaster |
DE102019124230B4 (en) | 2019-09-10 | 2023-01-12 | Infineon Technologies Ag | ELECTRONIC SHUTDOWN DEVICE AND METHOD OF SHUTDOWN AN EQUIPMENT |
US11410676B2 (en) * | 2020-11-18 | 2022-08-09 | Haier Us Appliance Solutions, Inc. | Sound monitoring and user assistance methods for a microwave oven |
Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
GB1511527A (en) * | 1974-08-08 | 1978-05-24 | Gillet Guilliet Rennepont Comp | Automatic control of sawing machines |
US4574633A (en) * | 1983-02-04 | 1986-03-11 | Citizen Watch Company Limited | Apparatus for detecting tool damage in automatically controlled machine tool |
DE19631579A1 (en) * | 1996-08-05 | 1998-02-12 | Vhf Computer Gmbh | Acoustic emergency shutdown apparatus for machines |
Family Cites Families (8)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US4602147A (en) * | 1983-11-28 | 1986-07-22 | Gell Harold A | Roasting timer |
US4952766A (en) * | 1987-10-26 | 1990-08-28 | Microwave Products Of America, Inc. | Sensor and pop detector for microwave popcorn control |
US4869233A (en) * | 1987-11-30 | 1989-09-26 | Gas Research Institute | Boiling condition detector |
US4996403A (en) * | 1990-02-05 | 1991-02-26 | The United States Of America As Represented By The United States Department Of Energy | Acoustic emission feedback control for control of boiling in a microwave oven |
KR940004051B1 (en) * | 1991-10-12 | 1994-05-11 | 주식회사 금성사 | Heating method of sonic device sensor |
US5493618A (en) * | 1993-05-07 | 1996-02-20 | Joseph Enterprises | Method and apparatus for activating switches in response to different acoustic signals |
US6118104A (en) * | 1999-03-19 | 2000-09-12 | General Electric Company | Method and apparatus for boil state detection based on acoustic signal features |
US6236025B1 (en) * | 1999-03-19 | 2001-05-22 | General Electric Company | Acoustic sensing system for boil state detection and method for determining boil state |
-
2001
- 2001-11-19 US US09/988,942 patent/US20030095673A1/en not_active Abandoned
-
2002
- 2002-10-29 CN CNA028228812A patent/CN1589465A/en active Pending
- 2002-10-29 WO PCT/IB2002/004586 patent/WO2003044774A1/en not_active Application Discontinuation
- 2002-10-29 AU AU2002339676A patent/AU2002339676A1/en not_active Abandoned
- 2002-10-29 JP JP2003546330A patent/JP2005509988A/en not_active Withdrawn
- 2002-10-29 EP EP02777724A patent/EP1449202A1/en not_active Withdrawn
Patent Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
GB1511527A (en) * | 1974-08-08 | 1978-05-24 | Gillet Guilliet Rennepont Comp | Automatic control of sawing machines |
US4574633A (en) * | 1983-02-04 | 1986-03-11 | Citizen Watch Company Limited | Apparatus for detecting tool damage in automatically controlled machine tool |
DE19631579A1 (en) * | 1996-08-05 | 1998-02-12 | Vhf Computer Gmbh | Acoustic emergency shutdown apparatus for machines |
Also Published As
Publication number | Publication date |
---|---|
JP2005509988A (en) | 2005-04-14 |
EP1449202A1 (en) | 2004-08-25 |
CN1589465A (en) | 2005-03-02 |
AU2002339676A1 (en) | 2003-06-10 |
US20030095673A1 (en) | 2003-05-22 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US6933685B2 (en) | Method and apparatus for controlling lighting based on user behavior | |
US10573171B2 (en) | Method of associating user input with a device | |
KR101907182B1 (en) | Application program control method and related apparatus | |
WO2006004890A2 (en) | Computer controlled electromyogram control system | |
US20030095673A1 (en) | Audio-based method and apparatus for controlling operation of an appliance | |
US20020174426A1 (en) | Method and apparatus for activating a media player based on user behavior | |
CN105425643A (en) | Cooking control method and device | |
US10284896B2 (en) | Smart TV controlling method and smart TV thereof | |
WO2002080531A2 (en) | Method and apparatus for controlling a media player based on user activity | |
CN116030812B (en) | Intelligent interconnection voice control method, device, equipment and medium for gas stove | |
CN110004659B (en) | Laundry treating apparatus and control method thereof | |
CN109885232A (en) | The control method of data input device, apparatus and system | |
WO2008155427A2 (en) | Fully learning classification system and method for hearing aids | |
CN110850749B (en) | Control method of cooking apparatus, and computer-readable storage medium | |
CN112596663A (en) | Method and device for adjusting played content, computer equipment and storage medium | |
CN114568948A (en) | Cooking control method, device and system | |
JP7349642B2 (en) | Presentation system that accepts turning off notifications | |
JP2020184106A (en) | Controller, household appliance, communication device, server, information presentation system, control program and control method | |
CN112702482B (en) | Method and device for adjusting focal length of camera | |
WO2024041796A1 (en) | Method and control unit for locking a user interface of a household appliance | |
US20210158682A1 (en) | Information notification system and information notification method | |
CN116413476A (en) | Sensor assembly configuration method, terminal, control system and readable storage device | |
CN110786745A (en) | Cooking control method, device and equipment and cooking system | |
CN115664877A (en) | Equipment control method, device, control equipment and medium | |
CN116048366A (en) | Control method and electronic equipment |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AK | Designated states |
Kind code of ref document: A1 Designated state(s): AE AG AL AM AT AU AZ BA BB BG BR BY BZ CA CH CN CO CR CU CZ DE DK DM DZ EC EE ES FI GB GD GE GH GM HR HU ID IL IN IS JP KE KG KP KR KZ LC LK LR LS LT LU LV MA MD MG MK MN MW MX MZ NO NZ OM PH PL PT RO RU SD SE SG SI SK SL TJ TM TN TR TT TZ UA UG UZ VC VN YU ZA ZM ZW |
|
AL | Designated countries for regional patents |
Kind code of ref document: A1 Designated state(s): GH GM KE LS MW MZ SD SL SZ TZ UG ZM ZW AM AZ BY KG KZ MD RU TJ TM AT BE BG CH CY CZ DE DK EE ES FI FR GB GR IE IT LU MC NL PT SE SK TR BF BJ CF CG CI CM GA GN GQ GW ML MR NE SN TD TG |
|
121 | Ep: the epo has been informed by wipo that ep was designated in this application | ||
WWE | Wipo information: entry into national phase |
Ref document number: 2002777724 Country of ref document: EP |
|
WWE | Wipo information: entry into national phase |
Ref document number: 2003546330 Country of ref document: JP |
|
WWE | Wipo information: entry into national phase |
Ref document number: 20028228812 Country of ref document: CN |
|
WWP | Wipo information: published in national office |
Ref document number: 2002777724 Country of ref document: EP |
|
WWW | Wipo information: withdrawn in national office |
Ref document number: 2002777724 Country of ref document: EP |