CN115398057B - System and method for monitoring operation of washing machine device using sound - Google Patents
System and method for monitoring operation of washing machine device using sound Download PDFInfo
- Publication number
- CN115398057B CN115398057B CN202180019122.3A CN202180019122A CN115398057B CN 115398057 B CN115398057 B CN 115398057B CN 202180019122 A CN202180019122 A CN 202180019122A CN 115398057 B CN115398057 B CN 115398057B
- Authority
- CN
- China
- Prior art keywords
- sound
- washing machine
- spectrogram
- machine apparatus
- washing
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000005406 washing Methods 0.000 title claims abstract description 106
- 238000012544 monitoring process Methods 0.000 title claims abstract description 11
- 238000000034 method Methods 0.000 title claims description 34
- 230000005236 sound signal Effects 0.000 claims abstract description 18
- 238000013473 artificial intelligence Methods 0.000 claims abstract description 5
- 238000012790 confirmation Methods 0.000 claims abstract description 5
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 claims description 16
- 238000013019 agitation Methods 0.000 claims description 8
- 238000013527 convolutional neural network Methods 0.000 claims description 8
- 239000000725 suspension Substances 0.000 claims description 2
- 238000010200 validation analysis Methods 0.000 claims description 2
- 239000012530 fluid Substances 0.000 description 30
- 238000004891 communication Methods 0.000 description 19
- 239000000654 additive Substances 0.000 description 7
- 239000012459 cleaning agent Substances 0.000 description 6
- 238000013528 artificial neural network Methods 0.000 description 5
- 238000012545 processing Methods 0.000 description 5
- 230000009471 action Effects 0.000 description 4
- 230000000996 additive effect Effects 0.000 description 4
- 238000004140 cleaning Methods 0.000 description 4
- 239000003599 detergent Substances 0.000 description 4
- 238000001035 drying Methods 0.000 description 4
- 239000007844 bleaching agent Substances 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 230000001413 cellular effect Effects 0.000 description 2
- 230000005284 excitation Effects 0.000 description 2
- 239000002979 fabric softener Substances 0.000 description 2
- 238000013508 migration Methods 0.000 description 2
- 230000005012 migration Effects 0.000 description 2
- 230000004044 response Effects 0.000 description 2
- 238000001228 spectrum Methods 0.000 description 2
- ORILYTVJVMAKLC-UHFFFAOYSA-N Adamantane Natural products C1C(C2)CC3CC1CC2C3 ORILYTVJVMAKLC-UHFFFAOYSA-N 0.000 description 1
- 230000004913 activation Effects 0.000 description 1
- 230000003190 augmentative effect Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 230000030808 detection of mechanical stimulus involved in sensory perception of sound Effects 0.000 description 1
- 238000002405 diagnostic procedure Methods 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 239000013505 freshwater Substances 0.000 description 1
- 230000005484 gravity Effects 0.000 description 1
- 239000007788 liquid Substances 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 239000000203 mixture Substances 0.000 description 1
- 238000003062 neural network model Methods 0.000 description 1
- 238000011022 operating instruction Methods 0.000 description 1
- 239000000843 powder Substances 0.000 description 1
- 230000001105 regulatory effect Effects 0.000 description 1
- 239000008400 supply water Substances 0.000 description 1
- 238000013526 transfer learning Methods 0.000 description 1
- 230000001960 triggered effect Effects 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Classifications
-
- D—TEXTILES; PAPER
- D06—TREATMENT OF TEXTILES OR THE LIKE; LAUNDERING; FLEXIBLE MATERIALS NOT OTHERWISE PROVIDED FOR
- D06F—LAUNDERING, DRYING, IRONING, PRESSING OR FOLDING TEXTILE ARTICLES
- D06F34/00—Details of control systems for washing machines, washer-dryers or laundry dryers
- D06F34/14—Arrangements for detecting or measuring specific parameters
-
- D—TEXTILES; PAPER
- D06—TREATMENT OF TEXTILES OR THE LIKE; LAUNDERING; FLEXIBLE MATERIALS NOT OTHERWISE PROVIDED FOR
- D06F—LAUNDERING, DRYING, IRONING, PRESSING OR FOLDING TEXTILE ARTICLES
- D06F33/00—Control of operations performed in washing machines or washer-dryers
- D06F33/30—Control of washing machines characterised by the purpose or target of the control
- D06F33/32—Control of operational steps, e.g. optimisation or improvement of operational steps depending on the condition of the laundry
-
- D—TEXTILES; PAPER
- D06—TREATMENT OF TEXTILES OR THE LIKE; LAUNDERING; FLEXIBLE MATERIALS NOT OTHERWISE PROVIDED FOR
- D06F—LAUNDERING, DRYING, IRONING, PRESSING OR FOLDING TEXTILE ARTICLES
- D06F33/00—Control of operations performed in washing machines or washer-dryers
- D06F33/30—Control of washing machines characterised by the purpose or target of the control
- D06F33/32—Control of operational steps, e.g. optimisation or improvement of operational steps depending on the condition of the laundry
- D06F33/34—Control of operational steps, e.g. optimisation or improvement of operational steps depending on the condition of the laundry of water filling
-
- D—TEXTILES; PAPER
- D06—TREATMENT OF TEXTILES OR THE LIKE; LAUNDERING; FLEXIBLE MATERIALS NOT OTHERWISE PROVIDED FOR
- D06F—LAUNDERING, DRYING, IRONING, PRESSING OR FOLDING TEXTILE ARTICLES
- D06F33/00—Control of operations performed in washing machines or washer-dryers
- D06F33/30—Control of washing machines characterised by the purpose or target of the control
- D06F33/32—Control of operational steps, e.g. optimisation or improvement of operational steps depending on the condition of the laundry
- D06F33/40—Control of operational steps, e.g. optimisation or improvement of operational steps depending on the condition of the laundry of centrifugal separation of water from the laundry
-
- D—TEXTILES; PAPER
- D06—TREATMENT OF TEXTILES OR THE LIKE; LAUNDERING; FLEXIBLE MATERIALS NOT OTHERWISE PROVIDED FOR
- D06F—LAUNDERING, DRYING, IRONING, PRESSING OR FOLDING TEXTILE ARTICLES
- D06F33/00—Control of operations performed in washing machines or washer-dryers
- D06F33/30—Control of washing machines characterised by the purpose or target of the control
- D06F33/44—Control of the operating time, e.g. reduction of overall operating time
-
- D—TEXTILES; PAPER
- D06—TREATMENT OF TEXTILES OR THE LIKE; LAUNDERING; FLEXIBLE MATERIALS NOT OTHERWISE PROVIDED FOR
- D06F—LAUNDERING, DRYING, IRONING, PRESSING OR FOLDING TEXTILE ARTICLES
- D06F2103/00—Parameters monitored or detected for the control of domestic laundry washing machines, washer-dryers or laundry dryers
-
- D—TEXTILES; PAPER
- D06—TREATMENT OF TEXTILES OR THE LIKE; LAUNDERING; FLEXIBLE MATERIALS NOT OTHERWISE PROVIDED FOR
- D06F—LAUNDERING, DRYING, IRONING, PRESSING OR FOLDING TEXTILE ARTICLES
- D06F2103/00—Parameters monitored or detected for the control of domestic laundry washing machines, washer-dryers or laundry dryers
- D06F2103/18—Washing liquid level
-
- D—TEXTILES; PAPER
- D06—TREATMENT OF TEXTILES OR THE LIKE; LAUNDERING; FLEXIBLE MATERIALS NOT OTHERWISE PROVIDED FOR
- D06F—LAUNDERING, DRYING, IRONING, PRESSING OR FOLDING TEXTILE ARTICLES
- D06F2103/00—Parameters monitored or detected for the control of domestic laundry washing machines, washer-dryers or laundry dryers
- D06F2103/24—Spin speed; Drum movements
-
- D—TEXTILES; PAPER
- D06—TREATMENT OF TEXTILES OR THE LIKE; LAUNDERING; FLEXIBLE MATERIALS NOT OTHERWISE PROVIDED FOR
- D06F—LAUNDERING, DRYING, IRONING, PRESSING OR FOLDING TEXTILE ARTICLES
- D06F2103/00—Parameters monitored or detected for the control of domestic laundry washing machines, washer-dryers or laundry dryers
- D06F2103/26—Imbalance; Noise level
-
- D—TEXTILES; PAPER
- D06—TREATMENT OF TEXTILES OR THE LIKE; LAUNDERING; FLEXIBLE MATERIALS NOT OTHERWISE PROVIDED FOR
- D06F—LAUNDERING, DRYING, IRONING, PRESSING OR FOLDING TEXTILE ARTICLES
- D06F2105/00—Systems or parameters controlled or affected by the control systems of washing machines, washer-dryers or laundry dryers
- D06F2105/52—Changing sequence of operational steps; Carrying out additional operational steps; Modifying operational steps, e.g. by extending duration of steps
-
- D—TEXTILES; PAPER
- D06—TREATMENT OF TEXTILES OR THE LIKE; LAUNDERING; FLEXIBLE MATERIALS NOT OTHERWISE PROVIDED FOR
- D06F—LAUNDERING, DRYING, IRONING, PRESSING OR FOLDING TEXTILE ARTICLES
- D06F34/00—Details of control systems for washing machines, washer-dryers or laundry dryers
- D06F34/04—Signal transfer or data transmission arrangements
Landscapes
- Engineering & Computer Science (AREA)
- Textile Engineering (AREA)
- Control Of Washing Machine And Dryer (AREA)
Abstract
A washing machine apparatus includes a microphone for monitoring sound generated during operation of the washing machine apparatus and a controller operatively coupled to the microphone. The controller is configured to obtain a sound signal generated during operation of the washing machine device and to convert the sound signal into a spectrogram representing a sound amplitude and a sound frequency that vary over time. The spectrogram is analyzed using an artificial intelligence image recognition program to confirm one or more sound signatures associated with particular operating conditions, and operation of the washing machine device is adjusted based at least in part on the confirmation of the sound signatures.
Description
Technical Field
The present subject matter relates generally to washing machine devices, or more particularly, to systems and methods for monitoring sounds within a washing machine device and analyzing the sounds to confirm a sound signature associated with a particular event.
Background
Washing machine devices typically include a tub for holding water or a washing fluid (e.g., water and a cleaning agent, bleach, and/or other washing additives). A basket is rotatably mounted within the tub and defines a washing chamber for receiving articles to be washed. During normal operation of such a washing machine device, washing fluid is directed into the tub and onto the items within the washing chamber of the basket. The basket or agitation member can be rotated at various speeds to agitate the articles within the washing chamber, to spin washing fluid from the articles within the washing chamber, and so forth. The drain pump assembly is operable to drain water from within the sump during a spin or drain cycle.
Note that it is often desirable to monitor sounds produced by the washing machine equipment during operation, for example to confirm unexpected objects in the washing load, diagnose mechanical faults, or detect other operating conditions. However, conventional washing machines lack any acoustic feedback system. Some washing machines can monitor sound and provide notification when the sound exceeds a certain threshold, but such systems have limited utility and effectiveness.
Accordingly, there is a need for a washing machine apparatus having features for improved operation. More particularly, a system and method for monitoring sounds produced by a washing machine device and validating sound signatures associated with particular operating conditions would be particularly beneficial.
Disclosure of Invention
Advantages of the invention will be set forth in part in the description which follows, or may be obvious from the description, or may be learned by practice of the invention.
According to an exemplary embodiment of the present disclosure, there is provided a washing machine apparatus including: a washing tub positioned within the cabinet and defining a washing chamber; a basket rotatably mounted within the tub and configured to receive a quantity of articles to be washed; and a motor operatively coupled to the basket to selectively rotate the basket. A microphone is provided to monitor sound generated during operation of the washing machine device, and a controller is operatively coupled to the microphone. The controller is configured to obtain a sound signal generated during operation of the washing machine device using the microphone, generate a spectrogram from the sound signal, the spectrogram representing a sound amplitude and a sound frequency over time, confirm a sound signature by analyzing the spectrogram using the image recognition program, and adjust at least one operating parameter of the washing machine device based at least in part on the confirmation of the sound signature.
According to another exemplary embodiment of the present disclosure, a method of operating a laundry machine apparatus is provided. The washing machine apparatus includes a basket rotatably mounted within a washing tub, a motor operatively coupled to the basket to selectively rotate the basket, and a microphone for monitoring sounds produced by the washing machine apparatus. The method includes obtaining a sound signal generated during operation of the washing machine device using a microphone, generating a spectrogram from the sound signal, the spectrogram representing a sound amplitude and a sound frequency over time, validating a sound signature by analyzing the spectrogram using an image recognition program, and adjusting at least one operating parameter of the washing machine device based at least in part on the validation of the sound signature.
According to another exemplary embodiment of the present disclosure, a device is provided that includes a microphone for monitoring sound generated during operation of the device and a controller operatively coupled to the microphone. The controller is configured to obtain a sound signal generated during operation of the device using the microphone, generate a spectrogram from the sound signal, the spectrogram representing a sound amplitude and a sound frequency over time, confirm a sound signature by analyzing the spectrogram using image processing techniques, and adjust at least one operating parameter of the device based at least in part on the confirmation of the sound signature.
These and other features, aspects, and advantages of the present invention will become better understood with reference to the following description and appended claims. The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments of the invention and together with the description, serve to explain the principles of the invention.
Drawings
A full and enabling disclosure of the present invention, including the best mode thereof to one of ordinary skill in the art, is set forth in the specification, which makes reference to the appended figures.
Fig. 1 provides a perspective view of an exemplary laundry machine apparatus according to an exemplary embodiment of the present subject matter.
Fig. 2 provides a side cross-sectional view of the exemplary laundry machine apparatus of fig. 1.
Fig. 3 illustrates a method of confirming an operation condition using sound generated by a washing machine device according to one embodiment of the present disclosure.
FIG. 4 provides an exemplary spectrogram in accordance with an exemplary embodiment of the present subject matter.
Repeat use of reference characters in the present specification and drawings is intended to represent same or analogous features or elements of the invention.
Detailed Description
Reference now will be made in detail to embodiments of the invention, one or more examples of which are illustrated in the drawings. The examples are provided to illustrate the invention and not to limit the invention. Indeed, it will be apparent to those skilled in the art that various modifications and variations can be made in the present invention without departing from the scope or spirit of the invention. For instance, features illustrated or described as part of one embodiment, can be used on another embodiment to yield a still further embodiment. Accordingly, it is intended that the present invention cover the modifications and variations of this invention provided they come within the scope of the appended claims and their equivalents.
Referring now to the drawings, FIG. 1 is a perspective view of an exemplary horizontal axis washing machine apparatus 100, and FIG. 2 is a side cross-sectional view of the washing machine apparatus 100. As shown, the washing machine apparatus 100 generally defines a vertical direction V, a lateral direction L, and a depth direction T, each of which are perpendicular to each other such that generally an orthogonal coordinate system is defined. The washing machine apparatus 100 includes a cabinet 102 extending in a vertical direction V between a top 104 and a bottom 106, in a lateral direction between a left side 108 and a right side 110, and in a depth direction T between a front 112 and a rear 114.
Referring to fig. 2, the basket 120 is rotatably installed in the cabinet 102 such that it can rotate about the rotation axis a. A motor 122 (e.g., a flat motor, etc.) is in mechanical communication with the basket 120 to selectively rotate the basket 120 (e.g., during a agitation or rinse cycle of the laundry machine apparatus 100). The basket 120 is received within the wash tub 124 and defines a wash chamber 126 configured to receive items to be washed. The wash tub 124 holds wash fluid and rinse fluid for agitation in the wash basket 120 within the wash tub 124. As used herein, "wash fluid" may refer to water, a cleaning agent, a fabric softener, a bleach, or any other suitable wash additive or combination thereof. Indeed, for simplicity of discussion, these terms may be used interchangeably herein without limiting the subject matter to any particular "wash fluid":
the basket 120 may define one or more agitator features that extend into the washing chamber 126 to assist in agitating and cleaning items disposed within the washing chamber 126 during operation of the washing machine apparatus 100. For example, as shown in FIG. 2, a plurality of ribs 128 extend from the basket 120 into the washing chamber 126. In this manner, for example, the ribs 128 may lift items disposed in the basket 120 during rotation of the basket 120.
Referring generally to fig. 1 and 2, the chassis 102 further includes a front panel 130 defining an opening 132 that allows a user to access the basket 120 within the tub 124. More specifically, the washing machine apparatus 100 includes a door 134 positioned to cover the opening 132 and rotatably mounted to the front panel 130. In this manner, the door 134 allows selective access to the opening 132 by being movable between an open position (not shown) that facilitates access to the wash tub 124 and a closed position (FIG. 1) that prevents access to the wash tub 124.
A window 136 in the door 134 allows viewing of the basket 120 when the door 134 is in a closed position, such as during operation of the washing machine apparatus 100. The door 134 also includes a handle (not shown) that may be pulled by a user, for example, when opening and closing the door 134. Further, although the door 134 is shown mounted to the front panel 130, it should be understood that the door 134 may be mounted to the other side of the chassis 102 or any other suitable support according to alternative embodiments.
Referring again to FIG. 2, the basket 120 also defines a plurality of perforations 140 to facilitate fluid communication between the interior of the basket 120 and the wash tub 124. The sump 142 is defined by the washing tub 124 at the bottom of the washing tub 124 in the vertical direction V. Accordingly, the sump 142 is configured to receive and substantially collect wash fluid during operation of the washing machine apparatus 100. For example, during operation of the washing machine apparatus 100, the washing fluid may be urged from the basket 120 through the plurality of perforations 140 to the sump 142 by gravity.
A drain pump assembly 144 is positioned below the wash tub 124 and is in fluid communication with the sump 142 for periodically draining dirty wash fluid from the washing machine apparatus 100. The drain pump assembly 144 may generally include a drain pump 146 in fluid communication with an external drain 148 and with the sump 142 via a drain hose 150. During the drain cycle, the drain pump 146 forces the wash fluid to flow from the sump 142 through the drain hose 150 to the external drain 148. More specifically, the drain pump 146 includes a motor (not shown) that is energized during a drain cycle such that the drain pump 146 draws the wash fluid from the sump 142 and forces it through the drain hose 150 to the external drain 148.
The nozzle 154 is configured to direct a fluid flow into the wash tub 124. For example, the nozzle 154 may be in fluid communication with a water source 155 (fig. 2) to direct fluid (e.g., clean water or wash fluid) into the wash tub 124. The nozzle 154 may also be in fluid communication with the sump 142. For example, the pump assembly 144 may direct the washing fluid disposed in the sump 142 to the nozzles 154 to circulate the washing fluid in the washing tub 124.
As shown in fig. 2, a cleaning agent drawer 156 is slidably mounted within the front panel 130. The detergent drawer 156 receives a wash additive (e.g., a detergent, fabric softener, bleach, or any other suitable liquid or powder) and directs the fluid additive to the wash tub 124 during operation of the washing machine device 100. According to the illustrated embodiment, a wash drawer 156 may also be fluidly coupled to the nozzle 154 to facilitate complete and accurate dispensing of the wash additive.
In addition, a water supply valve or control valve 158 may provide water flow from a water supply (e.g., municipal water supply 155) into the cleaning agent dispenser 156 and into the wash tub 124. In this manner, the control valve 158 is generally operable to supply water into the cleaning agent dispenser 156 to produce a flow of cleaning fluid, for example, for a wash cycle, or fresh water, for example, for a rinse cycle. It should be appreciated that the control valve 158 may be located at any other suitable location within the housing 102. Furthermore, although control valve 158 is described herein as regulating the flow of "wash fluid," it should be understood that this term includes water, cleaning agents, other additives, or some mixture thereof.
A control panel 160 including a plurality of input selectors 162 is coupled to the front panel 130. The control panel 160 and the input selector 162 together form a user interface input for an operator to select machine cycles and features. For example, in one embodiment, the display 164 indicates the selected feature, countdown timer, and/or other item of interest to the machine user.
The operation of the washing machine apparatus 100 is controlled by a controller or processing device 166 (fig. 1) operatively coupled to the control panel 160 for user operation to select washing machine cycles and features. In response to user manipulation of the control panel 160, the controller 166 operates the various components of the washing machine device 100 to perform selected machine cycles and features.
The controller 166 may include a memory and a microprocessor, such as a general purpose or special purpose microprocessor, operable to execute programming instructions or micro-control code associated with the cleaning cycle. The memory may represent random access memory (such as DRAM) or read only memory (such as ROM or FLASH). In one embodiment, a processor executes programming instructions stored in a memory. The memory may be a separate component from the processor or may be contained within the processor. Alternatively, the controller 166 may be constructed without the use of a microprocessor, e.g., using a combination of discrete analog and/or digital logic circuits (such as switches, amplifiers, integrators, comparators, flip-flops, and gates, etc.) to perform control functions without reliance on software. The control panel 160 and other components of the washing machine device 100 may communicate with the controller 166 via one or more signal lines or a shared communication bus.
During operation of the washing machine apparatus 100, the soiled laundry items are loaded into the basket 120 through the opening 132 and a washing operation is initiated by an operator manipulating the input selector 162. The wash tub 124 is filled with water, detergent, and/or other fluid additives, for example, via a nozzle 154 and/or a detergent drawer 156. One or more valves (e.g., control valve 158) may be controlled by the washing machine apparatus 100 to enable filling of the basket 120 to a level appropriate for the amount of items being washed and/or rinsed. By way of example, for a wash mode, once the basket 120 is properly filled with fluid, the contents of the basket 120 may be agitated (e.g., by the ribs 128) to wash soiled items in the basket 120.
After the agitation phase of the washing cycle is completed, the washing tub 124 may be discharged. The soiled items may then be rinsed by again adding fluid to the wash tub 124, depending on the specifics of the cleaning cycle selected by the user. The ribs 128 may again provide agitation within the basket 120. One or more spin cycles may also be used. In particular, a spin cycle may be applied after the wash cycle and/or after the rinse cycle in order to spin the wash fluid from the items being washed. During the final spin cycle, basket 120 rotates at a relatively high speed and drain pump assembly 144 may drain wash fluid from sump 142. After the items disposed in the basket 120 are cleaned, washed, and/or rinsed, a user may remove the items from the basket 120, for example, by opening the door 134 and accessing the basket 120 through the opening 132.
The washing machine device 100 may further include a microphone 180 for monitoring sound waves, noise, or other vibrations generated during operation of the washing machine device 100. For example, the microphone 180 may be one or more microphones, acoustic detection devices, vibration sensors, or any other suitable acoustic transducers located at one or more locations in or around the washing machine apparatus 100. For example, according to an exemplary embodiment, the microphone 180 may be mounted within the housing 102. Additionally or alternatively, the microphone 180 may be located elsewhere within the room or residence in which the washing machine device 100 is located. In this regard, any suitable microphone 180 that is acoustically coupled to the washing machine apparatus 100 may be used to monitor the sound produced by the washing machine apparatus 100.
Note that the sound generated during operation of the washing machine device may be associated with one or more operating conditions, failure modes, event occurrences, the presence of one or more different items within the washing load, and the like. For example, if a user accidentally leaves loose coins or bands in the wash load, the noise of such items striking the basket 120 may produce a unique sound signature, which may be identified, for example, by a natural resonant frequency, amplitude, time-based excitation, excitation rate (e.g., the speed at which a particular sound is triggered), time decay of the generated sound wave, or any other acoustic signature or characteristic. As explained in more detail below, aspects of the present subject matter relate to systems and methods for monitoring sounds produced by a device, converting the sounds into three-dimensional spectrograms, and identifying sound signatures in the spectrograms using an artificial intelligence image recognition program.
Further, referring again to fig. 1, the washing machine apparatus 100 may generally include an external communication system 190 configured to allow a user to interact with the washing machine apparatus 100 using a remote device 192. Specifically, according to an exemplary embodiment, the external communication system 190 is configured to allow communication between users, devices, and a remote server or network 194. According to an exemplary embodiment, the washing machine apparatus 100 may communicate with the remote device 192 directly (e.g., through a Local Area Network (LAN), wi-Fi, bluetooth, etc.) or indirectly (e.g., via the network 194), and with a remote server, e.g., to receive notifications, provide acknowledgements, input operational data, send sound signals and sound signatures, etc.
In general, the remote device 192 may be any suitable device for providing and/or receiving communications or commands from a user. In this regard, the remote device 192 may include, for example, a personal telephone, tablet, notebook, or other mobile device. Additionally or alternatively, communication between the device and the user may be accomplished directly through a device control panel (e.g., control panel 160).
In general, network 194 may be any type of communication network. For example, the network 194 may include one or more of a wireless network, a wired network, a personal area network, a local area network, a wide area network, the internet, a cellular network, and the like. In general, communication with the network may use any of a variety of communication protocols (e.g., TCP/IP, HTTP, SMTP, FTP), coding or formats (e.g., HTML, XML), and/or protection schemes (e.g., VPN, secure HTTP, SSL).
External communication system 190 is described herein in accordance with an exemplary embodiment of the present subject matter. However, it should be understood that the exemplary functions and configurations of the external communication system 190 provided herein are merely examples to facilitate describing aspects of the subject matter. The system configuration may vary, other communication means may be used to communicate directly or indirectly with one or more devices, other communication protocols and steps may be implemented, and so on. Such variations and modifications are to be regarded as being within the scope of the present subject matter.
Although described in the context of particular embodiments of the horizontal axis washing machine apparatus 100 using the teachings disclosed herein, it should be understood that the horizontal axis washing machine apparatus 100 is provided by way of example only. Other laundry machine appliances having different configurations, different appearances, and/or different features may also be used with the present subject matter, such as vertical axis laundry machine appliances. Further, the systems and methods described herein may be used to monitor sound produced by any other suitable device or devices.
Now that the configuration of the laundry machine apparatus 100 and the configuration of the controller 166 according to the exemplary embodiment have been provided, an exemplary method 200 of operating the laundry machine apparatus will be described. Although the following discussion relates to an exemplary method 200 of operating the washing machine apparatus 100, those skilled in the art will appreciate that the exemplary method 200 is applicable to the operation of a variety of other washing machine apparatuses, such as vertical axis washing machine apparatuses. In exemplary embodiments, the various method steps disclosed herein may be performed by the controller 166 or another dedicated controller.
Referring generally to fig. 3, a method of operating a washing machine apparatus is provided. According to an exemplary embodiment, method 200 includes: at step 210, a sound signal generated during operation of the washing machine device is obtained using a microphone. For example, continuing with the above example, the microphone 180 may be used to detect noise, sound, vibration, or other sound waves generated during operation of the washing machine device 100. Additionally or alternatively, step 210 may include monitoring sounds produced by the washing machine device 100 when not in operation, sounds produced during a diagnostic process, or any other suitable beep, indication, or sound wave emitted from the washing machine device 100.
Step 220 includes generating a spectrogram from the sound signal. In this regard, for example, the controller 166 may be configured to convert a sound clip or sound recording into a sound spectrum for subsequent analysis. Thus, the original recording of the sound from step 210 may be in the form of noise amplitude versus time, noise frequency versus time, noise amplitude versus noise frequency (e.g., a full fourier transform or FFT), or any other suitable two-dimensional representation of the measured sound. In addition, any suitable sound duration may be measured at step 210 and converted at step 220. For example, according to an exemplary embodiment, the sound signal is between about 0.1 seconds and 10 seconds, between about 1 to 5 seconds, or about 3 seconds.
Note that the spectrogram generated at step 220 may be a three-dimensional representation of sound pressure or amplitude for a given frequency and time. In particular, the spectrogram may be a two-dimensional graph, while the third dimension is represented by a color. According to an exemplary embodiment, the spectrogram represents a sound frequency and a sound amplitude over time. For example, such a spectrogram may be a visual representation of the spectrum of the signal over time, sometimes referred to as a waterfall plot. FIG. 4 provides an exemplary spectrogram that can be generated and analyzed in accordance with aspects of the present subject matter. Note that once the sound signals are converted to a spectrogram, the controller 166 may use various image recognition programs or processing tools to confirm the noise sources and operating conditions, and may use such information to improve machine performance, such as by scheduling maintenance accesses, adjusting operating parameters, providing user notifications, and the like. In this regard, the sonogram image may be augmented with time elements and color temperature may be used to represent signal strength or noise amplitude to improve knowledge of the state or operation of the device.
Step 230 includes analyzing the spectrogram to confirm the voice signature using an image recognition program. For example, image recognition programs that rely on artificial intelligence, neural networks, or any other suitable well-known image processing techniques may be used while remaining within the scope of the present subject matter. In particular, the use of such sonogram images provides several advantages over existing voice recognition programs.
For example, the use of a spectrogram provides the possibility to use various complex image recognition models. According to an exemplary embodiment, the partial image recognition program may compare/classify the spectrograms using a single-label image Convolutional Neural Network (CNN) as a primary algorithm. As used herein, the term "image recognition" and similar terms may be used to generally refer to any suitable method of observing, analyzing, image decomposing, feature extraction, image classification, etc., a spectrogram generated from a sound signal measured by the washing machine device 100. It should be appreciated that any suitable image recognition software or program may be used to analyze the spectrogram, and the controller 166 may be programmed to execute these programs and take corrective action.
According to an exemplary embodiment, the controller may implement a form of image recognition known as convolutional neural network ("CNN") image recognition. In general, CNN may include acquiring an input image (e.g., a spectrogram) and using a convolutional neural network to confirm a unique signature in the image, which is generally referred to herein as a "voice signature. According to still other embodiments, the image recognition program may use any other suitable neural network program.
Additionally or alternatively, an Adam optimizer may be used, a binary cross entropy may be used as the loss function, and softmax may be used as the last layer activation. According to alternative embodiments, any other suitable image classification technique may be used. For example, various migration techniques may be used, but the use of these techniques is not required. If learning is using a migration technique, the neural network architecture can be pre-trained with a common data set, such as VGG16/VGG19/ResNet50, and then the last layer retrained with a device-specific data set.
Additionally or alternatively, the image recognition program may detect a dry or other event, which depends on a comparison of the initial conditions. For example, the dry initial spectrogram image may be subtracted from the spectrogram image when the garment is being dried. The subtracted image can be used to train the neural network in two categories: drying and non-drying. The VGG16 may be a selected neural network architecture if no transfer learning is used. Additionally or alternatively, two spectrogram images may be stacked, e.g., a dry initial spectrogram image from the spectrogram images is at the top and a spectrogram image when dried is at the bottom of the image. In other words, according to an exemplary embodiment, the two images may be joined in any suitable manner and order. Furthermore, according to alternative embodiments, two or more images may be combined by subtracting two sonogram images or modifying such images in any other suitable manner. The combined image may be used in a similar manner to train a neural network in two categories: drying and non-drying. Image combining may be avoided if the detection of sound events does not require comparison with the initial conditions. For example, to detect whether the washing machine is on, various sonogram recordings of the event may be collected, marked and trained.
Note that additional advantages of using sonograms include privacy. For example, sound data collected as images is inherently more private. In this regard, since the spectrogram does not contain information about the exact or even approximate stage of the signal it represents, the sound may be protected and may not be derivable from the spectrogram. Thus, it may not be possible to reverse the procedure and generate a copy of the original signal from the spectrogram. Furthermore, the sonogram image may allow for more efficient use of memory because it can be compressed. Note that compressing the spectrogram can make it easier to transmit or less data intensive. Thus, for example, the controller 166 may be further configured to transmit the spectrogram (e.g., or compressed spectrogram) to a remote server (e.g., remote server 194, etc.) for analysis. The controller 166 may be further configured to receive analytical feedback from a remote server 194. In this way, data processing may be offloaded from the controller 166.
Note that the controller 166 may be further configured to learn a sound signature associated with the washing machine device 100. For example, common conditions or operating noise may be intentionally generated to train the neural network model. The model may then be used to detect a particular sound signature associated with a particular event. Such voice signatures may be stored on the local controller 166 or on the remote server 194. Further, the sound signature may be device specific, may be stored according to a particular model or device configuration, or may be associated with a washing machine device or other device in any other suitable manner.
Step 240 includes adjusting at least one operating parameter of the laundry machine device based at least in part on the confirmation of the sound signature. In this regard, if a sound signature associated with a particular condition is confirmed at step 230, the controller 166 may take corrective action in response to detecting the sound signature, such as by adjusting one or more operating parameters or performing some other action.
As used herein, an "operating parameter" of the washing machine apparatus 100 is any cycle setting, operating time, component setting, spin rate, component configuration, or other operating characteristic that may affect the performance of the washing machine apparatus 100. Thus, reference to an operating parameter adjustment or "adjusting at least one operating parameter" is intended to refer to a control action that improves system performance based on a voice signature or other system parameter. For example, adjusting the operating parameters may include adjusting the agitation time or agitation profile, adjusting the water level, limiting the spin rate of the basket 120, confirming service needs, providing operating instructions to the user, and the like. Other operating parameter adjustments are also possible and are within the scope of the present subject matter.
Further, according to an example embodiment, adjusting the operating parameter may include providing a user notification when the voice signature indicates that a predetermined operating condition exists. For example, according to one exemplary embodiment, the sound signature may be associated with sound generated from one or more of a bearing, a belt, a motor 122, a water valve (e.g., drip or stuck in an on position), a pump, a suspension system, harmonics of a structural component, unwanted contact between components or subsystems, and the like. When a sound signature is generated that indicates a particular operating condition (e.g., a potential failure of one of these components, etc.), a user notification may be provided via display 164 or directly to a user's remote device 192 (e.g., a cellular telephone, via a wireless connection).
For purposes of illustration and discussion, FIG. 3 shows steps performed in a particular order. Using the disclosure provided herein, one of ordinary skill in the art will appreciate that the steps of any of the methods discussed herein may be adjusted, rearranged, expanded, omitted, or modified in various ways without departing from the scope of the present disclosure. Further, although aspects of the method 200 are described using the washing machine apparatus 100 as an example, it should be appreciated that the methods may be applied to the operation of any suitable washing machine apparatus.
This written description uses examples to disclose the invention, including the best mode, and also to enable any person skilled in the art to practice the invention, including making and using any devices or systems and performing any incorporated methods. The patentable scope of the invention is defined by the claims and may include other examples that occur to those skilled in the art. Such other examples are intended to be within the scope of the claims if they include structural elements that do not differ from the literal language of the claims, or if they include equivalent structural elements with insubstantial differences from the literal languages of the claims.
Claims (16)
1. A washing machine apparatus comprising:
a washing tub positioned within the cabinet and defining a washing chamber;
a wash basket rotatably mounted within the wash tub and configured to receive a quantity of items to be washed;
a motor operatively coupled to the basket to selectively rotate the basket;
a microphone for monitoring sounds generated during operation of the washing machine device; and
a controller operatively coupled to the microphone, the controller configured to:
obtaining a sound signal generated during operation of the washing machine device using the microphone;
generating a spectrogram from the sound signal, the spectrogram representing a sound amplitude and a sound frequency over time;
validating a voice signature by analyzing the spectrogram using an image recognition program; and
adjusting at least one operating parameter of the washing machine device based at least in part on the confirmation of the sound signature;
the sound signature is associated with sound generated from at least one of a bearing, a belt, a motor, a water valve, a pump, a suspension system, harmonics of a structural component, or unwanted contact between components or subsystems.
2. The washing machine device of claim 1, wherein the image recognition program uses artificial intelligence to analyze the spectrogram.
3. The washing machine device of claim 1, wherein the image recognition program comprises a convolutional neural network.
4. The laundry machine apparatus of claim 1, wherein the controller is further configured to:
a plurality of sound signatures associated with various operating conditions are learned.
5. The laundry machine apparatus of claim 1, wherein adjusting the at least one operating parameter comprises:
adjusting agitation time or profile, adjusting water level, limiting spin rate, confirming service demand, or providing operational guidance to the user.
6. The laundry machine apparatus of claim 1, wherein adjusting the at least one operating parameter comprises:
an operating period is selected based on the sound signature.
7. The laundry machine apparatus of claim 1, wherein the controller is further configured to:
a user notification is provided when the voice signature indicates that a predetermined operational characteristic exists.
8. The laundry machine apparatus of claim 1, wherein the sound signature is associated with the presence of non-washable articles, and wherein adjusting the at least one operating parameter comprises stopping a wash cycle.
9. The laundry machine apparatus of claim 1, wherein the controller is further configured to:
transmitting the spectrogram to a remote server for analysis; and
analysis feedback from the remote server is received.
10. The laundry machine apparatus of claim 1, wherein the microphone is positioned outside the cabinet and remote from the laundry machine apparatus.
11. A method of operating a washing machine apparatus employing a washing machine apparatus as claimed in any one of claims 1 to 10, wherein the method comprises:
obtaining a sound signal generated during operation of the washing machine device using the microphone;
generating a spectrogram from the sound signal, the spectrogram representing a sound amplitude and a sound frequency over time;
validating a voice signature by analyzing the spectrogram using an image recognition program; and
at least one operating parameter of the washing machine device is adjusted based at least in part on the validation of the voice signature.
12. The method of claim 11, wherein the image recognition program uses artificial intelligence to analyze the spectrogram.
13. The method of claim 11, wherein the image recognition program comprises a convolutional neural network.
14. The method of claim 11, further comprising:
a plurality of sound signatures associated with various operating conditions are learned.
15. The method of claim 11, wherein the sound signature is associated with the presence of a non-washable article, and wherein adjusting the at least one operating parameter comprises stopping a wash cycle.
16. The method of claim 11, further comprising:
transmitting the spectrogram to a remote server for analysis; and
analysis feedback from the remote server is received.
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US16/807,269 US20210277564A1 (en) | 2020-03-03 | 2020-03-03 | System and method for using sound to monitor the operation of a washing machine appliance |
PCT/CN2021/082469 WO2021175336A1 (en) | 2020-03-03 | 2021-03-23 | System and method for using sound to monitor the operation of a washing machine appliance |
Publications (2)
Publication Number | Publication Date |
---|---|
CN115398057A CN115398057A (en) | 2022-11-25 |
CN115398057B true CN115398057B (en) | 2024-01-30 |
Family
ID=77555569
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202180019122.3A Active CN115398057B (en) | 2020-03-03 | 2021-03-23 | System and method for monitoring operation of washing machine device using sound |
Country Status (3)
Country | Link |
---|---|
US (1) | US20210277564A1 (en) |
CN (1) | CN115398057B (en) |
WO (1) | WO2021175336A1 (en) |
Families Citing this family (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CA3115423A1 (en) * | 2020-05-01 | 2021-11-01 | Systemes De Controle Actif Soft Db Inc. | A system and a method for sound recognition |
US11802894B2 (en) * | 2020-09-17 | 2023-10-31 | Silicon Laboratories Inc. | Compressing information in an end node using an autoencoder neural network |
US11692301B2 (en) * | 2020-10-16 | 2023-07-04 | Haier Us Appliance Solutions, Inc. | System and method for using sound to monitor the operation of a dryer appliance |
EP4248226A1 (en) * | 2020-11-17 | 2023-09-27 | Alliance Laundry Systems LLC | Monitoring laundry machine operation using machine learning analysis of acoustic transducer signal information |
CN114488861B (en) * | 2022-01-24 | 2024-08-20 | 杭州老板电器股份有限公司 | Cooking parameter adjustment method and device and electronic equipment |
Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5842288A (en) * | 1996-12-10 | 1998-12-01 | U.S. Controls Corporation | Clothes dryer with chiming alarm |
JP2007144208A (en) * | 1995-01-17 | 2007-06-14 | Hitachi Ltd | Domestic electric appliance |
CN101970744A (en) * | 2008-03-11 | 2011-02-09 | Bsh博世和西门子家用器具有限公司 | Laundry treatment appliance comprising a drum and a sensor, and method for the operation thereof |
CN102547543A (en) * | 2010-12-10 | 2012-07-04 | 杨国屏 | Method for improving accuracy of sound heard by hearing-impaired listener and hearing aid |
CN106868796A (en) * | 2017-03-09 | 2017-06-20 | 青岛海尔智能技术研发有限公司 | A kind of washing machine noise reduction method and washing machine |
CN108642802A (en) * | 2018-04-02 | 2018-10-12 | 青岛海尔洗衣机有限公司 | A kind of control method and washing machine of washing machine |
-
2020
- 2020-03-03 US US16/807,269 patent/US20210277564A1/en not_active Abandoned
-
2021
- 2021-03-23 CN CN202180019122.3A patent/CN115398057B/en active Active
- 2021-03-23 WO PCT/CN2021/082469 patent/WO2021175336A1/en active Application Filing
Patent Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2007144208A (en) * | 1995-01-17 | 2007-06-14 | Hitachi Ltd | Domestic electric appliance |
US5842288A (en) * | 1996-12-10 | 1998-12-01 | U.S. Controls Corporation | Clothes dryer with chiming alarm |
CN101970744A (en) * | 2008-03-11 | 2011-02-09 | Bsh博世和西门子家用器具有限公司 | Laundry treatment appliance comprising a drum and a sensor, and method for the operation thereof |
CN102547543A (en) * | 2010-12-10 | 2012-07-04 | 杨国屏 | Method for improving accuracy of sound heard by hearing-impaired listener and hearing aid |
CN106868796A (en) * | 2017-03-09 | 2017-06-20 | 青岛海尔智能技术研发有限公司 | A kind of washing machine noise reduction method and washing machine |
CN108642802A (en) * | 2018-04-02 | 2018-10-12 | 青岛海尔洗衣机有限公司 | A kind of control method and washing machine of washing machine |
Also Published As
Publication number | Publication date |
---|---|
WO2021175336A1 (en) | 2021-09-10 |
US20210277564A1 (en) | 2021-09-09 |
CN115398057A (en) | 2022-11-25 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN115398057B (en) | System and method for monitoring operation of washing machine device using sound | |
KR101709483B1 (en) | Home appliance and operating method thereof | |
KR102652592B1 (en) | Washing machine and Method for controlling the same | |
US20130185079A1 (en) | Home appliance, home appliance system, and method for operating same | |
US10870940B2 (en) | Washing machine appliance and methods for moisture removal | |
CN102888735B (en) | Laundering device vibration control | |
US11718945B2 (en) | Laundry treating apparatus having camera, and control method thereof | |
CN108625092A (en) | Wash drying device and control method | |
US11692301B2 (en) | System and method for using sound to monitor the operation of a dryer appliance | |
JP7245725B2 (en) | Washing machine and washing machine abnormality diagnosis method | |
US20150059416A1 (en) | Systems and Methods for Detecting an Imbalanced Load in a Washing Machine Appliance Having a Balancing Apparatus | |
US20150368840A1 (en) | Cleaning systems and methods for operating the same | |
US20230124027A1 (en) | Systems and methods for capturing images for use in artificial intelligence processes in a laundry appliance | |
CN103088601B (en) | Roller washing machine and control method | |
US20230366139A1 (en) | Automatic self-clean cycle prior to drying cycle in a laundry appliance | |
US11053619B2 (en) | Smart auto wash cycle for a washing machine appliance | |
KR102627368B1 (en) | A washing machine and method for determining a washing of a tub of the washing machine, and a server and a program therefor | |
US7131159B2 (en) | Method of controlling wobbling washing machine | |
US12006610B2 (en) | Automatic drying cycle after execution of self clean cycle in a laundry appliance | |
US20220364290A1 (en) | Adaptive self clean cycle in a laundry appliance | |
CN203144743U (en) | Drum washing machine | |
US12000076B2 (en) | Front load washer with locking bolt removal confirmation | |
US20230357976A1 (en) | Secure remote testing of washing machine appliances | |
CN118110000A (en) | Method and device for detecting running condition of washing machine, storage medium and electronic device | |
US20240133100A1 (en) | Laundry treatment appliance and method of using the same according to matched laundry loads |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant |