CN115398057A

CN115398057A - System and method for monitoring operation of washing machine apparatus using sound

Info

Publication number: CN115398057A
Application number: CN202180019122.3A
Authority: CN
Inventors: 哈利德·贾马尔·玛莎勒; 大卫·斯科特·邓恩; 胡安·曼纽尔·许尔塔
Original assignee: Qingdao Haier Washing Machine Co Ltd; Haier Smart Home Co Ltd; Haier US Appliance Solutions Inc
Current assignee: Qingdao Haier Washing Machine Co Ltd; Haier Smart Home Co Ltd; Haier US Appliance Solutions Inc
Priority date: 2020-03-03
Filing date: 2021-03-23
Publication date: 2022-11-25
Anticipated expiration: 2041-03-23
Also published as: US20210277564A1; WO2021175336A1; CN115398057B

Abstract

A washing machine appliance includes a microphone for monitoring sounds generated during operation of the washing machine appliance and a controller operatively coupled to the microphone. The controller is configured to obtain sound signals generated during operation of the washing machine apparatus and convert the sound signals into a spectrogram representing sound amplitudes and sound frequencies over time. The method includes analyzing the spectrogram using an artificial intelligence image recognition program to confirm one or more voice signatures associated with particular operating conditions, and adjusting operation of the washing machine appliance based at least in part on the confirmation of the voice signatures.

Description

System and method for monitoring operation of washing machine apparatus using sound

Technical Field

The present subject matter relates generally to washing machine appliances, or more particularly, to systems and methods for monitoring sounds within washing machine appliances and analyzing the sounds to confirm a sound signature associated with a particular event.

Background

Washing machine appliances typically include a tub for holding water or washing fluid (e.g., water and cleaning agents, bleach, and/or other washing additives). The basket is rotatably mounted within the tub and defines a washing chamber for receiving articles to be washed. During normal operation of such a washing machine appliance, washing fluid is directed into the tub and onto the articles within the washing chamber of the basket. The basket or agitation element can be rotated at various speeds to agitate the articles within the washing chamber, to spin wash fluid from the articles within the washing chamber, and the like. The drain pump assembly is operable to drain water from the sump during a spin-off or drain cycle.

It is noted that it is often desirable to monitor the sound generated by a washing machine appliance during operation, for example to identify unexpected objects in the wash load, to diagnose mechanical faults, or to detect other operating conditions. However, conventional washing machines lack any acoustic feedback system. Some washing machines may monitor the sound and provide a notification when the sound exceeds a certain threshold, but such systems have limited utility and effectiveness.

Accordingly, there is a need for a washing machine apparatus having features for improved operation. More specifically, a system and method for monitoring sounds generated by a washing machine appliance and validating a sound signature associated with a particular operating condition would be particularly beneficial.

Disclosure of Invention

Advantages of the invention will be set forth in part in the description which follows, or may be obvious from the description, or may be learned by practice of the invention.

According to an exemplary embodiment of the present disclosure, there is provided a washing machine apparatus including: a washing tub positioned within the cabinet and defining a washing chamber; a wash basket rotatably mounted within the wash tub and configured to receive a quantity of items to be washed; and a motor operatively coupled to the basket to selectively rotate the basket. A microphone is provided to monitor sounds generated during operation of the washing machine appliance, and a controller is operatively coupled to the microphone. The controller is configured to use the microphone to obtain a sound signal generated during operation of the washing machine appliance, to generate a spectrogram therefrom, the spectrogram representing sound amplitude and sound frequency over time, the method may include validating a voice signature by analyzing the spectrogram using an image recognition program, and adjusting at least one operating parameter of the washing machine appliance based at least in part on the validation of the voice signature.

According to another exemplary embodiment of the present disclosure, a method of operating a washing machine apparatus is provided. The washing machine appliance includes a basket rotatably mounted within a wash tub, a motor operatively coupled to the basket to selectively rotate the basket, and a microphone for monitoring sounds produced by the washing machine appliance. The method includes obtaining a sound signal generated during operation of the washing machine appliance using a microphone, generating a spectrogram from the sound signal, the spectrogram representing sound amplitude and sound frequency over time, validating a sound signature by analyzing the spectrogram using an image recognition program, and adjusting at least one operating parameter of the washing machine appliance based at least in part on the validation of the sound signature.

According to another exemplary embodiment of the present disclosure, an apparatus is provided that includes a microphone for monitoring sound generated during operation of the apparatus and a controller operatively coupled to the microphone. The controller is configured to use the microphone to obtain a sound signal generated during operation of the apparatus, generate a spectrogram from the sound signal, the spectrogram representing sound amplitude and sound frequency over time, confirm a sound signature by analyzing the spectrogram using image processing techniques, and adjust at least one operating parameter of the apparatus based at least in part on the confirmation of the sound signature.

These and other features, aspects, and advantages of the present invention will become better understood with reference to the following description and appended claims. The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments of the invention and together with the description, serve to explain the principles of the invention.

Drawings

A full and enabling disclosure of the present invention, including the best mode thereof to one of ordinary skill in the art, is set forth in the specification, which makes reference to the appended figures.

Fig. 1 provides a perspective view of an exemplary washing machine apparatus according to an exemplary embodiment of the present subject matter.

Fig. 2 provides a side sectional view of the exemplary washing machine appliance of fig. 1.

Fig. 3 illustrates a method of confirming an operating condition using a sound generated by a washing machine apparatus according to one embodiment of the present disclosure.

Fig. 4 provides an exemplary spectrogram according to an exemplary embodiment of the present subject matter.

Repeat use of reference characters in the present specification and drawings is intended to represent same or analogous features or elements of the invention.

Detailed Description

Reference now will be made in detail to embodiments of the invention, one or more examples of which are illustrated in the drawings. The examples are provided to illustrate the invention and not to limit the invention. In fact, it will be apparent to those skilled in the art that various modifications and variations can be made in the present invention without departing from the scope or spirit of the invention. For instance, features illustrated or described as part of one embodiment, can be used with another embodiment to yield a still further embodiment. Thus, it is intended that the present invention cover the modifications and variations of this invention provided they come within the scope of the appended claims and their equivalents.

Referring now to the drawings, FIG. 1 is a perspective view of an exemplary horizontal axis washing machine apparatus 100, and FIG. 2 is a side sectional view of the washing machine apparatus 100. As shown, the laundry machine apparatus 100 generally defines a vertical direction V, a lateral direction L, and a depth direction T, each of which are perpendicular to each other such that an orthogonal coordinate system is generally defined. The washing machine apparatus 100 includes a cabinet 102 that extends in a vertical direction V between a top 104 and a bottom 106, in a lateral direction between a left side 108 and a right side 110, and in a depth direction T between a front 112 and a rear 114.

Referring to FIG. 2, the basket 120 is rotatably mounted within the cabinet 102 such that it is rotatable about an axis of rotation A. A motor 122 (e.g., a flat motor, etc.) is in mechanical communication with basket 120 to selectively rotate basket 120 (e.g., during an agitation or rinse cycle of washing machine apparatus 100). The wash basket 120 is received within a wash tub 124 and defines a wash chamber 126 configured to receive items to be washed. The wash tub 124 holds wash fluid and rinse fluid for agitation in the basket 120 within the wash tub 124. As used herein, "wash fluid" may refer to water, a cleaning agent, a fabric softener, a bleach, or any other suitable wash additive, or a combination thereof. Indeed, for simplicity of discussion, these terms may be used interchangeably herein without limiting the subject matter to any particular "wash fluid"

The wash basket 120 may define one or more agitator features that extend into the wash chamber 126 to assist in agitating and cleaning articles disposed within the wash chamber 126 during operation of the washing machine apparatus 100. For example, as shown in FIG. 2, a plurality of ribs 128 extend from basket 120 into washing chamber 126. In this manner, for example, the ribs 128 may lift items disposed in the basket 120 during rotation of the basket 120.

Referring generally to fig. 1 and 2, the cabinet 102 also includes a front panel 130 defining an opening 132 that allows a user to access the basket 120 within the tub 124. More specifically, the washing machine apparatus 100 includes a door 134 positioned to cover the opening 132 and rotatably mounted to the front panel 130. In this manner, the door 134 allows selective access to the opening 132 by being movable between an open position (not shown) that facilitates access to the washtub 124 and a closed position (fig. 1) that prevents access to the washtub 124.

A window 136 in the door 134 allows viewing of the basket 120 when the door 134 is in the closed position, such as during operation of the washing machine apparatus 100. The door 134 also includes a handle (not shown) that a user may pull, for example, when opening and closing the door 134. Further, while the door 134 is shown mounted to the front panel 130, it should be understood that the door 134 may be mounted to another side of the cabinet 102 or any other suitable support according to alternative embodiments.

Referring again to fig. 2, the wash basket 120 further defines a plurality of perforations 140 to facilitate fluid communication between the interior of the basket 120 and the wash tub 124. The sump 142 is defined by the washing tub 124 at the bottom of the washing tub 124 in a vertical direction V. Accordingly, the sump 142 is configured to receive and substantially collect wash fluid during operation of the washing machine apparatus 100. For example, during operation of the washing machine appliance 100, wash fluid may be urged by gravity from the basket 120 through the plurality of perforations 140 to the sump 142.

A drain pump assembly 144 is located below the wash tub 124 and is in fluid communication with the sump 142 for periodically draining dirty wash fluid from the washing machine arrangement 100. The drain pump assembly 144 may generally include a drain pump 146 in fluid communication with an external drain 148 and with the sump 142 via a drain hose 150. During a drain cycle, drain pump 146 forces wash fluid to flow from sump 142 through drain hose 150 to external drain 148. More specifically, drain pump 146 includes a motor (not shown) that is energized during a drain cycle, causing drain pump 146 to draw wash fluid from sump 142 and push it through drain hose 150 to external drain 148.

The nozzle 154 is configured to direct a flow of fluid into the wash tub 124. For example, the nozzle 154 may be in fluid communication with a water source 155 (fig. 2) to direct a fluid (e.g., clean water or wash fluid) into the wash tub 124. The nozzle 154 may also be in fluid communication with the sump 142. For example, the pump assembly 144 may direct wash fluid disposed in the sump 142 to the nozzle 154 to circulate the wash fluid in the wash tub 124.

As shown in fig. 2, the cleaning agent drawer 156 is slidably mounted within the front panel 130. The detergent drawer 156 receives a wash additive (e.g., detergent, fabric softener, bleach, or any other suitable liquid or powder) and directs the fluid additive to the wash tub 124 during operation of the washing machine apparatus 100. According to the illustrated embodiment, a wash agent drawer 156 may also be fluidly coupled to the nozzle 154 to facilitate complete and accurate dispensing of the wash additive.

In addition, a water supply or control valve 158 may provide a flow of water from a water supply (e.g., a municipal water supply 155) into the cleaning agent dispenser 156 and into the wash tub 124. In this manner, the control valve 158 is generally operable to supply water into the cleaning agent dispenser 156 to generate a flow of cleaning fluid, for example, for a wash cycle, or a flow of fresh water, for example, for a rinse cycle. It should be understood that the control valve 158 may be located at any other suitable location within the enclosure 102. Further, while the control valve 158 is described herein as regulating the flow of "wash fluid," it should be understood that this term includes water, cleaning agents, other additives, or some mixture thereof.

A control panel 160 including a plurality of input selectors 162 is coupled to the front panel 130. The control panel 160 and the input selector 162 collectively form a user interface input for an operator to select machine cycles and features. For example, in one embodiment, the display 164 indicates to the machine user the selected feature, countdown timer, and/or other items of interest.

The operation of the washing machine apparatus 100 is controlled by a controller or processing device 166 (fig. 1) that is operatively coupled to the control panel 160 for user manipulation to select washing machine cycles and features. In response to user manipulation of the control panel 160, the controller 166 operates the various components of the washing machine apparatus 100 to perform the selected machine cycle and features.

The controller 166 may include a memory and a microprocessor, such as a general or special purpose microprocessor, operable to execute programmed instructions or microcontrol code associated with the cleaning cycle. The memory may represent random access memory (such as DRAM), or read only memory (such as ROM or FLASH). In one embodiment, the processor executes programming instructions stored in the memory. The memory may be a separate component from the processor or may be included within the processor. Alternatively, the controller 166 may be constructed without the use of a microprocessor, e.g., using a combination of discrete analog and/or digital logic circuits (such as switches, amplifiers, integrators, comparators, flip-flops, and gates, etc.) to perform the control functions without relying on software. The control panel 160 and other components of the washing machine apparatus 100 may communicate with the controller 166 via one or more signal lines or a shared communication bus.

During operation of the washing machine apparatus 100, soiled laundry items are loaded into the basket 120 through the opening 132 and a wash operation is initiated by the operator manipulating the input selector 162. The washing tub 124 is filled with water, cleaning agent and/or other fluid additives, for example via a nozzle 154 and/or a cleaning agent drawer 156. One or more valves (e.g., control valve 158) may be controlled by the washing machine apparatus 100 to effect filling of the basket 120 to a level appropriate for the amount of articles being washed and/or rinsed. By way of example, for the wash mode, once the basket 120 is appropriately filled with fluid, the contents of the basket 120 may be agitated (e.g., by the ribs 128) to wash soiled items of clothing disposed in the basket 120.

After the agitation phase of the wash cycle is completed, the wash tub 124 may be drained. The soiled laundry items may then be rinsed by again adding fluid to the wash tub 124, depending on the specifics of the cleaning cycle selected by the user. The ribs 128 may again provide agitation within the basket 120. One or more spin-off periods may also be used. In particular, a spin cycle may be applied after the wash cycle and/or after the rinse cycle in order to spin wash fluid from the items being washed. During the final spin cycle, the basket 120 rotates at a relatively high speed, and the drain pump assembly 144 may drain the wash fluid from the sump 142. After the items disposed in the basket 120 are cleaned, washed, and/or rinsed, a user may remove the items from the basket 120, for example, by opening the door 134 and into the basket 120 through the opening 132.

The washing machine apparatus 100 may further include a microphone 180 for monitoring sound waves, noise or other vibrations generated during operation of the washing machine apparatus 100. For example, the microphone 180 may be one or more microphones, acoustic detection devices, vibration sensors, or any other suitable acoustic transducer, located at one or more locations in or around the washing machine apparatus 100. For example, according to an exemplary embodiment, the microphone 180 may be mounted within the housing 102. Additionally or alternatively, the microphone 180 may be located elsewhere within the room or dwelling in which the washing machine apparatus 100 is located. In this regard, any suitable microphone 180 acoustically coupled with the washing machine apparatus 100 may be used to monitor the sound produced by the washing machine apparatus 100.

Note that the sound generated during operation of the washing machine appliance may be associated with one or more operating conditions, failure modes, event occurrences, the presence of one or more different items within the wash load, and the like. For example, if a user accidentally leaves loose coins or belts in the wash load, the noise of such items striking the wash basket 120 may create a unique sound signature that may be identified, for example, by a natural resonant frequency, amplitude, time-based excitation, excitation rate (e.g., the speed at which a particular sound is triggered), time decay of the generated sound waves, or any other acoustic signature or characteristic. As explained in more detail below, aspects of the present subject matter relate to systems and methods for monitoring sounds produced by a device, converting these sounds into three-dimensional spectrograms, and identifying sound signatures in the spectrograms using artificial intelligent image recognition programs.

Further, referring again to FIG. 1, the washing machine apparatus 100 may generally include an external communication system 190 configured to allow a user to interact with the washing machine apparatus 100 using a remote device 192. In particular, according to an exemplary embodiment, external communication system 190 is configured to allow communication between users, devices, and a remote server or network 194. According to an exemplary embodiment, the washing machine apparatus 100 may communicate directly (e.g., over a Local Area Network (LAN), wi-Fi, bluetooth, etc.) or indirectly (e.g., via the network 194) with the remote device 192, as well as with a remote server, e.g., to receive notifications, provide confirmations, input operational data, send voice signals and voice signatures, etc.

In general, remote device 192 may be any suitable device for providing and/or receiving communications or commands from a user. In this regard, the remote device 192 may include, for example, a personal telephone, a tablet computer, a laptop computer, or other mobile device. Additionally or alternatively, communication between the device and the user may be accomplished directly through a device control panel (e.g., control panel 160).

In general, the network 194 may be any type of communication network. For example, the network 194 may include one or more of a wireless network, a wired network, a personal area network, a local area network, a wide area network, the internet, a cellular network, and the like. In general, communications with the network may use any of a variety of communication protocols (e.g., TCP/IP, HTTP, SMTP, FTP), encodings or formats (e.g., HTML, XML), and/or protection schemes (e.g., VPN, secure HTTP, SSL).

The external communication system 190 is described herein in accordance with exemplary embodiments of the present subject matter. However, it should be understood that the exemplary functions and configurations of the external communication system 190 provided herein are merely examples to facilitate describing aspects of the present subject matter. The system configuration may vary, other communication means may be used to communicate directly or indirectly with one or more devices, other communication protocols and steps may be implemented, and so forth. Such variations and modifications are to be considered within the scope of the present subject matter.

Although described in the context of particular embodiments of a horizontal axis washing machine apparatus 100 using the teachings disclosed herein, it should be understood that the horizontal axis washing machine apparatus 100 is provided by way of example only. Other washing machine apparatuses having different configurations, different appearances, and/or different features may also be used with the present subject matter, such as a vertical axis washing machine apparatus. Further, the systems and methods described herein may be used to monitor sound produced by any other suitable device or devices.

Now that the configuration of the washing machine apparatus 100 and the configuration of the controller 166 according to the exemplary embodiment have been provided, an exemplary method 200 of operating the washing machine apparatus will be described. Although the following discussion relates to an exemplary method 200 of operating a washing machine apparatus 100, those skilled in the art will appreciate that the exemplary method 200 is applicable to the operation of a variety of other washing machine apparatuses, such as a vertical axis washing machine apparatus. In an exemplary embodiment, the various method steps disclosed herein may be performed by the controller 166 or another dedicated controller.

Referring generally to fig. 3, a method of operating a washing machine appliance is provided. According to an exemplary embodiment, the method 200 includes: in step 210, sound signals generated during operation of the washing machine appliance are obtained using a microphone. For example, continuing the example above, the microphone 180 may be used to detect noise, sound, vibration, or other sound waves generated during operation of the washing machine apparatus 100. Additionally or alternatively, step 210 may include monitoring sounds generated when the washing machine apparatus 100 is not operating, sounds generated during diagnostics, or any other suitable beep, indication, or sound waves emitted from the washing machine apparatus 100.

Step 220 includes generating a spectrogram from the sound signal. In this regard, for example, the controller 166 may be configured to convert the sound segments or sound recordings into spectrograms for subsequent analysis. Thus, the original recording of the sound from step 210 may be in the form of noise amplitude versus time, noise frequency versus time, noise amplitude versus noise frequency (e.g., a full Fourier transform or FFT), or any other suitable two-dimensional representation of the measured sound. In addition, any suitable sound duration may be measured at step 210 and converted at step 220. For example, according to an exemplary embodiment, the sound signal is between about 0.1 seconds and 10 seconds, between about 1 to 5 seconds, or about 3 seconds.

Note that the spectrogram generated at step 220 may be a three-dimensional representation of sound pressure or amplitude at a given frequency and time. In particular, the spectrogram may be a two-dimensional graph, while the third dimension is represented by a color. According to an exemplary embodiment, the spectrogram represents sound frequencies and sound amplitudes over time. For example, such a spectrogram may be a visual representation of the spectrum of a signal over time, sometimes referred to as a waterfall plot. FIG. 4 provides an exemplary spectrogram that can be generated and analyzed in accordance with aspects of the present subject matter. Note that once the acoustic signal is converted into a spectrogram, controller 166 may use various image recognition programs or processing tools to identify the source of the noise and the operating conditions, and may use this information to improve machine performance, such as by scheduling maintenance access, adjusting operating parameters, providing user notification, and the like. In this regard, the spectrogram image may add a time element, and may use color temperature to represent signal strength or noise amplitude to improve understanding of device state or operation.

Step 230 includes analyzing the spectrogram using an image recognition program to validate the voice signature. For example, image recognition programs that rely on artificial intelligence, neural networks, or any other suitable known image processing techniques may be used while remaining within the scope of the present subject matter. In particular, the use of such spectrogram images provides several advantages over existing sound recognition programs.

For example, the use of spectrograms offers the possibility of using various sophisticated image recognition models. According to an exemplary embodiment, the partial image recognition program may use a single label image Convolutional Neural Network (CNN) as a primary algorithm to compare/classify spectrograms. As used herein, the term "image recognition" and similar terms may be used generally to refer to any suitable method of viewing, analyzing, image decomposition, feature extraction, image classification, etc., of a spectrogram produced from a sound signal measured by the washing machine apparatus 100. It should be appreciated that the spectrogram may be analyzed using any suitable image recognition software or program, and the controller 166 may be programmed to execute these programs and take corrective action.

According to an exemplary embodiment, the controller may implement a form of image recognition referred to as convolutional neural network ("CNN") image recognition. In general, CNN may include acquiring an input image (e.g., a spectrogram) and validating a unique signature in the image using a convolutional neural network, which is generally referred to herein as a "sound signature. According to still other embodiments, the image recognition program may use any other suitable neural network program.

Additionally or alternatively, an Adam optimizer may be used, binary cross entropy may be used as a loss function, and softmax may be used as a last layer activation. According to alternative embodiments, any other suitable image classification technique may be used. For example, various migration techniques may be used, but the use of these techniques is not required. If learning using migration techniques, the neural network architecture can be pre-trained with a common data set, such as VGG16/VGG19/ResNet50, and then the last layer retrained with a device-specific data set.

Additionally or alternatively, the image recognition program may detect dryness or other events, depending on the comparison of the initial conditions. For example, when the garment is being dried, the dry initial spectrogram image can be subtracted from the spectrogram image. The subtracted images can be used to train the neural network in two categories: dry, not dry. The VGG16 may be the neural network architecture of choice if no transfer learning is used. Additionally or alternatively, two spectrogram images may be stacked, e.g., a dry initial spectrogram image from the spectrogram image on top and a spectrogram image when dry on the bottom of the image. In other words, according to an exemplary embodiment, two images may be concatenated in any suitable manner and order. Furthermore, according to alternative embodiments, two or more images may be combined by subtracting two spectrogram images or modifying such images in any other suitable manner. This combined image can be used in a similar manner to train the neural network in two classes: dry, not dry. Image combining may be avoided if the detection of the sound event does not need to be compared to the initial conditions. For example, to detect whether a washing machine is on, various spectrogram recordings of the event may be collected, labeled, and trained.

Note that an additional advantage of using spectrograms includes privacy. For example, sound data collected as images is inherently more private. In this regard, since a spectrogram does not contain information about the exact or even approximate phase of the signal it represents, sound may be protected and may not be derivable from the spectrogram. Thus, it may not be possible to reverse the procedure and generate a copy of the original signal from the spectrogram. Furthermore, the spectrogram image may allow for more efficient use of memory, as it can be compressed. Note that compressing the spectrogram can make it easier to transmit or less data intensive. Thus, for example, the controller 166 may be further configured to transmit the spectrogram (e.g., or compressed spectrogram) to a remote server (e.g., remote server 194, etc.) for analysis. The controller 166 may be further configured to receive analysis feedback from the remote server 194. In this manner, data processing may be offloaded from the controller 166.

Note that the controller 166 may be further configured to learn a voice signature associated with the washing machine apparatus 100. For example, common conditions or operational noise may be intentionally generated to train the neural network model. The model may then be used to detect a particular sound signature associated with a particular event. Such a voice signature may be stored on the local controller 166 or the remote server 194. Further, the voice signature may be device specific, may be stored according to a particular model or device configuration, or may be associated with the washing machine device or other device in any other suitable manner.

Step 240 includes adjusting at least one operating parameter of the washing machine appliance based at least in part on the validation of the sound signature. In this regard, if a voice signature associated with a particular condition is validated at step 230, the controller 166 may take corrective action in response to detecting the voice signature, such as by adjusting one or more operating parameters or performing some other action.

As used herein, an "operating parameter" of the washing machine apparatus 100 is any cycle setting, operating time, component setting, spin speed, part configuration, or other operating characteristic that may affect the performance of the washing machine apparatus 100. Thus, reference to an operating parameter adjustment or "adjusting at least one operating parameter" is intended to refer to a control action that improves system performance based on a voice signature or other system parameter. For example, adjusting the operating parameters may include adjusting an agitation time or agitation profile, adjusting a water level, limiting spin and spin speed of the basket 120, confirming service requirements, providing operational guidance to a user, and the like. Other operating parameter adjustments are also possible and within the scope of the present subject matter.

Further, according to an exemplary embodiment, adjusting the operating parameter may include providing a user notification when the voice signature indicates that the predetermined operating condition exists. For example, according to one exemplary embodiment, the sound signature may be associated with sound generated from one or more of bearings, belts, motors 122, water valves (e.g., drip or stuck in an on position), pumps, suspension systems, harmonics of structural components, unwanted contact between components or subsystems, and the like. When a voice signature is generated that indicates a particular operating condition (e.g., a potential failure of one of these components, etc.), a user notification may be provided via the display 164 or directly to a user's remote device 192 (e.g., a cellular telephone, via a wireless connection).

For purposes of illustration and discussion, FIG. 3 shows the steps performed in a particular order. Using the disclosure provided herein, one of ordinary skill in the art will appreciate that the steps of any of the methods discussed herein may be adjusted, rearranged, expanded, omitted, or modified in various ways without departing from the scope of the present disclosure. Further, while aspects of the method 200 are illustrated using the washing machine apparatus 100 as an example, it should be understood that the methods may be applied to the operation of any suitable washing machine apparatus.

This written description uses examples to disclose the invention, including the best mode, and also to enable any person skilled in the art to practice the invention, including making and using any devices or systems and performing any incorporated methods. The patentable scope of the invention is defined by the claims and may include other examples that occur to those skilled in the art. Such other examples are intended to be within the scope of the claims if they include structural elements that do not differ from the literal language of the claims, or if they include equivalent structural elements with insubstantial differences from the literal languages of the claims.

Claims

1. A washing machine appliance comprising:

a washing tub positioned within the cabinet and defining a washing chamber;

a wash basket rotatably mounted within the wash tub and configured to receive a quantity of articles to be washed;

a motor operably coupled to the basket to selectively rotate the basket;

a microphone for monitoring sound generated during operation of the washing machine appliance; and

a controller operably coupled to the microphone, the controller configured to:

obtaining a sound signal generated during operation of the washing machine appliance using the microphone;

generating a spectrogram from the sound signal, the spectrogram representing sound amplitude and sound frequency over time;

validating a voice signature by analyzing the spectrogram using an image recognition program; and

adjusting at least one operating parameter of the washing machine appliance based at least in part on the validation of the sound signature.

2. The washing machine apparatus of claim 1, wherein the image recognition program uses artificial intelligence to analyze the spectrogram.

3. The washing machine apparatus of claim 1, wherein the image recognition program includes a convolutional neural network.

4. The washing machine apparatus of claim 1, wherein the controller is further configured to:

a plurality of voice signatures associated with various operating conditions are learned.

5. The washing machine appliance of claim 1, wherein the sound signature is associated with sound generated from at least one of bearings, belts, motors, water valves, pumps, suspension systems, harmonics of structural components, or unwanted contact between components or subsystems.

6. The washing machine appliance of claim 1, wherein adjusting the at least one operating parameter comprises:

adjusting the churning time or profile, adjusting the water level, limiting the speed of the spin, confirming service requirements, or providing operational guidance to the user.

7. The washing machine appliance of claim 1, wherein adjusting the at least one operating parameter comprises:

an operating period is selected based on the voice signature.

8. The washing machine apparatus of claim 1, wherein the controller is further configured to:

providing a user notification when the voice signature indicates the presence of a predetermined operational characteristic.

9. The washing machine appliance of claim 1, wherein the sound signature is associated with the presence of a non-washable article, and wherein adjusting the at least one operating parameter includes stopping a wash cycle.

10. The washing machine apparatus of claim 1, wherein the controller is further configured to:

transmitting the spectrogram to a remote server for analysis; and

receiving analysis feedback from the remote server.

11. The washing machine appliance of claim 1, wherein the microphone is positioned outside the cabinet and remote from the washing machine appliance.

12. A method of operating a washing machine appliance including a basket rotatably mounted within a wash tub, a motor operatively coupled to the basket to selectively rotate the basket, and a microphone for monitoring sound generated by the washing machine appliance, the method comprising:

13. The method of claim 12, wherein the image recognition program uses artificial intelligence to analyze the spectrogram.

14. The method of claim 12, wherein the image recognition procedure comprises a convolutional neural network.

15. The method of claim 12, further comprising:

16. The method of claim 12, wherein the sound signature is associated with the presence of non-washable articles, and wherein adjusting the at least one operating parameter comprises stopping a washing cycle.

17. The method of claim 12, further comprising:

transmitting the spectrogram to a remote server for analysis; and

receiving analysis feedback from the remote server.

18. An apparatus, comprising:

a microphone for monitoring sound produced during operation of the device; and

a controller operably coupled to the microphone, the controller configured to:

using the microphone to obtain a sound signal produced during operation of the device;

validating a voice signature by analyzing the spectrogram using image processing techniques; and

adjusting at least one operating parameter of the device based at least in part on the validation of the voice signature.

19. The apparatus of claim 18, wherein the image recognition program uses artificial intelligence to analyze the spectrogram.

20. The apparatus of claim 18, wherein the image recognition procedure comprises a convolutional neural network.