CN115398057B - System and method for monitoring operation of washing machine device using sound - Google Patents

System and method for monitoring operation of washing machine device using sound Download PDF

Info

Publication number
CN115398057B
CN115398057B CN202180019122.3A CN202180019122A CN115398057B CN 115398057 B CN115398057 B CN 115398057B CN 202180019122 A CN202180019122 A CN 202180019122A CN 115398057 B CN115398057 B CN 115398057B
Authority
CN
China
Prior art keywords
sound
washing machine
spectrogram
machine apparatus
washing
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202180019122.3A
Other languages
Chinese (zh)
Other versions
CN115398057A (en
Inventor
哈利德·贾马尔·玛莎勒
大卫·斯科特·邓恩
胡安·曼纽尔·许尔塔
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Qingdao Haier Washing Machine Co Ltd
Haier Smart Home Co Ltd
Haier US Appliance Solutions Inc
Original Assignee
Qingdao Haier Washing Machine Co Ltd
Haier Smart Home Co Ltd
Haier US Appliance Solutions Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Qingdao Haier Washing Machine Co Ltd, Haier Smart Home Co Ltd, Haier US Appliance Solutions Inc filed Critical Qingdao Haier Washing Machine Co Ltd
Publication of CN115398057A publication Critical patent/CN115398057A/en
Application granted granted Critical
Publication of CN115398057B publication Critical patent/CN115398057B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • DTEXTILES; PAPER
    • D06TREATMENT OF TEXTILES OR THE LIKE; LAUNDERING; FLEXIBLE MATERIALS NOT OTHERWISE PROVIDED FOR
    • D06FLAUNDERING, DRYING, IRONING, PRESSING OR FOLDING TEXTILE ARTICLES
    • D06F34/00Details of control systems for washing machines, washer-dryers or laundry dryers
    • D06F34/14Arrangements for detecting or measuring specific parameters
    • DTEXTILES; PAPER
    • D06TREATMENT OF TEXTILES OR THE LIKE; LAUNDERING; FLEXIBLE MATERIALS NOT OTHERWISE PROVIDED FOR
    • D06FLAUNDERING, DRYING, IRONING, PRESSING OR FOLDING TEXTILE ARTICLES
    • D06F33/00Control of operations performed in washing machines or washer-dryers 
    • D06F33/30Control of washing machines characterised by the purpose or target of the control 
    • D06F33/32Control of operational steps, e.g. optimisation or improvement of operational steps depending on the condition of the laundry
    • DTEXTILES; PAPER
    • D06TREATMENT OF TEXTILES OR THE LIKE; LAUNDERING; FLEXIBLE MATERIALS NOT OTHERWISE PROVIDED FOR
    • D06FLAUNDERING, DRYING, IRONING, PRESSING OR FOLDING TEXTILE ARTICLES
    • D06F33/00Control of operations performed in washing machines or washer-dryers 
    • D06F33/30Control of washing machines characterised by the purpose or target of the control 
    • D06F33/32Control of operational steps, e.g. optimisation or improvement of operational steps depending on the condition of the laundry
    • D06F33/34Control of operational steps, e.g. optimisation or improvement of operational steps depending on the condition of the laundry of water filling
    • DTEXTILES; PAPER
    • D06TREATMENT OF TEXTILES OR THE LIKE; LAUNDERING; FLEXIBLE MATERIALS NOT OTHERWISE PROVIDED FOR
    • D06FLAUNDERING, DRYING, IRONING, PRESSING OR FOLDING TEXTILE ARTICLES
    • D06F33/00Control of operations performed in washing machines or washer-dryers 
    • D06F33/30Control of washing machines characterised by the purpose or target of the control 
    • D06F33/32Control of operational steps, e.g. optimisation or improvement of operational steps depending on the condition of the laundry
    • D06F33/40Control of operational steps, e.g. optimisation or improvement of operational steps depending on the condition of the laundry of centrifugal separation of water from the laundry
    • DTEXTILES; PAPER
    • D06TREATMENT OF TEXTILES OR THE LIKE; LAUNDERING; FLEXIBLE MATERIALS NOT OTHERWISE PROVIDED FOR
    • D06FLAUNDERING, DRYING, IRONING, PRESSING OR FOLDING TEXTILE ARTICLES
    • D06F33/00Control of operations performed in washing machines or washer-dryers 
    • D06F33/30Control of washing machines characterised by the purpose or target of the control 
    • D06F33/44Control of the operating time, e.g. reduction of overall operating time
    • DTEXTILES; PAPER
    • D06TREATMENT OF TEXTILES OR THE LIKE; LAUNDERING; FLEXIBLE MATERIALS NOT OTHERWISE PROVIDED FOR
    • D06FLAUNDERING, DRYING, IRONING, PRESSING OR FOLDING TEXTILE ARTICLES
    • D06F2103/00Parameters monitored or detected for the control of domestic laundry washing machines, washer-dryers or laundry dryers
    • DTEXTILES; PAPER
    • D06TREATMENT OF TEXTILES OR THE LIKE; LAUNDERING; FLEXIBLE MATERIALS NOT OTHERWISE PROVIDED FOR
    • D06FLAUNDERING, DRYING, IRONING, PRESSING OR FOLDING TEXTILE ARTICLES
    • D06F2103/00Parameters monitored or detected for the control of domestic laundry washing machines, washer-dryers or laundry dryers
    • D06F2103/18Washing liquid level
    • DTEXTILES; PAPER
    • D06TREATMENT OF TEXTILES OR THE LIKE; LAUNDERING; FLEXIBLE MATERIALS NOT OTHERWISE PROVIDED FOR
    • D06FLAUNDERING, DRYING, IRONING, PRESSING OR FOLDING TEXTILE ARTICLES
    • D06F2103/00Parameters monitored or detected for the control of domestic laundry washing machines, washer-dryers or laundry dryers
    • D06F2103/24Spin speed; Drum movements
    • DTEXTILES; PAPER
    • D06TREATMENT OF TEXTILES OR THE LIKE; LAUNDERING; FLEXIBLE MATERIALS NOT OTHERWISE PROVIDED FOR
    • D06FLAUNDERING, DRYING, IRONING, PRESSING OR FOLDING TEXTILE ARTICLES
    • D06F2103/00Parameters monitored or detected for the control of domestic laundry washing machines, washer-dryers or laundry dryers
    • D06F2103/26Imbalance; Noise level
    • DTEXTILES; PAPER
    • D06TREATMENT OF TEXTILES OR THE LIKE; LAUNDERING; FLEXIBLE MATERIALS NOT OTHERWISE PROVIDED FOR
    • D06FLAUNDERING, DRYING, IRONING, PRESSING OR FOLDING TEXTILE ARTICLES
    • D06F2105/00Systems or parameters controlled or affected by the control systems of washing machines, washer-dryers or laundry dryers
    • D06F2105/52Changing sequence of operational steps; Carrying out additional operational steps; Modifying operational steps, e.g. by extending duration of steps
    • DTEXTILES; PAPER
    • D06TREATMENT OF TEXTILES OR THE LIKE; LAUNDERING; FLEXIBLE MATERIALS NOT OTHERWISE PROVIDED FOR
    • D06FLAUNDERING, DRYING, IRONING, PRESSING OR FOLDING TEXTILE ARTICLES
    • D06F34/00Details of control systems for washing machines, washer-dryers or laundry dryers
    • D06F34/04Signal transfer or data transmission arrangements

Landscapes

  • Engineering & Computer Science (AREA)
  • Textile Engineering (AREA)
  • Control Of Washing Machine And Dryer (AREA)

Abstract

A washing machine apparatus includes a microphone for monitoring sound generated during operation of the washing machine apparatus and a controller operatively coupled to the microphone. The controller is configured to obtain a sound signal generated during operation of the washing machine device and to convert the sound signal into a spectrogram representing a sound amplitude and a sound frequency that vary over time. The spectrogram is analyzed using an artificial intelligence image recognition program to confirm one or more sound signatures associated with particular operating conditions, and operation of the washing machine device is adjusted based at least in part on the confirmation of the sound signatures.

Description

System and method for monitoring operation of washing machine device using sound
Technical Field
The present subject matter relates generally to washing machine devices, or more particularly, to systems and methods for monitoring sounds within a washing machine device and analyzing the sounds to confirm a sound signature associated with a particular event.
Background
Washing machine devices typically include a tub for holding water or a washing fluid (e.g., water and a cleaning agent, bleach, and/or other washing additives). A basket is rotatably mounted within the tub and defines a washing chamber for receiving articles to be washed. During normal operation of such a washing machine device, washing fluid is directed into the tub and onto the items within the washing chamber of the basket. The basket or agitation member can be rotated at various speeds to agitate the articles within the washing chamber, to spin washing fluid from the articles within the washing chamber, and so forth. The drain pump assembly is operable to drain water from within the sump during a spin or drain cycle.
Note that it is often desirable to monitor sounds produced by the washing machine equipment during operation, for example to confirm unexpected objects in the washing load, diagnose mechanical faults, or detect other operating conditions. However, conventional washing machines lack any acoustic feedback system. Some washing machines can monitor sound and provide notification when the sound exceeds a certain threshold, but such systems have limited utility and effectiveness.
Accordingly, there is a need for a washing machine apparatus having features for improved operation. More particularly, a system and method for monitoring sounds produced by a washing machine device and validating sound signatures associated with particular operating conditions would be particularly beneficial.
Disclosure of Invention
Advantages of the invention will be set forth in part in the description which follows, or may be obvious from the description, or may be learned by practice of the invention.
According to an exemplary embodiment of the present disclosure, there is provided a washing machine apparatus including: a washing tub positioned within the cabinet and defining a washing chamber; a basket rotatably mounted within the tub and configured to receive a quantity of articles to be washed; and a motor operatively coupled to the basket to selectively rotate the basket. A microphone is provided to monitor sound generated during operation of the washing machine device, and a controller is operatively coupled to the microphone. The controller is configured to obtain a sound signal generated during operation of the washing machine device using the microphone, generate a spectrogram from the sound signal, the spectrogram representing a sound amplitude and a sound frequency over time, confirm a sound signature by analyzing the spectrogram using the image recognition program, and adjust at least one operating parameter of the washing machine device based at least in part on the confirmation of the sound signature.
According to another exemplary embodiment of the present disclosure, a method of operating a laundry machine apparatus is provided. The washing machine apparatus includes a basket rotatably mounted within a washing tub, a motor operatively coupled to the basket to selectively rotate the basket, and a microphone for monitoring sounds produced by the washing machine apparatus. The method includes obtaining a sound signal generated during operation of the washing machine device using a microphone, generating a spectrogram from the sound signal, the spectrogram representing a sound amplitude and a sound frequency over time, validating a sound signature by analyzing the spectrogram using an image recognition program, and adjusting at least one operating parameter of the washing machine device based at least in part on the validation of the sound signature.
According to another exemplary embodiment of the present disclosure, a device is provided that includes a microphone for monitoring sound generated during operation of the device and a controller operatively coupled to the microphone. The controller is configured to obtain a sound signal generated during operation of the device using the microphone, generate a spectrogram from the sound signal, the spectrogram representing a sound amplitude and a sound frequency over time, confirm a sound signature by analyzing the spectrogram using image processing techniques, and adjust at least one operating parameter of the device based at least in part on the confirmation of the sound signature.
These and other features, aspects, and advantages of the present invention will become better understood with reference to the following description and appended claims. The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments of the invention and together with the description, serve to explain the principles of the invention.
Drawings
A full and enabling disclosure of the present invention, including the best mode thereof to one of ordinary skill in the art, is set forth in the specification, which makes reference to the appended figures.
Fig. 1 provides a perspective view of an exemplary laundry machine apparatus according to an exemplary embodiment of the present subject matter.
Fig. 2 provides a side cross-sectional view of the exemplary laundry machine apparatus of fig. 1.
Fig. 3 illustrates a method of confirming an operation condition using sound generated by a washing machine device according to one embodiment of the present disclosure.
FIG. 4 provides an exemplary spectrogram in accordance with an exemplary embodiment of the present subject matter.
Repeat use of reference characters in the present specification and drawings is intended to represent same or analogous features or elements of the invention.
Detailed Description
Reference now will be made in detail to embodiments of the invention, one or more examples of which are illustrated in the drawings. The examples are provided to illustrate the invention and not to limit the invention. Indeed, it will be apparent to those skilled in the art that various modifications and variations can be made in the present invention without departing from the scope or spirit of the invention. For instance, features illustrated or described as part of one embodiment, can be used on another embodiment to yield a still further embodiment. Accordingly, it is intended that the present invention cover the modifications and variations of this invention provided they come within the scope of the appended claims and their equivalents.
Referring now to the drawings, FIG. 1 is a perspective view of an exemplary horizontal axis washing machine apparatus 100, and FIG. 2 is a side cross-sectional view of the washing machine apparatus 100. As shown, the washing machine apparatus 100 generally defines a vertical direction V, a lateral direction L, and a depth direction T, each of which are perpendicular to each other such that generally an orthogonal coordinate system is defined. The washing machine apparatus 100 includes a cabinet 102 extending in a vertical direction V between a top 104 and a bottom 106, in a lateral direction between a left side 108 and a right side 110, and in a depth direction T between a front 112 and a rear 114.
Referring to fig. 2, the basket 120 is rotatably installed in the cabinet 102 such that it can rotate about the rotation axis a. A motor 122 (e.g., a flat motor, etc.) is in mechanical communication with the basket 120 to selectively rotate the basket 120 (e.g., during a agitation or rinse cycle of the laundry machine apparatus 100). The basket 120 is received within the wash tub 124 and defines a wash chamber 126 configured to receive items to be washed. The wash tub 124 holds wash fluid and rinse fluid for agitation in the wash basket 120 within the wash tub 124. As used herein, "wash fluid" may refer to water, a cleaning agent, a fabric softener, a bleach, or any other suitable wash additive or combination thereof. Indeed, for simplicity of discussion, these terms may be used interchangeably herein without limiting the subject matter to any particular "wash fluid":
the basket 120 may define one or more agitator features that extend into the washing chamber 126 to assist in agitating and cleaning items disposed within the washing chamber 126 during operation of the washing machine apparatus 100. For example, as shown in FIG. 2, a plurality of ribs 128 extend from the basket 120 into the washing chamber 126. In this manner, for example, the ribs 128 may lift items disposed in the basket 120 during rotation of the basket 120.
Referring generally to fig. 1 and 2, the chassis 102 further includes a front panel 130 defining an opening 132 that allows a user to access the basket 120 within the tub 124. More specifically, the washing machine apparatus 100 includes a door 134 positioned to cover the opening 132 and rotatably mounted to the front panel 130. In this manner, the door 134 allows selective access to the opening 132 by being movable between an open position (not shown) that facilitates access to the wash tub 124 and a closed position (FIG. 1) that prevents access to the wash tub 124.
A window 136 in the door 134 allows viewing of the basket 120 when the door 134 is in a closed position, such as during operation of the washing machine apparatus 100. The door 134 also includes a handle (not shown) that may be pulled by a user, for example, when opening and closing the door 134. Further, although the door 134 is shown mounted to the front panel 130, it should be understood that the door 134 may be mounted to the other side of the chassis 102 or any other suitable support according to alternative embodiments.
Referring again to FIG. 2, the basket 120 also defines a plurality of perforations 140 to facilitate fluid communication between the interior of the basket 120 and the wash tub 124. The sump 142 is defined by the washing tub 124 at the bottom of the washing tub 124 in the vertical direction V. Accordingly, the sump 142 is configured to receive and substantially collect wash fluid during operation of the washing machine apparatus 100. For example, during operation of the washing machine apparatus 100, the washing fluid may be urged from the basket 120 through the plurality of perforations 140 to the sump 142 by gravity.
A drain pump assembly 144 is positioned below the wash tub 124 and is in fluid communication with the sump 142 for periodically draining dirty wash fluid from the washing machine apparatus 100. The drain pump assembly 144 may generally include a drain pump 146 in fluid communication with an external drain 148 and with the sump 142 via a drain hose 150. During the drain cycle, the drain pump 146 forces the wash fluid to flow from the sump 142 through the drain hose 150 to the external drain 148. More specifically, the drain pump 146 includes a motor (not shown) that is energized during a drain cycle such that the drain pump 146 draws the wash fluid from the sump 142 and forces it through the drain hose 150 to the external drain 148.
The nozzle 154 is configured to direct a fluid flow into the wash tub 124. For example, the nozzle 154 may be in fluid communication with a water source 155 (fig. 2) to direct fluid (e.g., clean water or wash fluid) into the wash tub 124. The nozzle 154 may also be in fluid communication with the sump 142. For example, the pump assembly 144 may direct the washing fluid disposed in the sump 142 to the nozzles 154 to circulate the washing fluid in the washing tub 124.
As shown in fig. 2, a cleaning agent drawer 156 is slidably mounted within the front panel 130. The detergent drawer 156 receives a wash additive (e.g., a detergent, fabric softener, bleach, or any other suitable liquid or powder) and directs the fluid additive to the wash tub 124 during operation of the washing machine device 100. According to the illustrated embodiment, a wash drawer 156 may also be fluidly coupled to the nozzle 154 to facilitate complete and accurate dispensing of the wash additive.
In addition, a water supply valve or control valve 158 may provide water flow from a water supply (e.g., municipal water supply 155) into the cleaning agent dispenser 156 and into the wash tub 124. In this manner, the control valve 158 is generally operable to supply water into the cleaning agent dispenser 156 to produce a flow of cleaning fluid, for example, for a wash cycle, or fresh water, for example, for a rinse cycle. It should be appreciated that the control valve 158 may be located at any other suitable location within the housing 102. Furthermore, although control valve 158 is described herein as regulating the flow of "wash fluid," it should be understood that this term includes water, cleaning agents, other additives, or some mixture thereof.
A control panel 160 including a plurality of input selectors 162 is coupled to the front panel 130. The control panel 160 and the input selector 162 together form a user interface input for an operator to select machine cycles and features. For example, in one embodiment, the display 164 indicates the selected feature, countdown timer, and/or other item of interest to the machine user.
The operation of the washing machine apparatus 100 is controlled by a controller or processing device 166 (fig. 1) operatively coupled to the control panel 160 for user operation to select washing machine cycles and features. In response to user manipulation of the control panel 160, the controller 166 operates the various components of the washing machine device 100 to perform selected machine cycles and features.
The controller 166 may include a memory and a microprocessor, such as a general purpose or special purpose microprocessor, operable to execute programming instructions or micro-control code associated with the cleaning cycle. The memory may represent random access memory (such as DRAM) or read only memory (such as ROM or FLASH). In one embodiment, a processor executes programming instructions stored in a memory. The memory may be a separate component from the processor or may be contained within the processor. Alternatively, the controller 166 may be constructed without the use of a microprocessor, e.g., using a combination of discrete analog and/or digital logic circuits (such as switches, amplifiers, integrators, comparators, flip-flops, and gates, etc.) to perform control functions without reliance on software. The control panel 160 and other components of the washing machine device 100 may communicate with the controller 166 via one or more signal lines or a shared communication bus.
During operation of the washing machine apparatus 100, the soiled laundry items are loaded into the basket 120 through the opening 132 and a washing operation is initiated by an operator manipulating the input selector 162. The wash tub 124 is filled with water, detergent, and/or other fluid additives, for example, via a nozzle 154 and/or a detergent drawer 156. One or more valves (e.g., control valve 158) may be controlled by the washing machine apparatus 100 to enable filling of the basket 120 to a level appropriate for the amount of items being washed and/or rinsed. By way of example, for a wash mode, once the basket 120 is properly filled with fluid, the contents of the basket 120 may be agitated (e.g., by the ribs 128) to wash soiled items in the basket 120.
After the agitation phase of the washing cycle is completed, the washing tub 124 may be discharged. The soiled items may then be rinsed by again adding fluid to the wash tub 124, depending on the specifics of the cleaning cycle selected by the user. The ribs 128 may again provide agitation within the basket 120. One or more spin cycles may also be used. In particular, a spin cycle may be applied after the wash cycle and/or after the rinse cycle in order to spin the wash fluid from the items being washed. During the final spin cycle, basket 120 rotates at a relatively high speed and drain pump assembly 144 may drain wash fluid from sump 142. After the items disposed in the basket 120 are cleaned, washed, and/or rinsed, a user may remove the items from the basket 120, for example, by opening the door 134 and accessing the basket 120 through the opening 132.
The washing machine device 100 may further include a microphone 180 for monitoring sound waves, noise, or other vibrations generated during operation of the washing machine device 100. For example, the microphone 180 may be one or more microphones, acoustic detection devices, vibration sensors, or any other suitable acoustic transducers located at one or more locations in or around the washing machine apparatus 100. For example, according to an exemplary embodiment, the microphone 180 may be mounted within the housing 102. Additionally or alternatively, the microphone 180 may be located elsewhere within the room or residence in which the washing machine device 100 is located. In this regard, any suitable microphone 180 that is acoustically coupled to the washing machine apparatus 100 may be used to monitor the sound produced by the washing machine apparatus 100.
Note that the sound generated during operation of the washing machine device may be associated with one or more operating conditions, failure modes, event occurrences, the presence of one or more different items within the washing load, and the like. For example, if a user accidentally leaves loose coins or bands in the wash load, the noise of such items striking the basket 120 may produce a unique sound signature, which may be identified, for example, by a natural resonant frequency, amplitude, time-based excitation, excitation rate (e.g., the speed at which a particular sound is triggered), time decay of the generated sound wave, or any other acoustic signature or characteristic. As explained in more detail below, aspects of the present subject matter relate to systems and methods for monitoring sounds produced by a device, converting the sounds into three-dimensional spectrograms, and identifying sound signatures in the spectrograms using an artificial intelligence image recognition program.
Further, referring again to fig. 1, the washing machine apparatus 100 may generally include an external communication system 190 configured to allow a user to interact with the washing machine apparatus 100 using a remote device 192. Specifically, according to an exemplary embodiment, the external communication system 190 is configured to allow communication between users, devices, and a remote server or network 194. According to an exemplary embodiment, the washing machine apparatus 100 may communicate with the remote device 192 directly (e.g., through a Local Area Network (LAN), wi-Fi, bluetooth, etc.) or indirectly (e.g., via the network 194), and with a remote server, e.g., to receive notifications, provide acknowledgements, input operational data, send sound signals and sound signatures, etc.
In general, the remote device 192 may be any suitable device for providing and/or receiving communications or commands from a user. In this regard, the remote device 192 may include, for example, a personal telephone, tablet, notebook, or other mobile device. Additionally or alternatively, communication between the device and the user may be accomplished directly through a device control panel (e.g., control panel 160).
In general, network 194 may be any type of communication network. For example, the network 194 may include one or more of a wireless network, a wired network, a personal area network, a local area network, a wide area network, the internet, a cellular network, and the like. In general, communication with the network may use any of a variety of communication protocols (e.g., TCP/IP, HTTP, SMTP, FTP), coding or formats (e.g., HTML, XML), and/or protection schemes (e.g., VPN, secure HTTP, SSL).
External communication system 190 is described herein in accordance with an exemplary embodiment of the present subject matter. However, it should be understood that the exemplary functions and configurations of the external communication system 190 provided herein are merely examples to facilitate describing aspects of the subject matter. The system configuration may vary, other communication means may be used to communicate directly or indirectly with one or more devices, other communication protocols and steps may be implemented, and so on. Such variations and modifications are to be regarded as being within the scope of the present subject matter.
Although described in the context of particular embodiments of the horizontal axis washing machine apparatus 100 using the teachings disclosed herein, it should be understood that the horizontal axis washing machine apparatus 100 is provided by way of example only. Other laundry machine appliances having different configurations, different appearances, and/or different features may also be used with the present subject matter, such as vertical axis laundry machine appliances. Further, the systems and methods described herein may be used to monitor sound produced by any other suitable device or devices.
Now that the configuration of the laundry machine apparatus 100 and the configuration of the controller 166 according to the exemplary embodiment have been provided, an exemplary method 200 of operating the laundry machine apparatus will be described. Although the following discussion relates to an exemplary method 200 of operating the washing machine apparatus 100, those skilled in the art will appreciate that the exemplary method 200 is applicable to the operation of a variety of other washing machine apparatuses, such as vertical axis washing machine apparatuses. In exemplary embodiments, the various method steps disclosed herein may be performed by the controller 166 or another dedicated controller.
Referring generally to fig. 3, a method of operating a washing machine apparatus is provided. According to an exemplary embodiment, method 200 includes: at step 210, a sound signal generated during operation of the washing machine device is obtained using a microphone. For example, continuing with the above example, the microphone 180 may be used to detect noise, sound, vibration, or other sound waves generated during operation of the washing machine device 100. Additionally or alternatively, step 210 may include monitoring sounds produced by the washing machine device 100 when not in operation, sounds produced during a diagnostic process, or any other suitable beep, indication, or sound wave emitted from the washing machine device 100.
Step 220 includes generating a spectrogram from the sound signal. In this regard, for example, the controller 166 may be configured to convert a sound clip or sound recording into a sound spectrum for subsequent analysis. Thus, the original recording of the sound from step 210 may be in the form of noise amplitude versus time, noise frequency versus time, noise amplitude versus noise frequency (e.g., a full fourier transform or FFT), or any other suitable two-dimensional representation of the measured sound. In addition, any suitable sound duration may be measured at step 210 and converted at step 220. For example, according to an exemplary embodiment, the sound signal is between about 0.1 seconds and 10 seconds, between about 1 to 5 seconds, or about 3 seconds.
Note that the spectrogram generated at step 220 may be a three-dimensional representation of sound pressure or amplitude for a given frequency and time. In particular, the spectrogram may be a two-dimensional graph, while the third dimension is represented by a color. According to an exemplary embodiment, the spectrogram represents a sound frequency and a sound amplitude over time. For example, such a spectrogram may be a visual representation of the spectrum of the signal over time, sometimes referred to as a waterfall plot. FIG. 4 provides an exemplary spectrogram that can be generated and analyzed in accordance with aspects of the present subject matter. Note that once the sound signals are converted to a spectrogram, the controller 166 may use various image recognition programs or processing tools to confirm the noise sources and operating conditions, and may use such information to improve machine performance, such as by scheduling maintenance accesses, adjusting operating parameters, providing user notifications, and the like. In this regard, the sonogram image may be augmented with time elements and color temperature may be used to represent signal strength or noise amplitude to improve knowledge of the state or operation of the device.
Step 230 includes analyzing the spectrogram to confirm the voice signature using an image recognition program. For example, image recognition programs that rely on artificial intelligence, neural networks, or any other suitable well-known image processing techniques may be used while remaining within the scope of the present subject matter. In particular, the use of such sonogram images provides several advantages over existing voice recognition programs.
For example, the use of a spectrogram provides the possibility to use various complex image recognition models. According to an exemplary embodiment, the partial image recognition program may compare/classify the spectrograms using a single-label image Convolutional Neural Network (CNN) as a primary algorithm. As used herein, the term "image recognition" and similar terms may be used to generally refer to any suitable method of observing, analyzing, image decomposing, feature extraction, image classification, etc., a spectrogram generated from a sound signal measured by the washing machine device 100. It should be appreciated that any suitable image recognition software or program may be used to analyze the spectrogram, and the controller 166 may be programmed to execute these programs and take corrective action.
According to an exemplary embodiment, the controller may implement a form of image recognition known as convolutional neural network ("CNN") image recognition. In general, CNN may include acquiring an input image (e.g., a spectrogram) and using a convolutional neural network to confirm a unique signature in the image, which is generally referred to herein as a "voice signature. According to still other embodiments, the image recognition program may use any other suitable neural network program.
Additionally or alternatively, an Adam optimizer may be used, a binary cross entropy may be used as the loss function, and softmax may be used as the last layer activation. According to alternative embodiments, any other suitable image classification technique may be used. For example, various migration techniques may be used, but the use of these techniques is not required. If learning is using a migration technique, the neural network architecture can be pre-trained with a common data set, such as VGG16/VGG19/ResNet50, and then the last layer retrained with a device-specific data set.
Additionally or alternatively, the image recognition program may detect a dry or other event, which depends on a comparison of the initial conditions. For example, the dry initial spectrogram image may be subtracted from the spectrogram image when the garment is being dried. The subtracted image can be used to train the neural network in two categories: drying and non-drying. The VGG16 may be a selected neural network architecture if no transfer learning is used. Additionally or alternatively, two spectrogram images may be stacked, e.g., a dry initial spectrogram image from the spectrogram images is at the top and a spectrogram image when dried is at the bottom of the image. In other words, according to an exemplary embodiment, the two images may be joined in any suitable manner and order. Furthermore, according to alternative embodiments, two or more images may be combined by subtracting two sonogram images or modifying such images in any other suitable manner. The combined image may be used in a similar manner to train a neural network in two categories: drying and non-drying. Image combining may be avoided if the detection of sound events does not require comparison with the initial conditions. For example, to detect whether the washing machine is on, various sonogram recordings of the event may be collected, marked and trained.
Note that additional advantages of using sonograms include privacy. For example, sound data collected as images is inherently more private. In this regard, since the spectrogram does not contain information about the exact or even approximate stage of the signal it represents, the sound may be protected and may not be derivable from the spectrogram. Thus, it may not be possible to reverse the procedure and generate a copy of the original signal from the spectrogram. Furthermore, the sonogram image may allow for more efficient use of memory because it can be compressed. Note that compressing the spectrogram can make it easier to transmit or less data intensive. Thus, for example, the controller 166 may be further configured to transmit the spectrogram (e.g., or compressed spectrogram) to a remote server (e.g., remote server 194, etc.) for analysis. The controller 166 may be further configured to receive analytical feedback from a remote server 194. In this way, data processing may be offloaded from the controller 166.
Note that the controller 166 may be further configured to learn a sound signature associated with the washing machine device 100. For example, common conditions or operating noise may be intentionally generated to train the neural network model. The model may then be used to detect a particular sound signature associated with a particular event. Such voice signatures may be stored on the local controller 166 or on the remote server 194. Further, the sound signature may be device specific, may be stored according to a particular model or device configuration, or may be associated with a washing machine device or other device in any other suitable manner.
Step 240 includes adjusting at least one operating parameter of the laundry machine device based at least in part on the confirmation of the sound signature. In this regard, if a sound signature associated with a particular condition is confirmed at step 230, the controller 166 may take corrective action in response to detecting the sound signature, such as by adjusting one or more operating parameters or performing some other action.
As used herein, an "operating parameter" of the washing machine apparatus 100 is any cycle setting, operating time, component setting, spin rate, component configuration, or other operating characteristic that may affect the performance of the washing machine apparatus 100. Thus, reference to an operating parameter adjustment or "adjusting at least one operating parameter" is intended to refer to a control action that improves system performance based on a voice signature or other system parameter. For example, adjusting the operating parameters may include adjusting the agitation time or agitation profile, adjusting the water level, limiting the spin rate of the basket 120, confirming service needs, providing operating instructions to the user, and the like. Other operating parameter adjustments are also possible and are within the scope of the present subject matter.
Further, according to an example embodiment, adjusting the operating parameter may include providing a user notification when the voice signature indicates that a predetermined operating condition exists. For example, according to one exemplary embodiment, the sound signature may be associated with sound generated from one or more of a bearing, a belt, a motor 122, a water valve (e.g., drip or stuck in an on position), a pump, a suspension system, harmonics of a structural component, unwanted contact between components or subsystems, and the like. When a sound signature is generated that indicates a particular operating condition (e.g., a potential failure of one of these components, etc.), a user notification may be provided via display 164 or directly to a user's remote device 192 (e.g., a cellular telephone, via a wireless connection).
For purposes of illustration and discussion, FIG. 3 shows steps performed in a particular order. Using the disclosure provided herein, one of ordinary skill in the art will appreciate that the steps of any of the methods discussed herein may be adjusted, rearranged, expanded, omitted, or modified in various ways without departing from the scope of the present disclosure. Further, although aspects of the method 200 are described using the washing machine apparatus 100 as an example, it should be appreciated that the methods may be applied to the operation of any suitable washing machine apparatus.
This written description uses examples to disclose the invention, including the best mode, and also to enable any person skilled in the art to practice the invention, including making and using any devices or systems and performing any incorporated methods. The patentable scope of the invention is defined by the claims and may include other examples that occur to those skilled in the art. Such other examples are intended to be within the scope of the claims if they include structural elements that do not differ from the literal language of the claims, or if they include equivalent structural elements with insubstantial differences from the literal languages of the claims.

Claims (16)

1. A washing machine apparatus comprising:
a washing tub positioned within the cabinet and defining a washing chamber;
a wash basket rotatably mounted within the wash tub and configured to receive a quantity of items to be washed;
a motor operatively coupled to the basket to selectively rotate the basket;
a microphone for monitoring sounds generated during operation of the washing machine device; and
a controller operatively coupled to the microphone, the controller configured to:
obtaining a sound signal generated during operation of the washing machine device using the microphone;
generating a spectrogram from the sound signal, the spectrogram representing a sound amplitude and a sound frequency over time;
validating a voice signature by analyzing the spectrogram using an image recognition program; and
adjusting at least one operating parameter of the washing machine device based at least in part on the confirmation of the sound signature;
the sound signature is associated with sound generated from at least one of a bearing, a belt, a motor, a water valve, a pump, a suspension system, harmonics of a structural component, or unwanted contact between components or subsystems.
2. The washing machine device of claim 1, wherein the image recognition program uses artificial intelligence to analyze the spectrogram.
3. The washing machine device of claim 1, wherein the image recognition program comprises a convolutional neural network.
4. The laundry machine apparatus of claim 1, wherein the controller is further configured to:
a plurality of sound signatures associated with various operating conditions are learned.
5. The laundry machine apparatus of claim 1, wherein adjusting the at least one operating parameter comprises:
adjusting agitation time or profile, adjusting water level, limiting spin rate, confirming service demand, or providing operational guidance to the user.
6. The laundry machine apparatus of claim 1, wherein adjusting the at least one operating parameter comprises:
an operating period is selected based on the sound signature.
7. The laundry machine apparatus of claim 1, wherein the controller is further configured to:
a user notification is provided when the voice signature indicates that a predetermined operational characteristic exists.
8. The laundry machine apparatus of claim 1, wherein the sound signature is associated with the presence of non-washable articles, and wherein adjusting the at least one operating parameter comprises stopping a wash cycle.
9. The laundry machine apparatus of claim 1, wherein the controller is further configured to:
transmitting the spectrogram to a remote server for analysis; and
analysis feedback from the remote server is received.
10. The laundry machine apparatus of claim 1, wherein the microphone is positioned outside the cabinet and remote from the laundry machine apparatus.
11. A method of operating a washing machine apparatus employing a washing machine apparatus as claimed in any one of claims 1 to 10, wherein the method comprises:
obtaining a sound signal generated during operation of the washing machine device using the microphone;
generating a spectrogram from the sound signal, the spectrogram representing a sound amplitude and a sound frequency over time;
validating a voice signature by analyzing the spectrogram using an image recognition program; and
at least one operating parameter of the washing machine device is adjusted based at least in part on the validation of the voice signature.
12. The method of claim 11, wherein the image recognition program uses artificial intelligence to analyze the spectrogram.
13. The method of claim 11, wherein the image recognition program comprises a convolutional neural network.
14. The method of claim 11, further comprising:
a plurality of sound signatures associated with various operating conditions are learned.
15. The method of claim 11, wherein the sound signature is associated with the presence of a non-washable article, and wherein adjusting the at least one operating parameter comprises stopping a wash cycle.
16. The method of claim 11, further comprising:
transmitting the spectrogram to a remote server for analysis; and
analysis feedback from the remote server is received.
CN202180019122.3A 2020-03-03 2021-03-23 System and method for monitoring operation of washing machine device using sound Active CN115398057B (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US16/807,269 US20210277564A1 (en) 2020-03-03 2020-03-03 System and method for using sound to monitor the operation of a washing machine appliance
PCT/CN2021/082469 WO2021175336A1 (en) 2020-03-03 2021-03-23 System and method for using sound to monitor the operation of a washing machine appliance

Publications (2)

Publication Number Publication Date
CN115398057A CN115398057A (en) 2022-11-25
CN115398057B true CN115398057B (en) 2024-01-30

Family

ID=77555569

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202180019122.3A Active CN115398057B (en) 2020-03-03 2021-03-23 System and method for monitoring operation of washing machine device using sound

Country Status (3)

Country Link
US (1) US20210277564A1 (en)
CN (1) CN115398057B (en)
WO (1) WO2021175336A1 (en)

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CA3115423A1 (en) * 2020-05-01 2021-11-01 Systemes De Controle Actif Soft Db Inc. A system and a method for sound recognition
US11802894B2 (en) * 2020-09-17 2023-10-31 Silicon Laboratories Inc. Compressing information in an end node using an autoencoder neural network
US11692301B2 (en) * 2020-10-16 2023-07-04 Haier Us Appliance Solutions, Inc. System and method for using sound to monitor the operation of a dryer appliance
EP4248226A1 (en) * 2020-11-17 2023-09-27 Alliance Laundry Systems LLC Monitoring laundry machine operation using machine learning analysis of acoustic transducer signal information
CN114488861B (en) * 2022-01-24 2024-08-20 杭州老板电器股份有限公司 Cooking parameter adjustment method and device and electronic equipment

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5842288A (en) * 1996-12-10 1998-12-01 U.S. Controls Corporation Clothes dryer with chiming alarm
JP2007144208A (en) * 1995-01-17 2007-06-14 Hitachi Ltd Domestic electric appliance
CN101970744A (en) * 2008-03-11 2011-02-09 Bsh博世和西门子家用器具有限公司 Laundry treatment appliance comprising a drum and a sensor, and method for the operation thereof
CN102547543A (en) * 2010-12-10 2012-07-04 杨国屏 Method for improving accuracy of sound heard by hearing-impaired listener and hearing aid
CN106868796A (en) * 2017-03-09 2017-06-20 青岛海尔智能技术研发有限公司 A kind of washing machine noise reduction method and washing machine
CN108642802A (en) * 2018-04-02 2018-10-12 青岛海尔洗衣机有限公司 A kind of control method and washing machine of washing machine

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2007144208A (en) * 1995-01-17 2007-06-14 Hitachi Ltd Domestic electric appliance
US5842288A (en) * 1996-12-10 1998-12-01 U.S. Controls Corporation Clothes dryer with chiming alarm
CN101970744A (en) * 2008-03-11 2011-02-09 Bsh博世和西门子家用器具有限公司 Laundry treatment appliance comprising a drum and a sensor, and method for the operation thereof
CN102547543A (en) * 2010-12-10 2012-07-04 杨国屏 Method for improving accuracy of sound heard by hearing-impaired listener and hearing aid
CN106868796A (en) * 2017-03-09 2017-06-20 青岛海尔智能技术研发有限公司 A kind of washing machine noise reduction method and washing machine
CN108642802A (en) * 2018-04-02 2018-10-12 青岛海尔洗衣机有限公司 A kind of control method and washing machine of washing machine

Also Published As

Publication number Publication date
WO2021175336A1 (en) 2021-09-10
US20210277564A1 (en) 2021-09-09
CN115398057A (en) 2022-11-25

Similar Documents

Publication Publication Date Title
CN115398057B (en) System and method for monitoring operation of washing machine device using sound
KR101709483B1 (en) Home appliance and operating method thereof
KR102652592B1 (en) Washing machine and Method for controlling the same
US20130185079A1 (en) Home appliance, home appliance system, and method for operating same
US10870940B2 (en) Washing machine appliance and methods for moisture removal
CN102888735B (en) Laundering device vibration control
US11718945B2 (en) Laundry treating apparatus having camera, and control method thereof
CN108625092A (en) Wash drying device and control method
US11692301B2 (en) System and method for using sound to monitor the operation of a dryer appliance
JP7245725B2 (en) Washing machine and washing machine abnormality diagnosis method
US20150059416A1 (en) Systems and Methods for Detecting an Imbalanced Load in a Washing Machine Appliance Having a Balancing Apparatus
US20150368840A1 (en) Cleaning systems and methods for operating the same
US20230124027A1 (en) Systems and methods for capturing images for use in artificial intelligence processes in a laundry appliance
CN103088601B (en) Roller washing machine and control method
US20230366139A1 (en) Automatic self-clean cycle prior to drying cycle in a laundry appliance
US11053619B2 (en) Smart auto wash cycle for a washing machine appliance
KR102627368B1 (en) A washing machine and method for determining a washing of a tub of the washing machine, and a server and a program therefor
US7131159B2 (en) Method of controlling wobbling washing machine
US12006610B2 (en) Automatic drying cycle after execution of self clean cycle in a laundry appliance
US20220364290A1 (en) Adaptive self clean cycle in a laundry appliance
CN203144743U (en) Drum washing machine
US12000076B2 (en) Front load washer with locking bolt removal confirmation
US20230357976A1 (en) Secure remote testing of washing machine appliances
CN118110000A (en) Method and device for detecting running condition of washing machine, storage medium and electronic device
US20240133100A1 (en) Laundry treatment appliance and method of using the same according to matched laundry loads

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant