WO2023197887A1

WO2023197887A1 - Intelligent control method for starting washing machine and device thereof

Info

Publication number: WO2023197887A1
Application number: PCT/CN2023/085390
Authority: WO
Inventors: 李世成; 黄振兴; 许升
Original assignee: 青岛胶南海尔洗衣机有限公司; 海尔智家股份有限公司
Priority date: 2022-04-13
Filing date: 2023-03-31
Publication date: 2023-10-19
Also published as: CN116949741A

Abstract

The present invention relates to the technical field of washing machines. Particularly provided are an intelligent control method for starting a washing machine and a device thereof, which are aimed at solving the technical problem of low control accuracy of existing methods for controlling the start of a washing machine on the basis of man-machine distance and human poses. For the purpose, the intelligent control method for starting a washing machine of the present invention comprises the following steps: acquiring a distance between a user and a washing machine; judging whether the distance between the user and the washing machine is smaller than a preset distance; when the distance between the user and the washing machine is smaller than the preset distance, determining whether the user has the intention to wash clothes; and, when the user has the intention to wash clothes, controlling to start the washing machine. Thus, accurate control of a washing machine is achieved.

Description

Intelligent control method and device for starting a washing machine

This application claims priority to Chinese patent application CN202210386275.8, which was submitted on April 13, 2022 and has an invention title of "Intelligent control method and device for starting a washing machine". The entire content of the above Chinese patent application is incorporated into this application by reference. middle.

Technical field

The present invention relates to the technical field of washing machines, and specifically provides an intelligent control method and device for starting a washing machine.

Background technique

As the level of science and technology continues to improve, more and more smart products have entered people's daily lives. Some electrical equipment in the prior art rely on detecting the distance between humans and machines to perform preset operations, and perform preset operations when a person reaches a set area around the electrical equipment. For example, when a washing machine opens the door automatically when putting clothes in, the existing technology uses the distance between the human and the machine and the posture of the human body to control the opening of the machine door. When the distance between the human and the machine is detected to be less than the preset distance threshold, it is recognized that the person intends to do laundry, and then the control The laundry discharge door of the washing machine is open. The accuracy of this method is low. If the user just passes by the washing machine, it will be recognized that the user intends to do laundry, causing the washing machine door to open accidentally.

Accordingly, a new intelligent control solution for washing machine startup is needed in this field to solve the above problems.

Contents of the invention

In order to overcome the above defects, the present invention is proposed to solve or at least partially solve the technical problem of low control accuracy in the existing method of controlling the startup of the washing machine based on the distance between the human and the machine and the posture of the human body. The invention provides an intelligent control method and device for starting a washing machine.

In a first aspect, the present invention provides an intelligent control method for starting a washing machine, including the following steps: obtaining the distance between the user and the washing machine; determining the distance between the user and the washing machine. Whether the distance is less than a preset distance; if the distance between the user and the washing machine is less than the preset distance, determine whether the user has an intention to do laundry; and if the user has an intention to do laundry, control the turning on of the washing machine.

In one embodiment, determining whether the user intends to do laundry includes: collecting video information containing the user; determining the user's hand area image based on the video information; constructing a target detection model; using the target detection model to detect the user's hand area. Area images are recognized to determine whether the user has laundry intention.

In one embodiment, determining the user's hand area image based on the video information includes: determining the user's hand area position of each video frame image in the video information based on the video information; based on the user's hand area position Obtain the user's hand area image from the video frame image.

In one embodiment, the target detection model adopts the FasterRCNN+ResNet50+FPN model; building the target detection model includes: the convolution module part between the input end and the intermediate end feature map in the FasterRCNN detection network model adopts the ResNet50 network framework The model is replaced by cross-combination with the FPN feature pyramid network model. The image is input to the input layer of the ResNet50 network framework model, and the outputs from the second to fifth feature extraction stages of the ResNet50 network framework model are connected to the FPN feature pyramid network. As input to the model, the FPN feature pyramid network model interpolates and fuses the stage feature maps output from the second feature extraction stage to the fifth feature extraction stage of the ResNet50 network framework model and outputs feature maps of different scales.

In one embodiment, whether the user has the intention to do laundry includes whether the user holds at least one of clothes and clothing appliances in his hands; the target detection model is used to identify the user's hand area image to determine whether the user has the intention to do laundry. The intention to do laundry includes: inputting the user's hand area image into the target detection model to obtain at least one of whether the user holds clothes or appliances containing clothes.

In one embodiment, determining whether the user intends to do laundry further includes: collecting video information containing the user; sending each video frame image in the video information containing the user to a cloud server to determine the user based on the cloud server Whether the user has the intention to do laundry; receiving the information sent by the cloud server as to whether the user has the intention to do laundry.

In one embodiment, the method further includes: when the user intends to do laundry, control all Open the door of the washing machine while turning on the washing machine.

In a second aspect, the present invention provides an intelligent control device for starting a washing machine, including: an acquisition module configured to obtain the distance between the user and the washing machine; and a determination module configured to determine the distance between the user and the washing machine. Whether the distance between the user and the washing machine is less than the preset distance; the determination module is configured to determine whether the user has the intention to do laundry when the distance between the user and the washing machine is less than the preset distance; and the control module is configured to determine whether the user has the intention to do laundry. In this case, control the washing machine to start.

In a third aspect, a washing device is provided. The washing device includes a processor and a storage device, the storage device is adapted to store a plurality of program codes, and the program codes are adapted to be loaded and run by the processor to execute the foregoing. The intelligent control method for starting a washing machine according to any one of the above.

In a fourth aspect, a computer-readable storage medium is provided in which a plurality of program codes are stored, and the program codes are adapted to be loaded and run by a processor to execute any one of the aforementioned washing machines. Intelligent control method for power-on.

One or more of the above technical solutions of the present invention have at least one or more of the following beneficial effects:

The invention provides an intelligent control method for starting a washing machine. First, the distance between the user and the washing machine is obtained, and then it is determined whether the distance between the user and the washing machine is less than the preset distance. If the distance between the user and the washing machine is less than the preset distance, In this case, it is determined whether the user has the intention of washing, and if the user has the intention of washing, the washing machine is controlled to start. In this way, when the human-machine distance is within the preset range, the washing machine is further controlled to start based on whether the user has the intention of washing. , thereby achieving precise control of the washing machine and improving the user experience.

First, the video frame images in the video information are preprocessed to obtain the user's hand area image, and then the hand area image is input into the built target detection model for recognition, thereby determining whether the user has laundry purposes. In this way, when the human-machine distance is within the preset range, the user's hand area is further detected to determine whether the user has laundry intention, which improves the accuracy of determining whether the user has laundry intention.

Description of the drawings

The disclosure of the present invention will become more understandable with reference to the accompanying drawings. Personnel skilled in the field It should be understood that these drawings are for illustrative purposes only and are not intended to limit the scope of the present invention. Additionally, like numbers in the figures identify similar parts, where:

Figure 1 is a schematic flowchart of the main steps of an intelligent control method for starting a washing machine according to an embodiment of the present invention;

Figure 2 is a schematic flowchart of the complete steps of an intelligent control method for starting a washing machine according to one embodiment of the present invention;

Figure 3 is a schematic block diagram of the main structure of an intelligent control device for starting a washing machine according to an embodiment of the present invention.

List of reference signs :
11: Acquisition module; 12: Judgment module; 13: Determination module; 14: Control module.

Detailed ways

Some embodiments of the invention are described below with reference to the accompanying drawings. Those skilled in the art should understand that these embodiments are only used to explain the technical principles of the present invention and are not intended to limit the scope of the present invention.

In the description of the present invention, "module" and "processor" may include hardware, software, or a combination of both. A module can include hardware circuits, various suitable sensors, communication ports, and memory. It can also include software parts, such as program code, or it can be a combination of software and hardware. The processor may be a central processing unit, a microprocessor, an image processor, a digital signal processor, or any other suitable processor. The processor has data and/or signal processing functions. The processor can be implemented in software, hardware, or a combination of both. Non-transitory computer-readable storage media include any suitable media that can store program code, such as magnetic disks, hard disks, optical disks, flash memory, read-only memory, random access memory, etc. The term "A and/or B" means all possible combinations of A and B, such as just A, just B, or A and B. The terms "at least one A or B" or "at least one of A and B" have a similar meaning to "A and/or B" and may include just A, just B or A and B. The singular forms "a," "the" and "the" may also include the plural form.

At present, the distance between the human and the machine and the posture of the human body are traditionally used to control the opening of the machine door. When the distance between the human and the machine is detected to be less than the preset distance threshold, it is recognized that the person intends to do laundry, and then the clothes throwing door of the washing machine is controlled to open. The accuracy of this method is relatively low. If the user just passes The washing machine will also be recognized as the user's intention to do laundry, causing the washing machine door to open accidentally. To this end, this application proposes an intelligent control method and device for starting a washing machine. First, the distance between the user and the washing machine is obtained, and then it is determined whether the distance between the user and the washing machine is less than the preset distance. When the distance is less than the preset distance, it is determined whether the user has the intention to do laundry, and when the user has the intention to do laundry, the washing machine is controlled to turn on. In this way, when the distance between the human and the machine is within the preset range, the system determines whether the user has the intention to do laundry. Use it for laundry purposes to further control the startup of the washing machine, thereby achieving precise control of the washing machine and improving the user experience.

Referring to FIG. 1 , FIG. 1 is a schematic flowchart of the main steps of an intelligent control method for starting a washing machine according to an embodiment of the present invention. As shown in Figure 1, the intelligent control method for starting up a washing machine in the embodiment of the present invention mainly includes the following steps S101 to S104.

Step S101: Obtain the distance between the user and the washing machine. Specifically, the distance between the user and the washing machine device can be obtained through a sensor provided on the washing machine body. In this step, the sensor may also collect the distance between the user and the washing machine equipment multiple times, and then average the distances between multiple users and the washing machine equipment to obtain the final distance between the user and the washing machine. Thereby reducing the measurement error and further improving the control efficiency of the washing machine.

Step S102: Determine whether the distance between the user and the washing machine is less than a preset distance. Specifically, the preset distance may be a distance measured in advance through experiments. When the application determines that the distance between the user and the washing machine is less than the preset distance, the following step S103 is executed, and when it is determined that the distance between the user and the washing machine is not less than the preset distance, step S101 is continued.

Step S103: When the distance between the user and the washing machine is less than the preset distance, determine whether the user has intention to do laundry. Specifically, the way to determine whether the user intends to do laundry may be to collect the voice instructions input by the user. For example, when the voice instructions input by the user such as "washing" or "turn on the washing machine" are collected, it is determined that the user has the intention to do laundry. In addition to the above methods, this application mainly determines whether the user intends to do laundry by determining whether the user has at least one of clothes and an appliance containing clothes. For example, when it is determined that the user holds clothes or an appliance containing clothes in his hand, it is determined that the user has the intention to do laundry.

Step S104: When the user intends to do laundry, control the washing machine to start.

Based on the above steps S101 to S104, first obtain the distance between the user and the washing machine, and then determine whether the distance between the user and the washing machine is less than the preset distance. When the distance between the human and the machine is less than the preset distance, it is determined whether the user has the intention to do laundry, and when the user has the intention to do laundry, the washing machine is controlled to turn on. In this way, when the distance between the human and the machine is within the preset range, the system determines whether the user has the intention to do laundry. Whether there is a laundry purpose to further control the startup of the washing machine, thereby achieving precise control of the washing machine and improving the user experience.

In one embodiment, determining whether the user intends to do laundry includes: collecting video information containing the user; determining the user's hand area image based on the video information; constructing a target detection model; using the target detection model to identify the user's hand area image to Determine if the user has laundry intention. Specifically, a camera is provided on the main body of the washing machine, and the camera can collect video information including the user in real time. In this application, the video frame images in the video information are first pre-processed to obtain the user's hand area image, and then the hand area image is input into the built target detection model for recognition, thereby determining whether the user has laundry purposes. In this way, when the human-machine distance is within the preset range, the user's hand area is further detected to determine whether the user has laundry intention, which improves the accuracy of determining whether the user has laundry intention.

In one embodiment, determining the user's hand area image based on the video information includes: determining the user's hand area position of each video frame image in the video information based on the video information; obtaining the user's hand area from the video frame image based on the user's hand area position. area image. Specifically, in the process of determining the user's hand area image based on video information, first the user's hand area position in the video frame image is determined, and then the image containing the user's hand area position is extracted from the video frame image. Out, the image of the user's hand area can be obtained. In this way, compared to directly using video frame images to identify whether the user is holding clothes or appliances with clothes in his hands, using images of the user's hand area to identify whether the user is holding clothes or appliances with clothes is more accurate. , which can further improve the recognition efficiency.

In one embodiment, the target detection model adopts the FasterRCNN+ResNet50+FPN model; building the target detection model includes: the convolution module part between the input end and the intermediate end feature map in the FasterRCNN detection network model adopts the ResNet50 network framework model and The FPN feature pyramid network model is replaced by cross-combination. The image is input to the input layer of the ResNet50 network frame model, and the outputs from the second feature extraction stage to the fifth feature extraction stage of the ResNet50 network frame model are connected to the FPN feature pyramid network model. Input, the FPN feature pyramid network model interpolates and fuses the stage feature maps output from the second feature extraction stage to the fifth feature extraction stage of the ResNet50 network framework model and outputs features of different scales. picture. Specifically, after the target detection model is constructed, the collected historical user hand area image sample set can be used to train it. Specifically, the historical user hand area image sample set can be divided into a training set and a test set according to a preset ratio. Train the target detection model.

This application uses ResNet50 as the network framework and further combines the FasterRCNN detection network model and the FPN feature pyramid network model to solve the problem of accuracy saturation degradation when the depth of the deep learning network increases, which can improve the training speed of the model and improve the prediction performance of the model. It can also It achieves the fusion of shallow geometric information and deep semantic information of the image, improving the detection effect of the model. In addition, the FasterRCNN+ResNet50+FPN model is used as a target detection model to identify the user's hand area image to determine whether the user has the intention to do laundry, further improving the recognition accuracy and efficiency.

In one embodiment, whether the user has an intention to do laundry includes whether the user holds at least one of clothes and appliances containing clothes in his hands; using a target detection model to identify the image of the user's hand area to determine whether the user has an intention to do laundry includes: The image of the user's hand area is input into the target detection model to obtain whether the user holds at least one of clothing and clothing equipment. Specifically, when the user holds at least one of clothes and clothes-installed appliances in his hands, it means that the user has laundry purposes. Specifically, the target detection model trained in the previous embodiment can be used to identify the user's hand area to identify whether the user is holding at least one of clothing and clothing equipment. When it is identified whether the user is holding clothing and clothing equipment, When there is at least one type of clothing appliance, it is determined that the user has laundry purpose.

In one embodiment, determining whether the user has an intention to do laundry further includes: collecting video information containing the user; and sending each video frame image in the video information containing the user to a cloud server to determine whether the user has an intention to do laundry based on the cloud server. ; Receive information sent by the cloud server as to whether the user intends to do laundry. In this implementation, the cloud server mainly determines whether the user has the intention to do laundry. Specifically, each video frame image in the video frame information collected by the camera is sent to the cloud server through the network, and the cloud server analyzes the video frame image. Analysis is performed to determine whether the user has the intention to do laundry. Regardless of whether the cloud server determines whether the user has the intention to do laundry, the result of whether the user has the intention to do laundry is sent to the control device. The cloud server is used to determine whether the user has the intention to do laundry. In this way, the load of the washing machine can be reduced.

In one embodiment, the method further includes: when the user intends to do laundry, control the While turning on the machine, open the door of the washing machine. Specifically, when it is determined that the user intends to do laundry, the washing machine can be controlled to start and the door of the washing machine can be opened at the same time, thereby reducing user operations and improving the user experience.

In addition, in another embodiment, as shown in Figure 2, the distance between the user and the device is first obtained, and based on this, it is determined whether the preset distance is reached. If so, the user's intention is recognized based on the image to determine whether the user has Laundry intention, and when the user has laundry intention, control the laundry equipment to turn on and open the washing machine door. In this way, the accuracy of washing machine control is further improved.

It should be pointed out that although the various steps are described in a specific order in the above embodiments, those skilled in the art can understand that in order to achieve the effects of the present invention, different steps do not have to be executed in such an order. They can be executed simultaneously (in parallel) or in other sequences, and these changes are within the scope of the present invention.

Furthermore, the present invention also provides an intelligent control device for starting a washing machine.

Referring to FIG. 3, FIG. 3 is a main structural block diagram of an intelligent control device for starting a washing machine according to one embodiment of the present invention. As shown in FIG. 3 , the intelligent control device for starting the washing machine in the embodiment of the present invention mainly includes an acquisition module 11 , a judgment module 12 , a determination module 13 and a control module 14 . In some embodiments, one or more of the acquisition module 11, the judgment module 12, the determination module 13 and the control module 14 can be combined into one module. In some embodiments, the acquisition module 11 may be configured to acquire the distance between the user and the washing machine. The determination module 12 may be configured to determine whether the distance between the user and the washing machine is less than a preset distance. The determination module 13 may be configured to determine whether the user has the intention to do laundry when the distance between the user and the washing machine is less than a preset distance. The control module 14 may be configured to control the washing machine to start when the user intends to do laundry. In one implementation, for a description of the specific implementation of the function, please refer to steps S101 to S104.

In the intelligent control device for starting the washing machine of this application, the acquisition module first obtains the distance between the user and the washing machine, and then the judgment module determines whether the distance between the user and the washing machine is less than the preset distance. When the distance between the user and the washing machine is less than In the case of a preset distance, the determination module determines whether the user has the intention to do laundry, and when the user has the intention to do laundry, the control module controls the starting of the washing machine. In this way, when the human-machine distance is within the preset range, according to Whether the user has laundry purposes, he can further control the startup of the washing machine, thereby realizing laundry control. The precise control of the machine improves the user experience.

The above-mentioned intelligent control device for starting a washing machine is used to execute the embodiment of the intelligent control method for starting a washing machine shown in Figure 1. The technical principles, technical problems solved and technical effects produced by the two are similar, and those skilled in the art can clearly understand It is understood that for the convenience and simplicity of description, the specific working process and related instructions of the intelligent control device for starting the washing machine can be referred to the content described in the embodiment of the intelligent control method for starting the washing machine, and will not be repeated here.

Those skilled in the art can understand that the present invention can implement all or part of the process in the method of the above-mentioned embodiment, and can also be completed by instructing relevant hardware through a computer program. The computer program can be stored in a computer-readable file. In the storage medium, when the computer program is executed by the processor, the steps of each of the above method embodiments can be implemented. Wherein, the computer program includes computer program code, which may be in the form of source code, object code, executable file or some intermediate form. The computer-readable storage medium may include: any entity or device capable of carrying the computer program code, media, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory, random access memory, electrical carrier wave signals, telecommunications signals, and software distribution media, etc. It should be noted that the content contained in the computer-readable storage medium can be appropriately added or deleted according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable storage media Storage media does not include electrical carrier signals and telecommunications signals.

Furthermore, the present invention also provides a washing equipment. In an embodiment of the washing equipment according to the present invention, the washing equipment includes a processor and a storage device. The storage device can be configured to store a program for executing the intelligent control method for starting the washing machine in the above method embodiment. The processor can be configured to use In executing the program in the storage device, the program includes but is not limited to the program that executes the intelligent control method for starting the washing machine in the above method embodiment. For ease of explanation, only the parts related to the embodiments of the present invention are shown. If specific technical details are not disclosed, please refer to the method part of the embodiments of the present invention.

Furthermore, the present invention also provides a computer-readable storage medium. In one embodiment of the computer-readable storage medium according to the present invention, the computer-readable storage medium can be configured to store a program that executes the intelligent control method for starting the washing machine of the above method embodiment. The program can be loaded and run by the processor to Implement the above intelligent control method for starting the washing machine. for For ease of explanation, only the parts related to the embodiments of the present invention are shown. If the specific technical details are not disclosed, please refer to the method part of the embodiments of the present invention. The computer-readable storage medium may be a storage device formed by various electronic devices. Optionally, in the embodiment of the present invention, the computer-readable storage medium is a non-transitory computer-readable storage medium.

Further, it should be understood that since the settings of each module are only to illustrate the functional units of the device of the present invention, the physical devices corresponding to these modules may be the processor itself, or a part of the software in the processor, a part of the hardware, or Part of the combination of software and hardware. Therefore, the number of individual modules in the figure is only illustrative.

Those skilled in the art can understand that various modules in the device can be split or combined adaptively. Such splitting or merging of specific modules will not cause the technical solutions to deviate from the principles of the present invention. Therefore, the technical solutions after splitting or merging will fall within the protection scope of the present invention.

So far, the technical solution of the present invention has been described with reference to the preferred embodiments shown in the drawings. However, those skilled in the art can easily understand that the protection scope of the present invention is obviously not limited to these specific embodiments. Without departing from the principles of the present invention, those skilled in the art can make equivalent changes or substitutions to relevant technical features, and technical solutions after these modifications or substitutions will fall within the protection scope of the present invention.

Claims

An intelligent control method for starting a washing machine, which is characterized by including the following steps:

Get the distance between the user and the washing machine;

Determine whether the distance between the user and the washing machine is less than a preset distance;

When the distance between the user and the washing machine is less than the preset distance, determine whether the user intends to do laundry; and

When the user intends to do laundry, the washing machine is controlled to start.
The intelligent control method for starting up a washing machine according to claim 1, wherein determining whether the user intends to do laundry includes:

Collect video information containing users;

Determine the user's hand area image based on the video information;

Build a target detection model;

The target detection model is used to identify the user's hand area image to determine whether the user intends to do laundry.
The intelligent control method for starting a washing machine according to claim 2, wherein determining the user's hand area image based on the video information includes:

Determine the user's hand area position of each video frame image in the video information based on the video information;

A user's hand area image is obtained from the video frame image based on the user's hand area position.
The intelligent control method for starting up a washing machine according to claim 2, characterized in that the target detection model adopts the FasterRCNN+ResNet50+FPN model; building the target detection model includes: detecting characteristics from the input end to the intermediate end in the FasterRCNN network model The convolution module part between the images is replaced by a cross combination of the ResNet50 network frame model and the FPN feature pyramid network model. The image is input to the input layer of the ResNet50 network frame model, and the second feature extraction stage of the ResNet50 network frame model is moved to the fifth The output of the feature extraction stage The outputs are all connected to the input of the FPN feature pyramid network model. The FPN feature pyramid network model interpolates and fuses the stage feature maps output from the second feature extraction stage to the fifth feature extraction stage of the ResNet50 network framework model and outputs feature maps of different scales.
The intelligent control method for starting a washing machine according to claim 2, wherein whether the user intends to do laundry includes at least one of whether the user holds clothes and appliances containing clothes; and the target detection model is used to detect the user's intention to do laundry. Recognizing the hand area image to determine whether the user has the intention to do laundry includes inputting the user's hand area image into the target detection model to obtain at least one of whether the user holds clothes and clothes appliances in his hands.
The intelligent control method for starting up a washing machine according to claim 1, wherein determining whether the user intends to do laundry further includes:

Collect video information containing users;

Send each video frame image in the video information containing the user to the cloud server to determine whether the user has the intention to do laundry based on the cloud server;

Receive information sent by the cloud server as to whether the user intends to do laundry.
The intelligent control method for starting up a washing machine according to claim 1, further comprising: when the user intends to do laundry, controlling the starting of the washing machine and opening the door of the washing machine at the same time.
An intelligent control device for starting a washing machine, which is characterized by including:

an acquisition module configured to acquire the distance between the user and the washing machine;

a judgment module configured to judge whether the distance between the user and the washing machine is less than a preset distance;

a determination module configured to determine whether the user has an intention to do laundry when the distance between the user and the washing machine is less than a preset distance; and

The control module is configured to control the starting of the washing machine when the user intends to do laundry.
A washing equipment, including a processor and a storage device, the storage device is adapted to store a plurality of program codes, characterized in that the program codes are adapted to be loaded and run by the processor to execute claims 1 to 7 The intelligent control method for starting a washing machine according to any one of the above.
A computer-readable storage medium in which a plurality of program codes are stored, characterized in that the program codes are adapted to be loaded and run by a processor to execute the smart start-up function of the washing machine according to any one of claims 1 to 7. Control Method.