KR102065424B1 - Artificial intelligence laundry treating machine providing an automatic washing course based on a vision - Google Patents

Abstract

The present invention relates to an artificial intelligence clothing treatment machine which is a home appliance with artificial intelligence (AI) applied thereto for receiving and treating an object and, more specifically, to an AI clothing treatment machine in which clothing is hung and treated. According to one embodiment of the present invention, the AI clothing treatment machine comprises: a cabinet; a clothing reception unit disposed in the cabinet to hold clothing to be treated thereon; a door disposed in the cabinet to open/close the clothing reception unit; a camera disposed in the cabinet or the door to generate image information of the clothing held in the clothing reception unit; a user interface disposed on the front surface of the door and receiving information (treatment information) about clothing treatment from a user; and a main processor controlling treatment of the object based on current treatment information set through the acquired current image information, and a result of a learning process through the acquired image information and the acquired treatment information. The present invention can set optimal treatment information.

Description

Artificial intelligence laundry treating machine providing an automatic washing course based on a vision}

The present invention relates to a home appliance in which artificial intelligence is implemented as a home appliance for accommodating an object and processing the received object, and more particularly, to an artificial intelligent clothing manager for hanging and processing clothes.

There are many types of home appliances that accept and process objects. That is, various kinds of home appliances may be provided according to the object and the method of processing the object.

Appliances that receive clothing and process the clothing may include a washing machine, a dryer, and a styler (clothing processor or clothing manager). The clothing may include various objects such as bedding, textile dolls, shoes as well as clothes worn by the user.

The treatment of clothes may include various treatments such as washing, drying, refreshing, and sterilization, and any one of the clothes treating apparatuses may be provided to be specific to a specific treatment among various treatments, and to perform a plurality of treatments. Can be provided.

In the case of a washing machine, washing and rinsing using the washing water may be performed, and a washing course in which dehydration is performed may be provided.

In the case of a dryer, a drying or refreshing course may be provided in which drying or refreshing is performed using hot air.

Such clothes treating apparatuses are provided with various treatment courses according to the type of object, and may include at least one or more sub-information in each treatment course.

Such home appliances may be extended to a dishwasher for processing tableware or a cooking appliance for cooking food through ingredients. The dishwasher or the ingredients may also be processed through various treatment courses depending on the object to be processed or according to the user's intention.

In a conventional home appliance, it is common for a user to input processing information after inputting an object to an object receiving unit, and the object is processed according to the input processing information.

In the case of a washing machine, after putting clothes for performing washing into an object receiving unit such as a drum, a user selects a specific washing course among various washing courses, and washing is generally performed according to the selected washing course.

For example, the user may select a "quilt course" when washing bedding, such as a blanket, or "wool course" when washing clothes of wool material.

It is common to provide a plurality of processing information for the object provided by the home appliance. For example, processing information is provided in specific course types such as a washing course, a drying course, a refreshing course, a washing course, and a cooking course according to the type of home appliances, and a plurality of processing informations are generally provided. In the case of a washing machine, for example, a plurality of processing information may be provided by a single household appliance such as a standard (general) course, a baby clothes course, a duvet course, and a functional clothes course.

Each of the processing information or the course information is provided so that the optimum processing is performed according to the type of processing object. That is, it can be said that a series of object treatment procedures and treatment conditions are set in advance so as to satisfy an optimum treatment result while preventing damage or damage to the object to be treated.

In the case of the washing machine, although various processing information is provided according to the type of clothes to be processed, the user often uses only the processing information selected by default. Therefore, there is a problem in that the washing machine cannot be used efficiently.

In addition, whenever the user wants to wash through the washing machine, a lot of effort is required to select and input processing information.

Recently, a clothes processor or a clothes manager that hangs clothes inside a cabinet and processes clothes is widely used. Unlike washing with washing water, the clothing manager may be a home appliance that refreshes or sterilizes clothes by supplying hot air, cold air, or steam to the clothes. Clothing managers can also be used to remove fine first or dry clothes wet with rain. For this reason, such a clothing manager may be referred to in various terms such as a refresher, a styler, and a clothing cleaner.

This clothing manager is provided to process a small amount of clothing. For example, it is provided to process one to three pieces of clothing, the clothing is located inside the object receiving portion in a state caught on the hanger. Of course, clothing such as a towel may be located inside the receiving portion in a well-opened state.

Most of the clothes managed through the clothes manager are clothes that are difficult to wash with water and are relatively expensive clothes. Therefore, the optimal management course should be set according to the clothing.

Recently, artificial intelligence has become a hot topic in home appliances. However, in the current home appliances, artificial intelligence is only about providing specific information to the user based on the data accumulated in the home appliances. Strictly speaking, it's not just artificial intelligence but data processing.

Therefore, it is necessary to provide a clothing manager to which artificial intelligence has a form in which learning is performed rather than simple cumulative data processing. Through this, the garment manager may automatically provide the optimal processing information without the user inputting the processing information one by one.

In particular, since the type and amount of the clothing processed through the clothing manager of the user's clothing is limited, it may be necessary to provide a clothing manager that is easy to use by learning and reflecting user personal characteristics.

Through one embodiment of the present invention, learning is performed to provide a clothing manager and a control method thereof that can set or set the optimal processing information without the user inputting the processing information.

Through one embodiment of the present invention, to provide a clothing manager and a control method thereof that can be easily performed by the user to manually input the processing information and to automatically set the processing information in the clothing manager.

Through one embodiment of the present invention, to provide a clothing manager and a control method thereof that can increase the user's satisfaction by informing the user that learning is being performed and evolving.

In one embodiment of the present invention, in order for the home appliance to learn the correct, it is to provide a clothing manager and a control method capable of performing the user's approval or the injection or forced learning by the user.

Through one embodiment of the present invention, the learning is performed only through the information obtained through a specific clothing manager used by the user, to provide a user-customized clothing manager and its control method. In addition, to minimize the information required for learning to reduce the time required for learning, and to provide a clothing manager and its control method that can simplify the learning process.

According to one embodiment of the present invention, the clothing manager and the control thereof may be performed during the object processing based on the set processing information or during the completion of the process, a correction process for changing the sub information or sub information of the processing information can be performed. To provide a method.

Through an embodiment of the present invention, to provide a clothing manager and a control method thereof that can automatically set the processing information for the object, but affects the object processing.

Through one embodiment of the present invention, through the learning through the image information formed by automatically photographing the object accommodated in the object receiving unit, to provide a clothing manager and its control method that can set the processing information.

Through one embodiment of the present invention, the learning is automatically performed, to provide a clothing manager and a control method thereof that the user can selectively use the learning results.

Through one embodiment of the present invention, it is to provide a clothing manager and a control method thereof that can provide a course setting only for them by learning a user-specific usage pattern.

Through one embodiment of the present invention, to provide a clothes manager and a control method thereof that the user can intuitively and easily determine whether the learning and whether the learning is possible and / or learning is possible.

In order to achieve the above object, according to an embodiment of the present invention, the cabinet; A clothes accommodating part provided in the cabinet to accommodate clothes to be treated; A door provided in the cabinet to open and close the clothing accommodating part; A camera provided in the cabinet or door to generate image information of the garment mounted inside the garment receiving portion; A user interface provided on a front surface of the door and configured to receive information (processing information) about clothes processing from a user; And an artificial garment manager including a main processor controlling the processing of an object based on the result of the learning process through the acquired acquired image information and the acquired acquired processing information and the current processed information set through the currently acquired image information. May be provided.

The AI clothing manager is a clothing manager that learns through image information generated by a camera. Therefore, it can be called a vision-based artificial intelligence clothing manager.

At least one of a speaker for outputting voice, a microphone for receiving voice, a display, and a button or a touch panel for receiving physical contact or pressurization by a user may be included.

And a door sensor for detecting whether the door is closed, and the camera is configured to generate the image information after the power of the clothing manager is applied and the door sensor is closed.

The learning process may be performed using the image information as an input factor and the processing information as an output factor.

The learning process may be performed through a learning processor embedded in the clothing manager or a learning processor of a server communicating with the clothing manager.

The set processing information can be informed to the user through the user interface. The user interface may inform the user of currently performed processing information based on the set processing information. The user may be informed of the remaining time information, the subprocessor currently being executed (eg, whether refresh, drying or deodorization is being performed).

The learning process may include determining whether a learning result corresponding to the currently acquired image information is present.

If there is a corresponding learning result in the determining step, the corresponding processing information may be output and set.

After setting the processing information, an approval step in which the user's approval of the set processing information is performed through the user interface may be performed.

If there is no user's approval in the approval step, a forced learning step of matching processing information allowed by the user with respect to the currently acquired image information may be performed.

If there is no corresponding learning result in the determining step, a forced learning step of matching processing information allowed by the user with respect to the currently acquired image information may be performed.

It is preferable that the number of combinations of image information distinguished from each other and processing information output corresponding to each image information is expanded through the learning process.

The user interface may include a start input unit which the user approves to start the object processing through the set processing information.

It may include a plurality of course information provided to be performed in the clothing manager.

The plurality of course information may include a plurality of course items, and a plurality of sub courses may be provided for each course item. The corresponding sub course may not be provided in the specific course information.

Any one of the plurality of course items may include a learning course for automatically setting the current processing information based on the learning result, and any one of the plurality of course items may be provided so that the user is set only manually. Can be.

The user interface includes a course selector configured to allow a user to select any one of the plurality of course information, and any one of the plurality of course information is automatically set by the current processing information based on the learning result. May be a learning course.

If the learning course is selected immediately before through the course selection unit, the learning course may be currently selected by default.

If a course other than the learning course is selected among the plurality of course information, the AI garment manager, characterized in that a learning process is performed through the selected course with the currently acquired image information.

When there is a learning result for at least one of the plurality of course information, the current processing information may be automatically set.

When there is a learning result for each of the plurality of course information, the current processing information may be automatically set. The plurality of course information may be preset to correspond to a part of all course information provided by the clothes manager.

In order to achieve the above object, according to an embodiment of the present invention, generating and obtaining image information through a camera; Obtaining object processing information set by the user; Learning through the planned acquired image information and object processing information; In addition, a control method of a clothing manager including a step of processing an object based on the current processing information set through the learned result and the currently acquired image information may be provided.

The control method may include a correction step as described above. It may also include an additional image generation step.

Features in the foregoing embodiments may be implemented in combination in other embodiments, unless they are exclusive or inconsistent in other embodiments.

Through one embodiment of the present invention, it is possible to provide a clothing manager and a control method thereof in which learning is performed so that the user can set or set the optimal processing information without inputting the processing information.

According to one embodiment of the present invention, it is possible to provide a clothing manager and a control method thereof, in which a user may manually input processing information and automatically set the processing information in the clothing manager.

Through one embodiment of the present invention, it is possible to provide a clothing manager and a control method thereof that can improve the user's satisfaction by informing the user that learning is being performed and evolving.

Through one embodiment of the present invention, in order for the home appliance to learn the correct, it is possible to provide a clothing manager and a control method capable of performing the user's approval or injection or forced learning by the user.

Through one embodiment of the present invention, the learning is performed only through the information obtained through a specific clothing manager used by the user, it is possible to provide a clothing manager and a control method thereof that can be customized. In addition, by minimizing the information required for learning to reduce the time required for learning, it is possible to provide a clothing manager and a control method thereof that can simplify the learning process.

According to one embodiment of the present invention, during the object processing or during the completion of the processing based on the set processing information, the clothing manager and the control that can be performed to change the lower information or sub information of the processing information can be performed It may provide a method.

According to one embodiment of the present invention, it is possible to provide a clothing manager and a control method thereof, which can automatically set processing information for an object that is affected but not general.

Through one embodiment of the present invention, through the learning through the image information formed by automatically photographing the object accommodated in the object receiving unit, it is possible to provide a clothing manager and a control method thereof that can set the processing information.

Through one embodiment of the present invention, it is possible to provide a clothing manager and a control method thereof, in which learning is automatically performed and the user can selectively use the learning result.

Through one embodiment of the present invention, it is possible to provide a clothing manager and a control method thereof that can provide a course setting specific to itself by learning a user-specific usage pattern.

According to one embodiment of the present invention, it is possible to provide a clothes manager and a control method thereof in which the user can intuitively and easily grasp whether the clothes manager performs learning and / or whether learning is possible.

1 is a perspective view showing an example of a clothing manager according to an embodiment of the present invention;
2 is a front view showing a user interface of the clothing manager according to an embodiment of the present invention;
3 is a control block diagram of a clothing manager according to an embodiment of the present invention;
4 is a flow chart showing a control method of the clothing manager according to an embodiment of the present invention;
5 is a flowchart illustrating a control method of a home appliance according to an embodiment of the present invention;
6 is a flowchart illustrating a control method of a home appliance according to an embodiment of the present invention;
7 is a flow chart illustrating the learning process shown in FIGS. 4 to 6;
8 shows an example of a database (DB) image;
9 shows an example of a query image;
10 shows an example of a support image;
11 illustrates a process of classifying and learning a query image.

Hereinafter, with reference to the accompanying drawings, it will be described in detail the artificial intelligent clothing manager and its control method according to an embodiment of the present invention.

Referring to FIG. 1, the clothing manager may include a cabinet 100 forming an exterior, an inner case 15 providing an accommodation space for accommodating clothes, and a door 300 that opens and closes the cabinet. The inner case 15 may be referred to as a clothing accommodating part provided in the cabinet to accommodate the clothes to be treated.

The clothing manager may include a mounting unit 400 or 500 provided at the door or the inner case to mount the garment, and a support unit 540 provided below the mounting unit to support the surface of the garment.

As a result, the clothes treating apparatus was capable of supplying hot air / cold air to a supply unit located under the cabinet, or pressing clothes to remove wrinkles at the same time while the clothes were placed in the receiving space inside the cabinet.

A machine room is provided below the inner case 15, and the machine room may be opened and closed by the machine room door 700. Various hardware may be provided in the machine room. Discharge port 230 for supplying steam or water (mist), hot air or cold air discharge port 240 and hot air or cold air suction port 250 may be provided. Therefore, a steam generator, a water tank, a water tank, a heater (heat pump) and a fan may be provided inside the machine room.

In addition, the front of the door 300 is provided with a control panel 100 for inputting the operation of the clothing manager, or to display the operating state of the clothing manager. That is, a user interface may be provided. The control panel 100 to the user interface may be provided to be distinguished from the door 300, or may be provided as part of the door.

The user may input or select object processing information through the user interface. The object processing information currently processed may be recognized through the user interface. Therefore, the user interface may be an input means for inputting information and an output means for outputting information.

As shown in Figure 1, the clothing manager according to an embodiment of the present invention may include a camera (45). The camera may be provided in the cabinet 10 or the door 300 to generate image information of an object (clothing) provided in the object accommodating part, that is, the clothes accommodating part.

For example, it may be provided to photograph the clothes hanging inside the inner case 15 to generate image information. The camera 45 may include an image sensor and a lighting device. The illumination device may comprise an LED and the image sensor may be an RGB camera.

The camera 45 may be provided on the rear surface of the door 300. Since the garment is hanging, it is desirable to create an image of the garment from top to bottom. Therefore, it is preferable that a camera is provided on the upper rear surface of the door 300.

The inside of the object receiving part is provided with an environment for object processing, which is different from the external environment. In particular, temperature and humidity differ. Since the temperature and humidity inside are intended, it is desirable that such temperature and humidity be maintained. Therefore, such object treatment is generally performed in the state in which the door 300 is closed.

To this end, a door sensor 50 for detecting a closed state of the door 300 may be provided, and the door sensor 50 may be provided in the door 1 or the cabinet 1 corresponding to the door. For example, when detecting that the door is closed through the door sensor 50, the object processing is performed. The door sensor 50 may be provided in the form of a switch.

In this embodiment as well, the door sensor 50 may be provided. However, the door sensor 50 in the present embodiment may be provided not only for detecting the closed state of the door but also in conjunction with the operation of the camera 45.

That is, the generation time of the image information through the camera 45 may be associated with the door closing detection time by the door sensor 50. For example, when the door sensor 50 detects a closed state of the door, the camera 45 may generate image information.

In order to operate the camera 45 and the door sensor 50, power must be applied to these components. Home appliances such as refrigerators are always supplied with power, but in general, the clothing manager may be referred to as home appliances that are not always powered.

Thus, sequential logic may be required between powering up, door sensor operation and camera operation. This logic will be described later.

Hereinafter, a user interface applicable to an embodiment of the present invention will be described in detail with reference to FIG. 2.

Figure 2 shows an example of the user interface of the clothing manager.

In the case of the clothing manager, the object processing information may include course information. Such course information may include a series of processes for treating clothes, for example, steam, refreshing, drying and sterilization, and these processes may be performed sequentially. That is, it means a predetermined algorithm so that this process is performed sequentially. Each course may have different control factors for the courses.

In the steam preparation process, air may be forcibly blown. And during the sterilization process, it is possible to remove the fine dust and odor inside the object receiving unit as well as the clothing at the same time.

In addition, due to the nature of the clothing manager, drying or sterilization can be performed in its own course. That is, it may be an original course in which the steam or refresh process is omitted.

Thus, a plurality of course information may be provided. The course information may be provided in plural according to the type of object or a special function. In addition, sub-course information may be provided in each course information. Therefore, the object processing information may include not only course information but also sub course information. Of course, the specific course information may not include corresponding sub course information.

The course information may be provided in plural, and each of the course information may include a plurality of sub course information. That is, it can be said that a plurality of sub courses are provided for each higher course information.

Due to the characteristics of the clothes manager, the course information may include a plurality of higher course information according to the object processing purpose and the clothes to be processed. 2 is an example provided with four higher course information.

A course selector 110 including first to fourth course selectors 120 to 123 is provided. The user selects a specific sub course through the course selection unit. The sub courses in the upper course may be provided exclusively to each other. That is, a sub course in a specific upper course and a sub course in another upper course may not be selected together, and only one of them may be selected.

The upper course information may be divided into letters or may be divided into areas. That is, the lower courses corresponding to each upper course may be gathered in a certain area to be distinguished from other upper courses and lower courses thereof.

The styling course may be selected through the first course selector 120. That is, one of three courses can be selected as an example of the styling course in detail. The styling course may be used to remove wrinkles, remove odors using steam. Therefore, it may include a steam preparation step, a refresh step of supplying steam to the clothes. Thereafter, a drying process may be performed.

Such a styling course may be suitable for general clothing. It may be suitable for southern or dress shirts.

The luxury clothing course may be selected through the second course selection unit 121. That is, one of three courses may be selected as an example of the high-end clothing course in detail. In addition, you can download and select new courses.

High-end clothing courses are typically made of heat-sensitive materials. Thus, temperature control is more important than styling courses.

The sanitary sterilization course may be selected through the third course selector 122. That is, one of four courses can be selected as an example of the sanitary sterilization course in detail. The sanitizing course is a more effective way to remove dust, viruses and odors from clothing.

The custom drying course may be selected through the fourth course selector 123. That is, it may be selected to dry wet clothing or the like. The customized drying course may include a course for indoor dehumidification because it removes moisture. Custom built courses allow you to choose one of three courses in detail.

The above-described styling course, high-grade clothing course and sanitary sterilization course can be said to be prepared for optimal treatment according to the type or material of clothing to be treated. The styling course may be general clothing, the luxury apparel course may be suit, wool, knit or functional clothing such as sportswear or mountaineering, and the sanitizing course may be optimal for bedding, towels and baby clothes.

Each time the specific course selection unit 120 to 123 is pressed, the lower course of the specific course may be sequentially selected. In this case, the corresponding sub course can be indicated by the LED lighting.

Therefore, these courses are very easy to identify the target clothing through the image. And, the number of garments processed at one time by these courses is small and relatively large. Therefore, identification of the target garment is very easy. In addition, the number and types of clothes that can be processed by these courses, that is, the user has a limited number. Thus, learning is very easy and can result in very accurate learning results. That is, the processing information can be set automatically by learning.

On the other hand, the above-described custom drying course may not be a frequently used course. In other words, the course is used only in special cases. In addition, the garment to be treated may not be specified. Therefore, the customized drying course can prevent the processing information from being automatically set by learning according to the clothing which is the object.

Of course, the camera can generate images of the clothes's shape, color, material, fluff and shape, as well as water droplets or wet patterns on the surface of the clothes. Therefore, processing information may be automatically set by learning in a course for performing a special function such as drying as well as the type of clothing.

Therefore, if necessary, any one of the course information of the plurality of course information of the clothing manager can be set automatically by learning, and any one course information can be set to exclude the automatic setting by learning.

In addition, if necessary, any one of the course information of the specific sub-course information can be set automatically by learning and one of the course information can be set to exclude the automatic setting by the learning.

For example, the indoor dehumidification course is a course that is not related to clothing processing. The course may be a course for dehumidification of the object receiving unit itself inside the cabinet, or a dehumidification of an indoor space where the cabinet is placed. In other words, the clothing manager may function as a separate dehumidifier. Therefore, it is preferable that learning is not performed when such a course is selected.

Nevertheless, in this case the camera can generate an image when the door is closed and the course is selected. Since the indoor dehumidification course is excluded from learning, there is no processing information corresponding to the acquired image. Therefore, in this case, it is preferable that the user is guided through the alarm so that the clothing can be taken out from the object receiving unit.

2 illustrates the display 130. The display unit 130 may be provided to display the remaining time, the current progress cycle, or the process. As an example of the display unit 130, steam preparation, refreshing, drying, sterilization, storage and reservation are shown.

When each of the upper course and the lower course is selected, the LED provided behind the back is turned on to display the selected information.

On the other hand, the control panel 100 may be provided with a power input unit 140 for applying and releasing the power of the clothing manager and the operation / pause selection unit 150 for executing or pausing the operation of the clothing manager. The operation / pause selection unit may be referred to as a start input unit for convenience.

Accordingly, the user inputs object processing information through the course selection unit 110, and object processing is performed according to the input process information. This series of processes can be referred to as manual setting mode.

An example of the manual setting mode is as follows.

The user opens the door 300 and closes the door 300 after inserting the object. After power is applied through the power input unit 140, the intensive course may be selected in the styling course 120 through the course selector 110.

When the input of the object processing information is finished, the user inputs the start input unit 150, and the clothing manager automatically processes the object based on the input processing information and then ends. In the intensive course, refreshing and drying by strong steam (maximum temperature of steam, amount of steam, etc.) may be sequentially performed.

In this embodiment, it is possible to provide a clothes manager capable of providing an automatic setting mode as well as the manual setting mode described above. That is, each time the user wants to process the object, it is possible to provide a clothing manager capable of automatically setting the object processing information without inputting the object processing information.

In particular, the present embodiment can provide a clothing manager that evolves while performing learning. In addition, the user may be able to recognize whether the learning is performed or evolved to provide a clothing manager that may increase user satisfaction.

In this embodiment, the user may learn through the image information about the object put into the object receiving unit to set the object processing information. That is, even if the user does not manually input the processing information, it is possible to provide a clothing manager that can set the processing information by reflecting the learning result.

While the user uses the clothing manager through the manual setting mode, the clothing manager can continuously perform the learning. That is, the learning process may be performed through image information obtained through the camera 45 and processing information obtained through the user interface. Details of the learning process will be described later.

The course set by reflecting the result of the learning process may be called a learning course. The mode in which the processing information is set using the learning course may be referred to as a learning setting mode. Unlike the manual setting mode described above, the learning setting mode may mean automatically setting the processing information even if the user does not manually input the processing information.

For example, when the user selects a learning course through the advanced clothing selection unit 121, which is, for example, the second course selection unit, the learning setting mode may be used as a default.

An example of the learning setting mode is as follows.

The user opens the door 300 and closes the door 300 after inserting the object. After applying power through the power input unit 140, a learning course may be input. When the door 300 is closed and the power is applied, the camera 45 automatically generates image information on the currently input object.

When the learning course is input, the current processing information is set through the current learning process result and the currently acquired image information. That is, the processing information can be set without the user inputting the processing information.

At this time, it is preferable that the user recognizes that the processing information to be set is processing information reflecting learning. That is, it is preferable to allow the user to recognize that the processing information corresponding to the current image is set based on the result of the learning based on the image information.

To this end, it is preferable to indicate that the user recognizes the process of outputting the learning result for about 1 second to 2 seconds. For example, the display through the display 130, the plurality of LEDs are variably lit, only the LED corresponding to the set process information may be lit. In addition, voice through the speaker may be guided.

The user may approve the set processing information through the start input unit 150. You may also approve it via voice input through a microphone.

When the approval step is completed, the processing of the object may be performed based on the set processing information.

On the other hand, the user can enter new processing information without approval in the approval step. In this case, forced learning may be performed through the currently acquired image information and newly input processing information. That is, injection learning or forced learning by a user may be performed. The results of such injection learning or forced learning may be prioritized over the learning results of other processes. That is, the learning result through the forced learning may be given priority over the learning result through the manual setting mode. By reflecting the priority of the learning result in the clothing manager, the user can recognize that the clothing manager evolves by learning.

If the learning setting mode is performed after the learning course is input immediately before, the learning course may be selected by default. That is, the learning course selection may be maintained continuously unless the user inputs the learning course again. Even if the power is turned off after the object processing using the learning course is finished, the learning course may be selected by default when the power is subsequently applied.

Therefore, the learning course may be provided in at least one of the first course selector and the third course selector. In addition, it may be provided as a separate selection unit encompassing the course selection unit together. When provided as a separate course selection unit, when a user selects a separate course selection unit, specific courses in the first to third course selection units may be automatically set according to the corresponding clothing.

The fourth course selector may be a course selector for manual selection only. Therefore, the fourth course selection unit may be regarded as having nothing to do with the learning course and with regard to learning and image generation. Therefore, it is preferable that image generation is not performed when a course is selected through the fourth course selection unit. That is, the premise itself for learning may not be performed.

On the other hand, in the initial case of not using a lot of clothing manager, the number or type of learning results does not vary. Therefore, there may not be a learning result corresponding to the currently acquired image information.

Therefore, the process of determining the presence or absence of the learning result in the learning setting mode may be performed.

If there is a learning result, corresponding processing information may be output and set. On the other hand, if there is no learning result, the user can be notified. If the user washes new clothing that has not been washed before, there will be no corresponding learning result. Therefore, in this case, the user may recognize that the clothing manager wants to receive new information for learning. Therefore, the user can also recognize that the clothing manager is evolving.

If there is no learning result, the user may be informed of this, and the user may be inputted with processing information or suggested similar processing information. That is, even if there is no processing information corresponding to the currently acquired image information, the most similar processing information may be proposed. In this case, the approval step may be performed as described above, and the forced learning step may also be performed.

The learning course may be provided separately from the course input unit 110. Alternatively, the learning course may be provided as a part of the course input unit 110. Even in the latter case, it is as described above to select and reflect the learning course.

The reason why the selection of the learning course is provided in any case may be that the user selects and uses the manual setting mode and the learning setting mode. If the user avoids the use of the manual setting mode from the beginning of the clothes manager and prefers the automatic setting mode, this learning course selection part may be omitted.

That is, when a sufficient amount of learning results is provided or there are learning results corresponding to the currently acquired image, the learning setting mode may be performed. On the contrary, if a sufficient amount of learning results is not provided or there is no learning result corresponding to the currently acquired image, the aforementioned forced learning may be performed.

In the case of forced learning, the user must enter the processing information manually. However, in this case, the user can recognize through the user interface that the clothing manager is trying to learn and evolve to perform the learning setting mode. Therefore, in addition to simply processing the object through the clothing manager, the user himself can obtain a play that can contribute to and witness the evolution of his clothing manager. This amusement can be further enhanced through voice. Therefore, it would be desirable for the user interface to include a microphone and / or a speaker.

Hereinafter, with reference to Figure 3 will be described in detail the control configuration of the clothing manager according to an embodiment of the present invention.

The clothing manager may include a main controller or main processor 160 that controls a series of processes of object processing. The main controller 160 controls the driving of the hardware 310 to perform the set processing information. The hardware 310 may be provided in various ways for each clothing manager. Steam generator 70 may be included. And it may include a water level sensor 12, the heater 50 for sensing the water level in the steam generator or the water level of the bucket for supplying water to the steam generator or the water level of the drain box. The heater 50 may be replaced with a heat pump 86.

If there is no drain box, it may include a separate drain pump 24. A separate heater 50 or fan 60 for drying may also be included in the hardware.

A learning controller or learning processor 166 may be provided to perform learning and output a learning result. The learning processor 166 may be provided separately from the main processor or embedded in the main processor. In the learning processor 166, a learning algorithm or learning logic, which will be described later, may be programmed.

Image information generated through the camera 45 and processing information input through the user interface 100 may be transmitted to the main controller 160. Image information and processing information transferred to the main controller 160 may be transferred to the learning controller 166. Of course, at least one of the image information and the processing information may be directly transmitted to the learning controller 166.

The learning process may be performed by the learning processor 166, and may use image information as an input factor and process information as output information.

On the other hand, in recent years, a lot of smart home appliances that communicate with the server has been provided. That is, the home appliance is provided with a communication module (not shown) to communicate with the server. Therefore, the learning processor 166 may be omitted from the clothes manager, and the server learning controller or the processor 210 may be provided in the server controller 200.

That is, the clothing manager delivers the input factor of the learning process to the server, and the server may perform the learning to deliver the learning result to the clothing manager. In this case, since the clothing manager does not require a separate learning processor, it is possible to lower the product cost.

On the other hand, if the user wants his own clothing manager or a clothing manager specialized for himself, it may be desirable to have a separate learning processor 166 in the clothing manager. That is, a specialized and personalized clothing manager of the user different from other users may be provided.

The clothing manager includes a user interface 100. Input and output of processing information may be performed through the user interface 100. Specific configurations of the user interface have been described with reference to FIG. 2.

Various input units or selectors 140, 150, and 110 in the user interface 100 may be provided so that a user may physically select or input the same. It may be provided in any form of a button or a touch panel that is input through physical contact or pressing. The input or selection unit may be provided through a touch menu in the touch display.

However, in the case of the power input unit, it may be preferable that the power input unit is provided in the form of a physical button separately from the other input units due to a decrease in user experience or standby power. In other words, the power input unit may be provided in the form of a power applying switch. It may be preferable that the start input unit 150 facing the power input unit 140 is also provided in the form of a physical button.

Hereinafter, a control method according to an embodiment of the present invention will be described in detail with reference to FIG. 4.

When power is supplied to the clothes manager (S10) and door closing is detected (S20), step S30 of generating and obtaining image information through a camera may be performed.

The image information generated by the camera should be image information on the current object input for object processing.

The user may apply power first and then input the object. The door is closed when the object is added. On the contrary, the user may apply power in a state where the object has been added and the door is closed.

In any case, it is preferable that power is applied (S10), door closing detection (S20) and image information acquisition (S30) sequentially. If a subsequent step is not performed within a preset time after the image information acquisition step S30, the power may be automatically turned off to terminate the step S80. At this time, the acquired image information will not be used for learning.

When the image information is generated or acquired (S40), the process of obtaining the process information set by the user or setting process information by learning (S40) may be performed.

In the case of the manual setting mode described above, in the step S40, processing information manually input by the user is acquired by the main controller. This processing information is used in the learning S50 together with the currently obtained image information. However, as long as the manual setting mode is maintained, the learning may be performed but the learning result may not be reflected.

When the user approves the processing information set manually by the voice or the start input unit, the main controller 160 performs object processing based on the processing information. When the object processing is completed, the main controller may perform power off to end the object processing (S80).

During the object processing, learning (S50) may be performed in parallel with the object processing.

Meanwhile, as described above, in the case of the clothing manager, the object processing information may include not only upper course information but also lower course information thereof.

A coat or a suit with a relatively large volume can be classified into the same kind of clothing, so that the treatment can be performed through a suit / coat course.

Through learning, various image information about a coat or a suit is input and a coat / suit course can be set automatically. If a course specialized in fur is not provided, various image information about the fur garment may be input through forced learning by a download course, and the download course may be automatically set.

On the other hand, a course such as fur may be performed as a coat / suit course. In this case, although not shown in Figure 2, a selection unit for selecting the lower information on the hot air temperature, steam or not, wind strength, shaking strength, etc. may be further provided. In other words, learning may be performed by the user selecting sub-optimal information optimized for fur along with coat / dress course selection.

Through learning, thermal damage may occur when the treatment for fur garments is performed as a futon course. This can be seen by changing the temperature around the garment. Therefore, in this case, it can be said that correction of the processing information set by learning is necessary. Therefore, it is preferable that step S70 is performed to determine whether correction of processing information by learning is necessary during object processing or before completion of object processing.

Such correction may be performed to correct and change course information itself, which is higher information, or to correct and change various lower course information or lower option information.

When the correction is necessary, such correction information (for example, a predetermined low hot air temperature among the process information set by the current learning and no steam use) may be reflected in the learning process S50.

Whether the correction and whether the correction is reflected may be notified to the user through the user interface during object processing or before termination.

Therefore, the end of the object processing (S80) is preferably performed only after the object processing process, the learning process, and the user notification process are all completed.

In the learning setting mode described above, in step S40, processing information is automatically set based on the learning result and the currently acquired image information.

Previous learning result If there is a learning result that matches the currently acquired image information, the corresponding processing information is outputted, and the outputted processing information may be automatically set.

It is desirable to allow the user to recognize that such setting of processing information is based on learning. The user interface can be used to make the user aware. The user can intuitively recognize that only the LED display set after the variable lighting of the LED displays is turned on, or the processing information is set by learning through the display through the display or the audio output through the speaker.

According to the set processing information, the object processing (S60), whether to correct or not (S70), the learning step (S50), etc. may be performed in the same manner.

As described above, in the step S40, the manual setting mode and the learning setting mode may be performed differently according to the user's selection or differently depending on the number of learning results.

While the manual setting mode is performed, learning may be continued but the learning result may not be reflected. Thereafter, when the user selects the learning setting mode, the learning setting mode may be used to reflect the learning result.

When the learning setting mode is performed by default, the learning is performed by the forced learning in the early stage of the clothing manager in which the learning result is not sufficient, and the processing information may be automatically set when the learning result is sufficient. When image information about each of the provided processing information is obtained, it is possible to perform a true meaning learning through image expansion. This will be described later.

Of course, in the learning setting mode, the user approval process / step, the forced learning, or the injection learning step may be similarly performed.

The learning process in this embodiment may be a process of expanding the number of combinations of image information distinguished from each other and processing information output in correspondence with each image information.

In the garment manager, the type and number of garments that a particular user can process through the garment manager are varied at a certain level. Even if new clothes are purchased, the clothes can be discarded, so the limits of types and number of clothes are predictable. In particular, unlike the washing machine, since the clothes washing is not desirable to manage clothes through the clothes manager, the type and number will be relatively small.

Therefore, the number of image information predictable in the clothing manager of a specific user and the number of combinations of processing information corresponding thereto may be gradually expanded to a predictable level. In other words, it will be able to endure the processor of the clothing manager itself without any performance expansion.

On the contrary, in the case where the image information of all the garments that can be processed and the combinations of the processing information thereof are databased in advance, there is little room for expansion and evolution by learning. In addition, it is very difficult to make a database, and if such a database is embedded in the clothing manager itself, it may cause an overload of its own processor. In addition, even when a database is embedded in the server, excessive time may be required for information transmission.

Therefore, without an initial database, it would be desirable to allow a database that is specific to a particular user to be extended by learning. In addition, while the learning itself is performed in the clothing manager or server, it may be desirable to manage the database for the learning results in the clothing manager itself.

Hereinafter, with reference to FIG. 5, a control method according to an embodiment of the present invention will be described in detail. In this embodiment, the learning setting mode and the manual setting mode can be clearly distinguished.

First, the step of applying power (S110), door closing detection (S120) and obtaining image information (S130) may be the same as the embodiment shown in FIG.

After the image information acquisition step is performed, it is determined whether to proceed with the learning setting mode or the manual setting mode (S140). Of course, the door closing detection and / or image information acquisition step may be performed after the determination step S140 is performed.

When the user selects the learning setting mode through a learning course input or the like, the learning setting mode may be performed. Learning course input may be variously performed through a user interface. In addition, the learning course may be set as a default. In this case, the learning setting mode may also proceed. The learning setting mode may be set as a default when the immediately used form of the clothing manager is the learning setting mode. If the learning setting mode is not selected or not set as the default, the manual setting mode may be performed.

When the learning setting mode is performed, step S143 of determining whether a learning result is present may be performed. That is, it may be determined whether object processing information corresponding to the currently obtained image information may be output. If there is a learning result, object processing information corresponding to the currently acquired image information is automatically set (S144). Object processing (S160) is performed based on the set object processing information.

On the other hand, the user approves the set object processing information (S145) may be performed. The user can grasp the object processing information set through the user interface, and the user can approve it. The user's approval may be performed through the user interface, such as selecting a start input, approving by voice, or not presenting a separate input or response for a predetermined time.

Of course, the user can disapprove the set object processing information. Similarly, disapproval by voice or by inputting new object processing information (S146) can be disapproved. When disapproval of the set object processing information and a user inputs new object processing information, learning may be performed (S147). Learning in this case can be called forced learning.

That is, it may be said that new learning is performed while ignoring the object processing information set according to the currently acquired image information and the previously learned result. In the forced learning, the object processing information input by the user may be said to take precedence over the object processing information input by the user in general learning.

Thereafter, the object processing is performed based on the object processing information input by the user (S160).

If there is no learning result, notification S148 may be performed. The voice or display may indicate that there is no learning result, and the user may be requested to input object processing information.

When the user inputs object processing information (S149), learning (S150) may be performed. Learning in this case can be called forced learning.

Thereafter, the object processing is performed based on the object processing information input by the user (S160).

In the manual setting mode in the determination step (S140), the user inputs object processing information (S141). At this time, learning is performed through the currently acquired image information and the object processing information input by the user (S142). Learning here is also general learning, not forced learning. Thereafter, the object processing is performed based on the object processing information input by the user.

FIG. 6 is a modified embodiment of the embodiment shown in FIG. 5.

When the user selects the learning setting mode, it may mean that the user does not manually input the object processing information. That is, it may be an expression of the intention that the user does not have to trouble selecting specific object processing information among the object processing information having a plurality or various combinations.

Therefore, in this embodiment, unlike in FIG. 5, if there is no learning result, step S152 of automatically recommending object processing information may be performed. This may be performed in various forms such as recommending object processing information, which is an output most similar to current image information. Of course, step S151 for notifying the recommendation or not indicating that there is no output result may be performed.

The user may approve or disapprove the recommended object processing information. Approval and disapproval procedures or methods may be the same as described above.

If the user approves, similarly, learning S154 may be performed to perform object processing S160 based on the approved object processing information.

If the user does not approve, the user may manually input new object processing information (S155) and learn (S156). Similarly, the object processing (S160) may be performed based on the object processing information input by the user.

Hereinafter, the learning that can be applied to an embodiment of the present invention will be described in detail with reference to FIG. 7.

As described above, the learning in the embodiments may be a process of expanding the number of combinations by matching the acquired image information and the acquired processing information.

As learning is performed using image information, a known learning algorithm or logic may be applied to the present embodiment.

Learning in this embodiment may be referred to as a process of performing learning by performing image expansion or data augmentation step S51 instead of forced learning. Data expansion may be performed through rotation, blurring, and brightness of the obtained images.

After data expansion, a feature vector may be extracted (S52) using a known convolutional neural network (CNN).

After the feature vector is extracted, a known one shot learning / fue shot learing learning is performed through the learning processor embedded in the clothes manager or the learning processor of the server (S53). The learning result obtained through learning is output (S54). That is, the corresponding processing information corresponding to the learning result is output and set.

Of course, as described above, it is also possible to use the output processing information as it is, not to reflect it, or to change the learned result through forced learning.

Meanwhile, in the present exemplary embodiment, a step (S55) of determining whether to correct is performed between the learning performed by one-shot running / fueshot running (S53) and the learning result output (S54) may be further performed. As object processing is performed, there is a possibility that a correction of processing information is required. Since the learning performance S53 and the learning result output S54 may be performed while the object processing is performed or before the end of the object processing, the step S55 of determining whether the correction is performed may be smoothly performed.

When the correction is not necessary, the learned result may be output as it is, and when the correction is necessary, the learned result may be output by reflecting the correction information.

Hereinafter, the learning process through the object image will be described in detail with reference to FIGS. 8 to 11. FIG. 8 shows an example of images of a database DB, FIG. 9 shows an example of a query image and FIG. 10 an example of a support image. Such an image can be generated with an RGB sensor, an RGB camera or a gray camera. 8 to 10 illustrate converting a color image into a black and white image for convenience. Each image is an example of a southern shirt, knit, dress pants and fur clothing.

As shown, it can be seen that the query image and the support image are part of the database image.

In the database image including the acquired image and the currently acquired image, the query image and the support image are randomly classified by a predetermined number.

The query image is an image to classify. Therefore, it may be referred to as an image that is currently acquired or before being expanded. In other words, it is an image before learning, and thus may not be an image having a corresponding output value. The support image may be referred to as a part necessary for obtaining information for clustering feature domains obtained from the same encoder.

Various classification techniques can be used to obtain feature information from the query image. In this embodiment, the CNN technique may be used as an example of the extension technique.

As shown in FIG. 11, a feature vector is generated through an encoder that performs an extension technique in a query image, and the feature vector is converted into an embedding space, that is, a distance domain. . In this way, the features of the same processing information will be close together in distance values, and the heterogeneous ones will be separated from each other.

The average of feature vectors from the support image is obtained. The degree according to the distance between the average of the support feature vector and the query feature vector is converted into a stochastic value as in the known formula. The known formula may be referred to as an extended formula known in one shot learing or few shot learning.

Therefore, the query image can be extended. That is, the image may be matched with the previously learned image through the query image, and the processing information may be output through the query image.

According to the above-described embodiment, the clothing manager may be provided so that the user can recognize the evolution by learning and improve the user satisfaction. In particular, the processing information can be automatically set, so that the user can save the trouble of inputting the processing information by himself and can be provided with a very convenient clothing manager.

10: cabinet 300: door
45: camera 50: door sensor
100: control panel (user interface)
110: course selection unit 130: display unit 140: power input unit 150: start input unit 160: main control unit 170: microphone 180: speaker

Claims (20)

cabinet;
A clothes accommodating part provided in the cabinet to accommodate clothes to be treated;
A door provided in the cabinet to open and close the clothing accommodating part;
A camera provided in the cabinet or door to generate image information of the garment mounted inside the garment receiving portion;
A user interface provided on a front surface of the door, the user interface being provided to receive processing information about the clothing processing from a user; And
Based on the result of the learning process by matching the image information acquired through the camera and the processing information acquired through the user interface, the image information currently acquired through the camera is converted into data augmentation and a feature vector ( feature vector) includes a main processor that performs extraction and outputs and sets corresponding current processing information, and controls to process an object based on the current processing information that is set,
The clothing receiving portion accommodates the clothing in a predetermined position,
The main processor is configured to check not only the type of the garment but also the wet state of the garment surface through the image information currently acquired through the camera, and considering the kind of the garment and the wet state of the garment surface. A clothing manager for setting processing information. The method of claim 1,
The user interface,
Clothing management apparatus comprising at least one of a speaker for outputting a voice, a microphone for receiving a voice, a display and a button or a touch panel for receiving a physical contact or pressure by the user. The method of claim 1,
A door sensor for detecting whether the door is closed,
The camera may be configured to generate the image information after the power of the clothing manager is applied and the door sensor is detected by the door sensor. The method of claim 1,
The learning process is characterized in that the clothing manager is performed using the image information as an input factor and the processing information as an output factor. The method of claim 1,
The learning process may be performed through a learning processor embedded in the clothes manager or a learning processor of a server communicating with the clothes manager. The method of claim 1,
The learning process includes determining whether there is a result of the learning process corresponding to the currently acquired image information. The method of claim 6,
If there is a result of the corresponding learning process in the step of determining, the clothing manager, characterized in that the corresponding processing information is output and set. The method of claim 7, wherein
After setting the processing information, the clothing manager characterized in that the approval step of performing the user's approval of the current processing information set through the user interface is performed. The method of claim 8,
If there is no user's approval in the approval step, the garment manager characterized in that the learning step of matching the processing information allowed by the user to the currently acquired image information is performed. The method according to any one of claims 6 to 9,
If there is no result of the corresponding learning process in the determining step, the garment manager characterized in that a forced learning step of matching the processing information allowed by the user with respect to the currently acquired image information is performed. The method of claim 1,
Through the learning process, the clothes manager characterized in that the number of combinations for the image information to be distinguished from each other and the processing information output corresponding to each image information is expanded. The method of claim 1,
The user interface, the garment manager, characterized in that it comprises a start input unit for accepting the user to start processing the object through the set current processing information. The method of claim 1,
The processing information is
Clothing manager characterized in that it comprises a plurality of course information provided to perform in the clothing manager. The method of claim 13,
The plurality of course information is provided with a plurality of course items, each of the course item is a clothing manager, characterized in that a plurality of sub-courses are provided. The method of claim 14,
Any one of the plurality of course items includes a learning course for automatically setting the current processing information based on a result of the learning process,
Clothing management device, characterized in that any one of the plurality of course items are provided so that the user is set only manually. The method of claim 13,
The user interface includes a course selector configured to allow a user to select any one of the plurality of course information.
Any one of the plurality of course information is a clothing manager characterized in that the learning process to automatically set the current processing information based on the results of the learning process. The method of claim 16,
If the learning course is selected at the time of processing the clothing through the clothing manager through the course selection unit, the clothing manager, characterized in that currently selected as the default setting value. The method of claim 16,
When a manual setting course other than the learning course is selected from among the plurality of course information, the garment manager characterized in that a learning process through the selected manual setting course is performed on the currently acquired image information. The method of claim 13,
And a current process information is automatically set when there is a result of a learning process for at least one or more course information among the plurality of course information. The method of claim 13,
And if there is a result of a learning process for each of the plurality of course information, the current processing information is automatically set.

Images