KR20210089880A - Laundry treating apparatus - Google Patents

Abstract

The present disclosure provides a laundry treatment apparatus. The laundry treatment apparatus of the present invention senses user image information, inputs the user image information into an artificial intelligence model, obtains laundry information corresponding to a user and user behavior information, and performs an operation corresponding to the laundry information. The laundry treatment apparatus of the present invention comprises: a washing unit; an output unit; a camera; and a processor.

Description

Laundry handling equipment {LAUNDRY TREATING APPARATUS}

The present disclosure discloses a laundry treatment apparatus that identifies a user by using user image information, recognizes the user's behavior information, and provides a washing operation suitable for the user behavior information. In addition, the present disclosure discloses a personalized laundry treatment apparatus by accumulating laundry operation usage information of individual users.

In general, a laundry treatment apparatus includes a washing machine, a dryer, and a washing machine capable of both a drying function and a washing function, and the washing machine may include any device capable of performing washing regardless of an operation method.

A washing machine is a device that performs washing, rinsing, and dehydration processes to remove contaminants such as clothes and bedding through water, detergent, and mechanical operations.

A dryer is a general term for a device that dries by applying wind to a dried object.

Although there is a washing operation that is repeatedly used, users have the inconvenience of having to manually set the washing operation every time they use the laundry treatment apparatus. With the advent of personalized washing machines, there is a problem in that, even when frequently used washing operations are recommended, when several people use one laundry processing apparatus, the individual washing operation settings cannot be reflected and frequently used washing settings are recommended.

In general, a laundry treatment apparatus includes a washing machine, a dryer, and a washing machine capable of both a drying function and a washing function, and the washing machine may include any device capable of performing washing regardless of an operation method.

A washing machine is a device that performs washing, rinsing, and dehydration processes to remove contaminants such as clothes and bedding through water, detergent, and mechanical operations.

A dryer is a general term for a device that dries by applying wind to a dried object.

Although there is a washing operation that is repeatedly used, users have the inconvenience of having to manually set the washing operation every time they use the laundry treatment apparatus. With the advent of personalized washing machines, there is a problem in that, even when frequently used washing operations are recommended, when several people use one laundry processing apparatus, the individual washing operation settings cannot be reflected and frequently used washing settings are recommended.

An object of the present disclosure is to provide a laundry treatment apparatus capable of automatically performing a laundry operation by identifying a user who wants to use the laundry treatment apparatus and behavioral information of the user using an artificial intelligence model.

In addition, an object of the present disclosure is to provide a laundry operation customized for each individual by using a past user, behavior information of the past user, and laundry information as learning data.

The present disclosure includes a washing unit that performs a washing operation, an output unit displaying information corresponding to the washing operation, a camera sensing user image information, and inputting user image information into an artificial intelligence model to obtain washing information, Disclosed is a processor that performs an operation corresponding to laundry information.

In addition, the artificial intelligence model of the present disclosure may include a machine learning algorithm learned using a usage history including past users and behavioral information of the user and laundry information corresponding to the user and behavioral information of the user.

The present disclosure may provide a user's personalized laundry operation by using an artificial intelligence model to obtain laundry information corresponding to the user and behavior information who want to use the laundry processing apparatus.

In addition, the present disclosure provides a user's intention to do laundry by providing an operation of the laundry treatment apparatus suitable for a user who has a history of using the laundry treatment apparatus by learning an artificial intelligence model using the past usage history. It is possible to provide a washing operation suitable for

1 shows an artificial intelligence device 100 according to an embodiment of the present disclosure.
2 shows an artificial intelligence server 200 according to an embodiment of the present disclosure.
3 shows an artificial intelligence system 1 according to an embodiment of the present disclosure.
4 shows an AI device according to an embodiment of the present disclosure.
5 shows a washing unit of the artificial intelligence device of the present disclosure.
6 shows a flow diagram of the present disclosure;
7 shows a scenario of the present disclosure.
8 shows an artificial intelligence model of the present disclosure.
9 shows a flowchart of the present disclosure.
10 shows a flowchart of the present disclosure.
11 shows a scenario of the present disclosure.
12 shows a flowchart of the present disclosure.
13 shows a flowchart of the present disclosure.
14 shows a scenario of the present disclosure.

Hereinafter, the detail of this invention is demonstrated.

The embodiments described below are only examples of the present invention, and the present invention may be modified in various forms. Accordingly, the specific features and functions disclosed below do not limit the scope of the claims.

Hereinafter, the embodiments disclosed in the present specification will be described in detail with reference to the accompanying drawings, but the same or similar components are assigned the same reference numerals regardless of reference numerals, and overlapping descriptions thereof will be omitted. The suffixes "module" and "part" for the components used in the following description are given or mixed in consideration of only the ease of writing the specification, and do not have a meaning or role distinct from each other by themselves. In addition, in describing the embodiments disclosed in the present specification, if it is determined that detailed descriptions of related known technologies may obscure the gist of the embodiments disclosed in the present specification, the detailed description thereof will be omitted. In addition, the accompanying drawings are only for easy understanding of the embodiments disclosed in the present specification, and the technical spirit disclosed in the present specification is not limited by the accompanying drawings, and all changes included in the spirit and scope of the present disclosure , should be understood to include equivalents or substitutes.

Terms including an ordinal number, such as first, second, etc., may be used to describe various elements, but the elements are not limited by the terms. The above terms are used only for the purpose of distinguishing one component from another.

When a component is referred to as being “connected” or “connected” to another component, it is understood that the other component may be directly connected or connected to the other component, but other components may exist in between. it should be On the other hand, when it is mentioned that a certain element is "directly connected" or "directly connected" to another element, it should be understood that the other element does not exist in the middle.

<Artificial Intelligence (AI)>

Artificial intelligence refers to a field that studies artificial intelligence or methodologies that can make it, and machine learning refers to a field that defines various problems dealt with in the field of artificial intelligence and studies methodologies to solve them. do. Machine learning is also defined as an algorithm that improves the performance of a certain task through constant experience.

An artificial neural network (ANN) is a model used in machine learning, and may refer to an overall model having problem-solving ability, which is composed of artificial neurons (nodes) that form a network by combining synapses. An artificial neural network may be defined by a connection pattern between neurons of different layers, a learning process that updates model parameters, and an activation function that generates an output value.

The artificial neural network may include an input layer, an output layer, and optionally one or more hidden layers. Each layer includes one or more neurons, and the artificial neural network may include neurons and synapses connecting neurons. In the artificial neural network, each neuron may output a function value of an activation function for input signals, weights, and biases input through synapses.

Model parameters refer to parameters determined through learning, and include the weight of synaptic connections and the bias of neurons. In addition, the hyperparameter refers to a parameter that must be set before learning in a machine learning algorithm, and includes a learning rate, the number of iterations, a mini-batch size, an initialization function, and the like.

The purpose of learning the artificial neural network can be seen as determining the model parameters that minimize the loss function. The loss function may be used as an index for determining optimal model parameters in the learning process of the artificial neural network.

Machine learning can be classified into supervised learning, unsupervised learning, and reinforcement learning according to a learning method.

Supervised learning refers to a method of training an artificial neural network in a state where a label for training data is given, and the label is the correct answer (or result value) that the artificial neural network should infer when the training data is input to the artificial neural network. can mean Unsupervised learning may refer to a method of training an artificial neural network in a state where no labels are given for training data. Reinforcement learning can refer to a learning method in which an agent defined in an environment learns to select an action or sequence of actions that maximizes the cumulative reward in each state.

Among artificial neural networks, machine learning implemented as a deep neural network (DNN) including a plurality of hidden layers is also called deep learning (deep learning), and deep learning is a part of machine learning. Hereinafter, machine learning is used in a sense including deep learning.

<Robot>

A robot can mean a machine that automatically handles or operates a task given by its own capabilities. In particular, a robot having a function of recognizing an environment and performing an operation by self-judgment may be referred to as an intelligent robot.

Robots can be classified into industrial, medical, home, military, etc. depending on the purpose or field of use.

The robot may be provided with a driving unit including an actuator or a motor to perform various physical operations such as moving the robot joints. In addition, the movable robot includes a wheel, a brake, a propeller, and the like in the driving unit, and can travel on the ground or fly in the air through the driving unit.

<Self-Driving>

Autonomous driving refers to a technology that drives by itself, and an autonomous driving vehicle refers to a vehicle that runs without a user's manipulation or with a minimal user's manipulation.

For example, autonomous driving includes technology for maintaining a driving lane, technology for automatically adjusting speed such as adaptive cruise control, technology for automatically driving along a predetermined route, technology for automatically setting a route when a destination is set, etc. All of these can be included.

The vehicle includes a vehicle having only an internal combustion engine, a hybrid vehicle having both an internal combustion engine and an electric motor, and an electric vehicle having only an electric motor, and may include not only automobiles, but also trains, motorcycles, and the like.

In this case, the autonomous vehicle may be viewed as a robot having an autonomous driving function.

<Extended Reality (XR)>

The extended reality is a generic term for virtual reality (VR), augmented reality (AR), and mixed reality (MR). VR technology provides only CG images of objects or backgrounds in the real world, AR technology provides virtual CG images on top of images of real objects, and MR technology is a computer that mixes and combines virtual objects in the real world. graphic technology.

MR technology is similar to AR technology in that it shows both real and virtual objects. However, there is a difference in that in AR technology, a virtual object is used in a form that complements a real object, whereas in MR technology, a virtual object and a real object are used with equal characteristics.

XR technology can be applied to HMD (Head-Mount Display), HUD (Head-Up Display), mobile phone, tablet PC, laptop, desktop, TV, digital signage, etc. can be called

1 shows an artificial intelligence device 100 according to an embodiment of the present disclosure.

AI device 100 is TV, projector, mobile phone, smartphone, desktop computer, notebook computer, digital broadcasting terminal, PDA (personal digital assistants), PMP (portable multimedia player), navigation, tablet PC, wearable device, set-top box (STB) ), a DMB receiver, a radio, a washing machine, a refrigerator, a desktop computer, a digital signage, a robot, a vehicle, etc., may be implemented as a fixed device or a movable device.

Referring to FIG. 1 , the terminal 100 includes a communication unit 110 , an input unit 120 , a learning processor 130 , a sensing unit 140 , an output unit 150 , a memory 170 and a processor 180 , and the like. may include

The communication unit 110 may transmit/receive data to and from external devices such as other AI devices 100a to 100e or the AI server 200 using wired/wireless communication technology. For example, the communication unit 110 may transmit/receive sensor information, a user input, a learning model, a control signal, and the like with external devices.

At this time, the communication technology used by the communication unit 110 includes GSM (Global System for Mobile communication), CDMA (Code Division Multi Access), LTE (Long Term Evolution), 5G, WLAN (Wireless LAN), Wi-Fi (Wireless-Fidelity) ), Bluetooth??, RFID (Radio Frequency Identification), Infrared Data Association (IrDA), ZigBee, NFC (Near Field Communication), and the like.

The input unit 120 may acquire various types of data.

In this case, the input unit 120 may include a camera for inputting an image signal, a microphone for receiving an audio signal, a user input unit for receiving information from a user, and the like. Here, the camera or microphone may be treated as a sensor, and a signal obtained from the camera or microphone may be referred to as sensing data or sensor information.

The input unit 120 may acquire training data for model training and input data to be used when acquiring an output using the training model. The input unit 120 may acquire raw input data, and in this case, the processor 180 or the learning processor 130 may extract an input feature as a preprocessing for the input data.

The learning processor 130 may train a model composed of an artificial neural network by using the training data. Here, the learned artificial neural network may be referred to as a learning model. The learning model may be used to infer a result value with respect to new input data other than the training data, and the inferred value may be used as a basis for a decision to perform a certain operation.

In this case, the learning processor 130 may perform AI processing together with the learning processor 240 of the AI server 200 .

In this case, the learning processor 130 may include a memory integrated or implemented in the AI device 100 . Alternatively, the learning processor 130 may be implemented using the memory 170 , an external memory directly coupled to the AI device 100 , or a memory maintained in an external device.

The sensing unit 140 may acquire at least one of internal information of the AI device 100 , information on the surrounding environment of the AI device 100 , and user information by using various sensors.

At this time, sensors included in the sensing unit 140 include a proximity sensor, an illuminance sensor, an acceleration sensor, a magnetic sensor, a gyro sensor, an inertial sensor, an RGB sensor, an IR sensor, a fingerprint recognition sensor, an ultrasonic sensor, an optical sensor, a microphone, and a lidar. , radar, etc.

The output unit 150 may generate an output related to visual, auditory or tactile sense.

In this case, the output unit 150 may include a display unit that outputs visual information, a speaker that outputs auditory information, and a haptic module that outputs tactile information.

The memory 170 may store data supporting various functions of the AI device 100 . For example, the memory 170 may store input data obtained from the input unit 120 , learning data, a learning model, a learning history, and the like.

The processor 180 may determine at least one executable operation of the AI device 100 based on information determined or generated using a data analysis algorithm or a machine learning algorithm. In addition, the processor 180 may control the components of the AI device 100 to perform the determined operation.

To this end, the processor 180 may request, search, receive, or utilize the data of the learning processor 130 or the memory 170, and may perform a predicted operation or an operation determined to be desirable among the at least one executable operation. It is possible to control the components of the AI device 100 to execute.

In this case, when the connection of the external device is required to perform the determined operation, the processor 180 may generate a control signal for controlling the corresponding external device and transmit the generated control signal to the corresponding external device.

The processor 180 may obtain intention information with respect to a user input and determine a user's requirement based on the obtained intention information.

In this case, the processor 180 uses at least one of a speech to text (STT) engine for converting a voice input into a character string or a natural language processing (NLP) engine for obtaining intention information of a natural language. Intention information corresponding to the input may be obtained.

At this time, at least one of the STT engine and the NLP engine may be configured as an artificial neural network, at least a part of which is learned according to a machine learning algorithm. And, at least one or more of the STT engine or the NLP engine is learned by the learning processor 130 , or learned by the learning processor 240 of the AI server 200 , or learned by distributed processing thereof. it could be

The processor 180 collects history information including the user's feedback on the operation contents or operation of the AI device 100 and stores it in the memory 170 or the learning processor 130, or the AI server 200 It can be transmitted to an external device. The collected historical information may be used to update the learning model.

The processor 180 may control at least some of the components of the AI device 100 in order to drive an application program stored in the memory 170 . Furthermore, in order to drive the application program, the processor 180 may operate two or more of the components included in the AI device 100 in combination with each other.

2 shows an artificial intelligence server 200 according to an embodiment of the present disclosure.

Referring to FIG. 2 , the AI server 200 may refer to a device that trains an artificial neural network using a machine learning algorithm or uses a learned artificial neural network. Here, the AI server 200 may be configured with a plurality of servers to perform distributed processing, and may be defined as a 5G network. In this case, the AI server 200 may be included as a part of the AI device 100 to perform at least a part of AI processing together.

The AI server 200 may include a communication unit 210 , a memory 230 , a learning processor 240 , and a processor 260 .

The communication unit 210 may transmit/receive data to and from an external device such as the AI device 100 .

The memory 230 may include a model storage unit 231 . The model storage unit 231 may store a model (or artificial neural network, 231a) being trained or learned through the learning processor 240 .

The learning processor 240 may train the artificial neural network 231a using the training data. The learning model may be used while being mounted on the AI server 200 of the artificial neural network, or may be used while being mounted on an external device such as the AI device 100 .

The learning model may be implemented in hardware, software, or a combination of hardware and software. When a part or all of the learning model is implemented in software, one or more instructions constituting the learning model may be stored in the memory 230 .

The processor 260 may infer a result value with respect to new input data using the learning model, and may generate a response or a control command based on the inferred result value.

3 shows an artificial intelligence system 1 according to an embodiment of the present disclosure.

Referring to FIG. 3 , the AI system 1 includes at least one of an AI server 200 , a robot 100a , an autonomous vehicle 100b , an XR device 100c , a smart phone 100d , or a home appliance 100e . It is connected to the cloud network 10 . Here, the robot 100a to which the AI technology is applied, the autonomous driving vehicle 100b, the XR device 100c, the smart phone 100d, or the home appliance 100e may be referred to as AI devices 100a to 100e.

The cloud network 10 may constitute a part of the cloud computing infrastructure or may refer to a network existing in the cloud computing infrastructure. Here, the cloud network 10 may be configured using a 3G network, a 4G or Long Term Evolution (LTE) network, or a 5G network.

That is, each of the devices 100a to 100e and 200 constituting the AI system 1 may be connected to each other through the cloud network 10 . In particular, each of the devices 100a to 100e and 200 may communicate with each other through the base station, but may also directly communicate with each other without passing through the base station.

The AI server 200 may include a server performing AI processing and a server performing an operation on big data.

The AI server 200 includes at least one or more of the AI devices constituting the AI system 1, such as a robot 100a, an autonomous vehicle 100b, an XR device 100c, a smartphone 100d, or a home appliance 100e. It is connected through the cloud network 10 and may help at least a part of AI processing of the connected AI devices 100a to 100e.

In this case, the AI server 200 may train the artificial neural network according to a machine learning algorithm on behalf of the AI devices 100a to 100e, and directly store the learning model or transmit it to the AI devices 100a to 100e.

At this time, the AI server 200 receives input data from the AI devices 100a to 100e, infers a result value with respect to the input data received using the learning model, and provides a response or control command based on the inferred result value. It can be generated and transmitted to the AI devices 100a to 100e.

Alternatively, the AI devices 100a to 100e may infer a result value with respect to input data using a direct learning model, and generate a response or a control command based on the inferred result value.

Hereinafter, various embodiments of the AI devices 100a to 100e to which the above-described technology is applied will be described. Here, the AI devices 100a to 100e shown in FIG. 3 can be viewed as specific examples of the AI device 100 shown in FIG. 1 .

<AI+Robot>

The robot 100a may be implemented as a guide robot, a transport robot, a cleaning robot, a wearable robot, an entertainment robot, a pet robot, an unmanned flying robot, etc. to which AI technology is applied.

The robot 100a may include a robot control module for controlling an operation, and the robot control module may mean a software module or a chip implemented as hardware.

The robot 100a obtains state information of the robot 100a by using sensor information obtained from various types of sensors, detects (recognizes) the surrounding environment and objects, generates map data, moves path and travels A plan may be determined, a response to a user interaction may be determined, or an action may be determined.

Here, the robot 100a may use sensor information obtained from at least one sensor among LiDAR, radar, and camera to determine a movement path and a travel plan.

The robot 100a may perform the above-described operations using a learning model composed of at least one artificial neural network. For example, the robot 100a may recognize a surrounding environment and an object using a learning model, and may determine an operation using the recognized surrounding environment information or object information. Here, the learning model may be directly learned from the robot 100a or learned from an external device such as the AI server 200 .

At this time, the robot 100a may perform an operation by generating a result by using the direct learning model, but transmits sensor information to an external device such as the AI server 200 and receives the result generated accordingly to perform the operation You may.

The robot 100a determines a movement path and travel plan using at least one of map data, object information detected from sensor information, or object information obtained from an external device, and controls the driving unit to apply the determined movement path and travel plan. Accordingly, the robot 100a may be driven.

The map data may include object identification information for various objects disposed in a space in which the robot 100a moves. For example, the map data may include object identification information for fixed objects such as walls and doors and movable objects such as flowerpots and desks. In addition, the object identification information may include a name, a type, a distance, a location, and the like.

In addition, the robot 100a may perform an operation or drive by controlling the driving unit based on the user's control/interaction. In this case, the robot 100a may acquire intention information of an interaction according to a user's motion or voice utterance, determine a response based on the acquired intention information, and perform the operation.

<AI + Autonomous Driving>

The autonomous driving vehicle 100b may be implemented as a mobile robot, a vehicle, an unmanned aerial vehicle, etc. by applying AI technology.

The autonomous driving vehicle 100b may include an autonomous driving control module for controlling an autonomous driving function, and the autonomous driving control module may mean a software module or a chip implemented as hardware. The autonomous driving control module may be included as a component of the autonomous driving vehicle 100b, or may be configured and connected to the outside of the autonomous driving vehicle 100b as separate hardware.

The autonomous driving vehicle 100b acquires state information of the autonomous driving vehicle 100b using sensor information obtained from various types of sensors, detects (recognizes) surrounding environments and objects, generates map data, A moving route and a driving plan may be determined, or an operation may be determined.

Here, the autonomous vehicle 100b may use sensor information obtained from at least one sensor among LiDAR, radar, and camera, similarly to the robot 100a, in order to determine a moving route and a driving plan.

In particular, the autonomous vehicle 100b may receive sensor information from external devices to recognize an environment or object for an area where the field of view is blocked or an area over a certain distance, or receive information recognized directly from external devices. .

The autonomous vehicle 100b may perform the above-described operations using a learning model composed of at least one artificial neural network. For example, the autonomous driving vehicle 100b may recognize a surrounding environment and an object using a learning model, and may determine a driving route using the recognized surrounding environment information or object information. Here, the learning model may be directly learned from the autonomous vehicle 100b or learned from an external device such as the AI server 200 .

In this case, the autonomous vehicle 100b may generate a result by using a direct learning model and perform an operation, but operates by transmitting sensor information to an external device such as the AI server 200 and receiving the result generated accordingly. can also be performed.

The autonomous driving vehicle 100b determines a moving route and a driving plan by using at least one of map data, object information detected from sensor information, or object information acquired from an external device, and controls the driving unit to determine the moving path and driving. The autonomous vehicle 100b may be driven according to a plan.

The map data may include object identification information for various objects disposed in a space (eg, a road) in which the autonomous vehicle 100b travels. For example, the map data may include object identification information on fixed objects such as street lights, rocks, and buildings, and movable objects such as vehicles and pedestrians. In addition, the object identification information may include a name, a type, a distance, a location, and the like.

In addition, the autonomous vehicle 100b may perform an operation or drive by controlling the driving unit based on the user's control/interaction. In this case, the autonomous vehicle 100b may acquire intention information of an interaction according to a user's motion or voice utterance, determine a response based on the obtained intention information, and perform the operation.

<AI+XR>

The XR apparatus 100c is an AI technology applied, a head-mount display (HMD), a head-up display (HUD) provided in a vehicle, a television, a mobile phone, a smart phone, a computer, a wearable device, a home appliance, and a digital signage. , a vehicle, a stationary robot, or a mobile robot.

The XR device 100c analyzes three-dimensional point cloud data or image data obtained through various sensors or from an external device to generate position data and attribute data for three-dimensional points, thereby providing information on surrounding space or real objects. It can be obtained and output by rendering the XR object to be output. For example, the XR apparatus 100c may output an XR object including additional information on the recognized object to correspond to the recognized object.

The XR apparatus 100c may perform the above operations by using a learning model composed of at least one artificial neural network. For example, the XR apparatus 100c may recognize a real object from 3D point cloud data or image data using a learning model, and may provide information corresponding to the recognized real object. Here, the learning model may be directly learned from the XR device 100c or learned from an external device such as the AI server 200 .

In this case, the XR device 100c may perform an operation by generating a result using the direct learning model, but transmits sensor information to an external device such as the AI server 200 and receives the result generated accordingly to perform the operation. can also be done

<AI+Robot+Autonomous Driving>

The robot 100a may be implemented as a guide robot, a transport robot, a cleaning robot, a wearable robot, an entertainment robot, a pet robot, an unmanned flying robot, etc. to which AI technology and autonomous driving technology are applied.

The robot 100a to which AI technology and autonomous driving technology are applied may mean a robot having an autonomous driving function or a robot 100a that interacts with the autonomous driving vehicle 100b.

The robot 100a having an autonomous driving function may collectively refer to devices that move by themselves according to a given movement line without user's control, or move by determining a movement line by themselves.

The robot 100a with the autonomous driving function and the autonomous driving vehicle 100b may use a common sensing method to determine one or more of a moving route or a driving plan. For example, the robot 100a having an autonomous driving function and the autonomous driving vehicle 100b may determine one or more of a movement route or a driving plan by using information sensed through lidar, radar, and camera.

The robot 100a interacting with the autonomous driving vehicle 100b exists separately from the autonomous driving vehicle 100b and is linked to the autonomous driving function inside the autonomous driving vehicle 100b or connected to the autonomous driving vehicle 100b. An operation associated with the user on board may be performed.

At this time, the robot 100a interacting with the autonomous driving vehicle 100b acquires sensor information on behalf of the autonomous driving vehicle 100b and provides it to the autonomous driving vehicle 100b, or obtains sensor information and obtains information about the surrounding environment or By generating object information and providing it to the autonomous driving vehicle 100b, the autonomous driving function of the autonomous driving vehicle 100b may be controlled or supported.

Alternatively, the robot 100a interacting with the autonomous driving vehicle 100b may monitor a user riding in the autonomous driving vehicle 100b or control a function of the autonomous driving vehicle 100b through interaction with the user. . For example, when it is determined that the driver is in a drowsy state, the robot 100a may activate the autonomous driving function of the autonomous driving vehicle 100b or assist the control of the driving unit of the autonomous driving vehicle 100b. Here, the function of the autonomous driving vehicle 100b controlled by the robot 100a may include not only an autonomous driving function, but also a function provided by a navigation system or an audio system provided in the autonomous driving vehicle 100b.

Alternatively, the robot 100a interacting with the autonomous driving vehicle 100b may provide information or assist a function to the autonomous driving vehicle 100b from the outside of the autonomous driving vehicle 100b. For example, the robot 100a may provide traffic information including signal information to the autonomous vehicle 100b, such as a smart traffic light, or interact with the autonomous vehicle 100b, such as an automatic electric charger for an electric vehicle. An electric charger can be automatically connected to the charging port.

<AI+Robot+XR>

The robot 100a may be implemented as a guide robot, a transport robot, a cleaning robot, a wearable robot, an entertainment robot, a pet robot, an unmanned flying robot, a drone, etc. to which AI technology and XR technology are applied.

The robot 100a to which the XR technology is applied may mean a robot that is a target of control/interaction within an XR image. In this case, the robot 100a is distinguished from the XR device 100c and may be interlocked with each other.

When the robot 100a, which is the target of control/interaction in the XR image, obtains sensor information from sensors including a camera, the robot 100a or the XR device 100c generates an XR image based on the sensor information. and the XR apparatus 100c may output the generated XR image. In addition, the robot 100a may operate based on a control signal input through the XR device 100c or a user's interaction.

For example, the user can check the XR image corresponding to the viewpoint of the remotely linked robot 100a through an external device such as the XR device 100c, and adjust the autonomous driving path of the robot 100a through interaction or , control motion or driving, or check information of surrounding objects.

<AI+Autonomous Driving+XR>

The autonomous vehicle 100b may be implemented as a mobile robot, a vehicle, an unmanned aerial vehicle, etc. by applying AI technology and XR technology.

The autonomous driving vehicle 100b to which the XR technology is applied may mean an autonomous driving vehicle equipped with a means for providing an XR image or an autonomous driving vehicle subject to control/interaction within the XR image. In particular, the autonomous driving vehicle 100b, which is the target of control/interaction in the XR image, is distinguished from the XR device 100c and may be interlocked with each other.

The autonomous driving vehicle 100b having means for providing an XR image may obtain sensor information from sensors including a camera, and output an XR image generated based on the acquired sensor information. For example, the autonomous vehicle 100b may provide an XR object corresponding to a real object or an object in the screen to the occupant by outputting an XR image with a HUD.

In this case, when the XR object is output to the HUD, at least a portion of the XR object may be output to overlap the actual object to which the passenger's gaze is directed. On the other hand, when the XR object is output to a display provided inside the autonomous driving vehicle 100b, at least a portion of the XR object may be output to overlap the object in the screen. For example, the autonomous vehicle 100b may output XR objects corresponding to objects such as a lane, other vehicles, traffic lights, traffic signs, two-wheeled vehicles, pedestrians, and buildings.

When the autonomous driving vehicle 100b, which is the subject of control/interaction in the XR image, acquires sensor information from sensors including a camera, the autonomous driving vehicle 100b or the XR device 100c performs An XR image is generated, and the XR apparatus 100c may output the generated XR image. In addition, the autonomous vehicle 100b may operate based on a control signal input through an external device such as the XR device 100c or a user's interaction.

4 shows the AI device 100 according to an embodiment of the present disclosure.

A description overlapping with FIG. 1 will be omitted.

In the present disclosure, the artificial intelligence apparatus 100 includes an edge device.

Referring to FIG. 4 , the input unit 120 includes a camera 121 for inputting an image signal, a microphone 122 for receiving an audio signal, and a user input unit for receiving information from a user. 123) may be included.

The voice data or image data collected by the input unit 120 may be analyzed and processed as a user's control command.

The input unit 120 is for inputting image information (or signal), audio information (or signal), data, or information input from a user. For input of image information, the AI device 100 may include one or more Cameras 121 may be provided.

The camera 121 processes an image frame such as a still image or a moving image obtained by an image sensor in a video call mode or a photographing mode. The processed image frame may be displayed on the display unit 151 or stored in the memory 170 .

The microphone 122 processes an external sound signal as electrical voice data. The processed voice data may be utilized in various ways according to a function (or a running application program) being performed by the AI device 100 . Meanwhile, various noise removal algorithms for removing noise generated in the process of receiving an external sound signal may be applied to the microphone 122 .

The user input unit 123 is for receiving information from a user, and when information is input through the user input unit 123 , the processor 180 may control the operation of the AI device 100 to correspond to the input information. .

The user input unit 123 is a mechanical input means (or a mechanical key, for example, a button located on the front/rear or side of the AI device 100, a dome switch, a jog wheel, a jog switch, etc.) and a touch input means. As an example, the touch input means consists of a virtual key, a soft key, or a visual key displayed on the touch screen through software processing, or a part other than the touch screen. It may be made of a touch key (touch key) disposed on the.

The sensing unit 140 may be referred to as a sensor unit.

The output unit 150 includes at least one of a display unit (Display Unit, 151), a sound output unit (Sound Output Unit, 152), a haptic module (Haptic Module, 153), and an optical output unit (Optical Output Unit, 154). can do.

The display unit 151 displays (outputs) information processed by the AI device 100 . For example, the display unit 151 may display execution screen information of an application program driven by the AI device 100 , or user interface (UI) and graphic user interface (GUI) information according to the execution screen information.

The display unit 151 may implement a touch screen by forming a layer structure with the touch sensor or being formed integrally with the touch sensor. Such a touch screen may function as the user input unit 123 providing an input interface between the AI device 100 and the user, and may provide an output interface between the terminal 100 and the user.

The sound output unit 152 may output audio data received from the communication unit 110 or stored in the memory 170 in a call signal reception, a call mode or a recording mode, a voice recognition mode, a broadcast reception mode, and the like.

The sound output unit 152 may include at least one of a receiver, a speaker, and a buzzer.

The haptic module 153 generates various tactile effects that the user can feel. A representative example of the tactile effect generated by the haptic module 153 may be vibration.

The light output unit 154 outputs a signal for notifying the occurrence of an event by using the light of the light source of the AI device 100 . Examples of the event generated by the AI device 100 may be message reception, call signal reception, missed call, alarm, schedule notification, email reception, information reception through an application, and the like.

5 is a block diagram for explaining the configuration of a laundry treatment apparatus according to an embodiment of the present invention.

Referring to FIG. 5 , the artificial intelligence device 100 may include a configuration of the laundry treatment device 300 . Alternatively, the artificial intelligence device 100 may be modularly configured as an internal component of the laundry treatment device 300 .

The laundry processing apparatus 300 may include the internal components of the artificial intelligence apparatus 100 and the laundry unit 310 shown above as parallel components.

The washing unit 310 includes at least one of a laundry module 311 performing a function related to washing, a drying module 312 performing a function related to drying, and a clothes management module 313 performing other functions related to managing clothes. may include.

The washing module 311 may perform washing-related functions such as immersion, washing, rinsing, and dehydration.

The drying module 312 may perform a function of drying laundry using various methods, and may typically dry laundry using wind (hot or cold air).

The clothing management module 313 may perform various functions related to clothing management, such as placing clothing, dry cleaning, dust removal, sterilization, wrinkle removal, and ironing.

The processor 180 or the control processor 314 provided in the washing unit 310 controls the components included in the washing module 311 , the drying module 312 , or the clothes management module 313 of the washing unit 310 to provide various It provides a washing function.

The input unit 120 and the sensor unit 140 may collect data related to user interaction related to use and control of the washing unit 310 . For example, the input unit 120 and the sensor unit 140 may collect course selection information and control information through voice or interaction.

The output unit 150 may output information related to the use and control of the washing unit 310 . For example, the output unit 150 may output course information, usage record, control information, and the like corresponding to washing, drying, and clothing management.

The memory 170 may store information related to the use and control of the washing unit 310 . For example, the memory 170 may store course information, usage records, control information, and the like corresponding to washing, drying, and clothing management.

Specifically, the washing module 311 includes a tub 311a for storing wash water, a drum 311b rotatably installed in the tub to receive laundry, a driving unit 311c for rotating the drum, and supplying the wash water. It may include a water supply unit 311d that circulates or discharges the wash water, a pump 311e that circulates or discharges the wash water, and a drain unit 311f that discharges the discharged wash water.

A drum 311b in which laundry is accommodated may be rotatably provided in the tub 311a. The drum 311b accommodates laundry, is disposed so that an inlet into which laundry is put is located on the front or upper surface, and is rotated about a horizontal or vertical rotation center line. A plurality of through holes may be formed in the drum 311b so that water in the tub 311a can be introduced into the drum 311b.

However, "horizontal" or "vertical" herein is not a term used in a mathematically strict sense. That is, even when the rotation center line is inclined at a predetermined angle with respect to the horizontal or vertical as in the embodiment, since it approaches the horizontal, it can be said that it is substantially horizontal or vertical.

The water supply unit 311d may include a water supply valve, a water supply pipe, and a water supply hose.

When water is supplied, wash water passing through the water supply valve and the water supply pipe may be mixed with the detergent in the detergent dispenser and then supplied to the tub 311a through the water supply hose.

On the other hand, a direct water supply pipe is connected to the water supply valve, so that the washing water can be directly supplied into the tub 311a without being mixed with the detergent through the direct water supply pipe.

The pump 311e functions as a drain pump 311e for discharging the wash water to the outside and a circulation pump 311e for circulating the wash water, but unlike the drain pump 311e and the circulation pump 311e, the drain pump 311e and the circulation pump 311e are separate. can be installed.

The pump 311e may be connected to a drain pipe provided in the drain unit 311f to discharge wash water to the outside through the drain pipe. In addition, the pump 311e may be connected to the circulating water supply pipe, and may spray the wash water stored in the tub 311a into the drum 311b through the circulating water supply pipe to circulate the wash water.

One or more protrusions protruding toward the inside of the drum 311b may be included on the inner surface of the drum 311b.

The protrusion may be a lifter disposed on the inner surface of the drum 311b or an embossing formed integrally. When a lifter is provided or embossing is formed on the inner surface of the drum 311b, laundry may be repeatedly lifted and dropped by the lifter when the drum 311b rotates.

The driving unit 311c rotates the drum 311b, and the driving shaft rotated by the driving unit 311c may pass through the rear surface of the tub 311a and be coupled to the drum 311b.

The driving unit 311c may include a motor capable of speed control.

In this case, the motor may be an inverter motor of a direct drive method.

The control processor 314 receives an output value (eg, output current) of the motor of the driving unit 311c as an input, and based on this, the number of rotations (or rotational speed) of the motor of the driving unit 311c is a preset target rotation. It can be controlled to follow the number (or target rotation speed). Also, the control processor 314 may control the driving of the motor of the driving unit 311c according to the driving pattern.

In addition, the drying module 312 includes a drum 312a into which laundry is put, a driving unit 312b for rotating the drum, a heating unit 312c for heating air, a blowing fan 312d for circulating internal air, and internal air. It may include an exhaust unit 312e for discharging the .

The drum 312a is a space in which a building is dried, and is rotatably installed inside the body. In addition, one or more lifters may be provided inside the drum 312a to lift and drop the building material.

The drum 312a is connected to an intake port (not shown), and air may be introduced into the drum 312a by the blowing fan 312d.

The driving unit 312b rotates the drum 312a, and a driving shaft rotated by the driving unit 312b may be coupled to the drum 312a.

The driving unit 312b may include a motor capable of speed control.

In this case, the motor may be an inverter motor of a direct drive method.

The control processor 314 receives an output value (eg, output current) of the motor of the driving unit 312b as an input, and based on this, the number of rotations (or rotational speed) of the motor of the driving unit 312b is a preset target rotation. It can be controlled to follow the number (or target rotation speed). Also, the control processor 314 may control the driving of the motor of the driving unit 312b according to the driving pattern.

The heating unit 312c may include a heating unit that heats the air inside the drum 312a or air introduced from the outside.

The heating unit heats the air using various energy sources such as gas or electric type, and in the case of electric type, a coil heater may be used.

The heating unit 312c may include a plurality of heating units, and each heating unit may not be identical to each other and may use various energy sources, and may have different performances.

The blowing fan 312d circulates the air heated by the heating unit 312c or the air inside the drum 312a.

The exhaust unit 312e serves to guide the air inside the drum 312a to be discharged to the outside, and may include an exhaust duct and an air filter.

In addition, the clothing management module 313 includes a clothing container 313a, which is a space in which clothing can be placed, a driving unit 313b for moving a holder provided in the clothing container 313a, a blowing fan 313c for circulating internal air, It may include an air filter 313d, a sterilization unit 313e, and a wrinkle management unit 313f.

The clothes container 313a is a space for storing clothes (or laundry) to be managed or processed, and may include a holder for fixing clothes. For example, the clothing container may include a hanger and a hook for mounting the hanger, or a three-dimensional shape such as a torso and a mannequin.

The clothing container 313a is connected to an intake port (not shown), and air may be introduced by the blowing fan 313c.

The driving unit 313b may drive the cradle to induce a preset movement with respect to the clothes mounted on the cradle.

For example, the driving unit 313b may operate to vibrate the holder and the clothes mounted on the holder according to a predetermined vibration pattern. By vibrating the mounted clothes, it is possible to remove dust or foreign substances attached or adhered to the clothes.

The driving unit 313b may include a motor capable of speed control.

In this case, the motor may be an inverter motor of a direct drive method.

The control processor 314 receives an output value (eg, output current) of the motor of the driving unit 313b as an input, and based on this, the number of rotations (or rotational speed) of the motor of the driving unit 313b is a preset target rotation. It can be controlled to follow the number (or target rotation speed). Also, the control processor 314 may control the driving of the motor of the driving unit 313b according to the driving pattern.

The blowing fan 313c supplies air introduced from the outside of the clothing container 313a or air from the inside of the clothing container 313a into the clothing container to circulate the air.

The blowing fan 313c may be installed so that the air supplied to the clothes mounted on the clothes container 313a collides with each other, or the air supply direction may be controlled.

For example, the blower fan 313c may spray air to the clothes mounted thereon to induce dust attached or adhered to the clothes to fall from the clothes, or may remove moisture from the clothes.

The air filter 313d filters dust or the like when the air inside the clothing container 313a is circulated or when the inside air is discharged to the outside.

The sterilizing unit 313e may include various sterilizing devices for sterilizing the mounted clothes.

For example, the sterilization device may include a sterilization device using ozone and a sterilization device using ultraviolet rays.

The wrinkle management unit 313f reduces or removes wrinkles of the placed clothes, and may include a steam supply, iron and ironing board.

The steam supply unit heats the supplied water to make steam, and may naturally supply the steam to the clothing container 313a or directly spray the steam on the mounted clothing.

The iron and ironing board are provided inside the clothing container 313a. In addition, the operation may be controlled according to the ironing operation information determined in consideration of the shape, location, and material of the ironing target garment.

In this case, the ironing operation information may include the location/movement of the iron and the ironing board, the ironing temperature/time, and the like.

The control processor 314 may control the movement of the iron and ironing board by controlling the driving unit separately provided in the driving unit 313b or the wrinkle management unit 313f, and may control the iron and the ironing board according to the ironing operation information. .

6 shows an overall flow diagram in the present disclosure.

Referring to FIG. 6 , the processor 180 of the laundry treatment apparatus 300 may obtain user image information ( S610 ). In an embodiment, the processor 180 may acquire user image information based on an image captured by a camera. In this case, the image captured by the camera may include an image including a plurality of images or a single image.

The processor 180 may obtain user image information from a camera, identify a user included in the user image information using the obtained user image information, and obtain the user's behavior information ( S620 ). In an embodiment, the processor 180 extracts feature information from user image information to identify a user matching the feature information of the corresponding user image information, and identifies behavior information corresponding to the feature information.

The processor 180 may obtain laundry information corresponding to user and behavior information (S630). In an embodiment, the processor 180 may obtain laundry information corresponding to the user and behavior information by inputting user image information into the artificial intelligence model. As another example, the processor 180 may obtain laundry information output by the artificial intelligence model by inputting user information and behavior information into the artificial intelligence model.

The example is a case in which feature information is extracted from user image information and input to an artificial intelligence model, and when user image information is input to an artificial intelligence model, feature information is extracted by an algorithm inside the artificial intelligence model, and corresponding to the feature information The laundry information is output and may be changed according to the configuration of the pre-processing process or algorithm, but is not limited to the above example.

When the processor 180 obtains laundry information corresponding to the user and behavior information, the processor 180 may provide customized settings to the user behavior according to the obtained laundry information (S640).

The processor 180 may perform an operation corresponding to the laundry information according to a setting customized to the user's behavior.

7 is a diagram illustrating a usage scenario of the present disclosure.

Referring to FIG. 7 , a user 610 puts laundry for washing in a laundry basket and approaches the washing machine.

The camera 121 may sense image information of an approaching user. The processor 180 may acquire user image information sensed by the camera 121 . In this case, the user image information may include information about the user 610 and behavior information for approaching with a laundry basket.

The processor 180 may extract features of the acquired user image information. The processor 180 may identify the user by using the feature information extracted from the user image information and recognize the user's behavior information. In this case, an artificial intelligence model may be used to extract the features, and the artificial intelligence model uses computer vision technology or extracts edge information of an image and classifies it into a predefined category, or an object and a user's face By detecting the user, it is possible to identify the user and recognize the user's behavior.

The processor 180 may input user 610 information and behavior information, 'access with a laundry basket,' into the AI model. Alternatively, the processor 180 may input user image information into the artificial intelligence model. The laundry processing apparatus may recognize the laundry information output by the artificial intelligence model as a user 610 customized laundry course, and perform an operation corresponding to the laundry information. Hereinafter, the operation of the artificial intelligence model will be described in detail with reference to FIG. 8 .

8 is a diagram illustrating an artificial intelligence model of the present disclosure.

The processor 180 may obtain the user image information received from the camera 121 and extract the feature 820 included in the user image information 810 . The feature 820 included in the user image information may include user identification information for distinguishing who the user is and behavior information containing the user's intention. That is, the camera 121 senses a plurality of user image information, and the processor 180 inputs the plurality of user images into the artificial intelligence model 830 to determine the user's intention, and the user's intention is the When related to the operation of the laundry processing device, an operation corresponding to the laundry information may be performed using the artificial intelligence model.

In this case, the user identification information may refer to characteristic information capable of distinguishing each user who uses the laundry treatment apparatus. And the behavior information is a behavioral pattern analysis of the user to classify whether the user's behavior included in the user image information is a behavior related to the laundry processing device, if the user's behavior is related to the laundry processing device, 1) laundry or It may refer to characteristic information including an action related to drying or 2) an action not related to laundry but for using an additional function included in the laundry treatment apparatus.

Specifically, the behavioral information is related to 'simple behavior' not related to the laundry treatment device and the laundry treatment device, and is related to 'access with an object' or the laundry treatment device, which is an operation related to washing or drying laundry, and the laundry treatment device It can include actions such as 'power on/off', 'check remaining amount of detergent and replenish detergent' using additional functions included in .

For example, the 'simple action' may include a case in which the user 810 passes around the laundry processing apparatus. Alternatively, it may include a case in which the user 810 is simply detected in the angle of view of the camera 121 of the laundry treatment device to obtain user image information.

Also, 'approaching with an object' may include a case in which a user approaches the laundry processing apparatus with an object to do laundry. In this case, the object may include at least one of a laundry basket, a detergent basket, and other laundry that means individual laundry.

In addition, as described above, the use of 'additional functions included in the laundry treatment apparatus' is not directly related to washing or drying laundry, such as power on/off operation and checking the remaining amount of detergent, but laundry necessary for washing or drying laundry. It may mean a function of a processing device.

The processor 180 may perform an operation corresponding to the laundry information by using the laundry information output by the artificial intelligence model. For example, if the user's behavioral information is related to laundry processing and the object is detected as a laundry basket, the artificial intelligence model uses the laundry information as 'washing time 13 minutes, rinsing 3 times, dehydration 4 times, water temperature, cold water, and water intensity'. It is possible to output washing information such as The processor 180 may control the laundry processing apparatus so that the laundry processing apparatus performs an operation corresponding to the laundry information.

In an embodiment of the present disclosure, when the user image information 810 is input, the artificial intelligence model 830 may output laundry information 840 as a result value. In this case, the artificial intelligence model may include a machine learning algorithm that learns based on empirical data and performs prediction. Representatively, it may include an algorithm such as a decision tree, a Bayesian network, a support vector machine (SVM), and an artificial neural network (ANN). In general, artificial neural networks calculate the output value from the following three factors: (1) the connection pattern between neurons in different layers (2) the learning process to update the weight of the connection (3) the weighted sum of the input received from the previous layer It can be defined by the activation function it creates In addition, the artificial neural network may include network models such as Deep Neural Network (DNN), Recurrent Neural Network (RNN), Bidirectional Recurrent Deep Neural Network (BRDNN), Multilayer Perceptron (MLP), Convolutional Neural Network (CNN). may be, but is not limited thereto.

Specifically, the input layer of the artificial intelligence model of the present disclosure extracts features of user image information and transmits them to the output layer. The output layer receives the feature signal from the hidden layer, and outputs laundry information, which is an output value based on the received signal. The input layer of the present disclosure may use user image information or a feature 820 included in user image information as an input value.

Hereinafter, a learning process of the artificial intelligence model 830 of the present disclosure will be described.

The artificial intelligence model of the present disclosure may include a machine learning algorithm trained to output laundry information corresponding to the user and the behavior information when user image information is input. Specifically, in the training method of the artificial intelligence model 830, the artificial intelligence model uses a usage history including past user and past behavior information and laundry information corresponding to the past user and past behavior information as learning data. Can be used. It will be described with reference to FIG. 9 below.

9 is a flowchart illustrating a learning process of an artificial intelligence model of the present disclosure.

The processor 180 of the present disclosure may identify a user using user image information (S910) and recognize user behavior information (S920). The processor 180 may extract a feature included in the user behavior information and determine whether an object is identified in the feature information (S930). When it is determined that the object is identified, the processor 180 may acquire an operation setting related to laundry processing in order to collect training data of the artificial intelligence model ( S940 ). The laundry setting related to the laundry treatment may include an operation of the laundry treatment apparatus including at least one of a washing time, a number of rinsing times, a number of spins, a washing water temperature, and a water current strength. In addition, laundry settings related to laundry treatment may be obtained when the user uses the laundry treatment apparatus.

The processor 180 may set the user and the user's behavior information and operation settings related to laundry processing as a usage history, and generate the usage history as training data of the artificial intelligence model ( S950 ). In this case, the training data may be stored in the memory 170 . The processor 180 may train the artificial intelligence model using the training data (S960).

For example, when the first user approaches the washing machine while carrying the laundry basket to do laundry, and operates the laundry treatment apparatus by setting the washing water temperature to cold water and strong current for washing 3 times and spin-drying 4 times, the processor of the present disclosure may acquire a first image including a feature of 'approaching with a laundry basket', which is the first user and behavior information of the first user, among a plurality of user images acquired by the camera.

The user may turn on the washing machine and input a washing operation into the washing machine. For example, the rinsing 3 times, dehydration 4 times, washing water temperature, cold water, and water strength strong may be set as washing information.

After acquiring the first image, the processor 180 determines that the first user, the first user approaches with a laundry basket (behavioral information), rinses 3 times, spins 4 times, washing water temperature, cold water, and water intensity strong (washing information) ) can be obtained through usage history. The usage history may be stored in the memory 170 .

The processor 180 may acquire a usage history corresponding to the user for a certain period, set the usage history as training data of the AI model, and train the AI model.

The artificial intelligence model learned using the training data may output laundry information corresponding to the behavior information of the first user when the first user and data including the behavior information of the first user are input.

Therefore, when the first user uses the learned artificial intelligence model, the first user-customized washing operation output by the learned artificial intelligence model may include the washing operation set by the first user in the past while holding the object. can In this case, the learning data of the learned artificial intelligence model may include a usage history including past user and past behavior information and laundry information corresponding to the past user and past behavior information.

In addition, in an embodiment of the present disclosure, when the object included in the behavior information is different, the characteristic information included in the user image information is also different, so the characteristic input to the artificial intelligence model is also changed.

Specifically, the object may include at least one of a laundry basket, a detergent basket, and other laundry, and correspondingly, the first user-customized laundry operation may be determined differently for each type of the laundry basket, the detergent basket, and the other laundry. can

For example, when the object is a laundry basket, laundry information including an overall laundry operation may be output as an output value of the artificial intelligence model, and in the case of a detergent basket, laundry information including an operation for detergent input may be output. Alternatively, if the object is other laundry (socks, towels, etc.) that does not include a basket, laundry information (cotton laundry, wool laundry, etc.) suitable for the other laundry may be output.

In an embodiment of the present disclosure, when the object is not identified (S930 - no), the processor 180 sets an additional operation to a laundry operation not related to laundry processing in order to collect training data of the artificial intelligence model. can be obtained (S941).

The operation additional to the washing operation may include at least one of turning on the power of the laundry processing apparatus and notifying the remaining amount of detergent.

For example, the first user accesses the washing machine. The first user opens the detergent container in which the laundry detergent is stored in order to check the remaining amount of laundry detergent.

The processor 180 may acquire a second image including a characteristic of 'approaching the washing machine to open the detergent container', which is the first user and behavior information of the first user among the plurality of user image information.

When the user performs an operation of opening the detergent container of the washing machine, the processor 180 may set the 'operation of opening the detergent container' as laundry information.

In addition, the processor 180 may acquire the first user and the operation of the first user 'accessing the washing machine' (action information) and opening the detergent container (washing information) as the usage history. The usage history may be stored in the memory 170 .

The processor 180 may acquire a usage history corresponding to the user for a certain period of time, set the usage history as training data of the AI model, and train the AI model.

That is, the learning data of the artificial intelligence model may include the first user as the user, access to the laundry processing device as the behavior information, and 'at least one of power on and checking the remaining amount of detergent' as the laundry information. .

Therefore, when the first user uses the learned artificial intelligence model, the processor 180 of the laundry processing device detects the first user from the user image information, and the behavior information of the first user is 'to the laundry processing device. In the case of 'approaching', an operation including at least one of power-on and detergent remaining amount notification output by the artificial intelligence model may be performed.

In an embodiment of the present disclosure, when the object is not identified and the additional function of the laundry processing apparatus is not operated, the processor 180 converts the behavior information included in the corresponding user image information into a 'simple behavior' unrelated to laundry processing. It can make judgments and generate training data for artificial intelligence models.

Specifically, in the case of 'simple behavior' not related to the laundry treatment, user image information, behavior information corresponding to the user image information, and 'do not perform any operation' may be generated as training data.

For example, when the user simply passes in front of the laundry treatment device, the processor 180 determines the user and the user's 'simple walking' as behavioral information, and when the user image information of the simple walking user is input, the laundry treatment device returns to the current state. It is possible to train the artificial intelligence model by setting the laundry information that maintains or does not perform additional operations as the correct value.

Thereafter, when a similar simple gait of the user is detected, the processor 180 maintains the current state or does not perform an additional operation. As described above, the artificial intelligence model identifies the user, collects behavioral information corresponding to the user, and outputs laundry information based on the user's behavioral pattern, thereby providing a washing operation suitable for the situation.

The scenario of FIG. 7 may be implemented through the processes described with reference to FIGS. 8 to 9 .

10 is a flowchart when using the learned artificial intelligence model of the present disclosure.

Referring to FIG. 10 , the camera of the laundry processing apparatus may sense user image information. The processor 180 may identify user and behavior information included in the user image information. The processor 180 may input user image information or feature data included in the user image information into the artificial intelligence model and perform an operation corresponding to the laundry information output by the artificial intelligence model.

If the learned user is included in the user image information input to the artificial intelligence model and the feature of 'approach with an object', which is the learned user's behavior information, is included, the artificial intelligence device outputs user-customized laundry information and the processor 180 may perform an operation corresponding to the user-customized laundry information.

In addition, when the learned user is included in the user image information input to the artificial intelligence model, and the feature of 'access to the laundry treatment device without having an object', which is the learned user's behavior information, the artificial intelligence device is The laundry information corresponding to providing an additional operation to the laundry operation may be output, and the processor may perform an operation corresponding to the laundry information.

In addition, when the learned user is included in the user image information input to the artificial intelligence model, and the feature of 'simple walking', which is the learned user's behavior information, is included, the artificial intelligence device returns the laundry information of 'not working'. output, and the processor may maintain the current state until additional user image information is sensed.

11 is a diagram illustrating a scenario in which an object is included in action information in user image information obtained by the laundry processing apparatus of the present disclosure.

Referring to FIG. 11 , a user 1110 may approach the laundry treatment apparatus 100 while carrying a laundry basket 1120 . The front camera of the laundry treatment apparatus 100 may sense the user image information 1100 of the approaching user 1110 . The processor 180 may obtain the user image information 1100 and input it into the artificial intelligence model to obtain laundry information. Alternatively, the characteristic data of the user 1110 and the characteristic data of the user's behavior information 1120 obtained in the pre-processing of the user image information 1100 may be input to the artificial intelligence model.

The artificial intelligence model of the laundry processing apparatus 100 may output laundry information corresponding to the user 1110 and the user's behavior information 1120 when data is input to the input layer. As described above, since the artificial intelligence model has been learned using the past usage history of the user 1110, when the user image information 1100 is input, the laundry information corresponding to the user 1110 and the user's behavior information 1120 is Washing information such as rinsing 3 times, spin-drying 4 times, water temperature, cold water, and the strength of the water can be output. Accordingly, when the user 1110 approaches the laundry treatment apparatus 1120 while carrying the laundry basket, the laundry treatment apparatus 100 may determine the user's washing intention and set a laundry operation.

12 is a flowchart when user image information is obtained during the operation of the laundry treatment apparatus of the present disclosure.

In an embodiment, since the laundry operation that the laundry treatment apparatus can perform during the operation of the laundry treatment apparatus is limited, laundry information to be obtained during the operation of the laundry treatment apparatus also varies.

For example, in the laundry treatment apparatus of the present disclosure, it is assumed that a user approaches the laundry treatment apparatus while carrying the additional laundry and inputs the additional laundry during a washing operation.

The camera of the laundry treatment apparatus may sense user image information. The processor 180 may identify the user from the user image information and recognize the behavior information. If the user image information, additional laundry may be identified as an object. The processor 180 sets the user included in the user image information and the behavior information 'access with additional laundry' and 'open additional laundry input container' as the usage history, and generates the set usage history as training data. can

As another example, in the laundry treatment apparatus of the present disclosure, it is assumed that a user approaches the laundry treatment apparatus to check the remaining amount of detergent and injects detergent into the detergent container of the laundry treatment apparatus during a washing operation.

The processor 180 of the laundry treatment device sets 'access to the laundry treatment device' and 'opens the detergent container', which are user and behavior information included in the obtained user image, as usage history, and generates the set usage history as training data. can The processor may train the artificial intelligence model using the usage history collected for a certain period of time.

13 is a flowchart illustrating an operation process of an artificial intelligence model when user image information is acquired during operation of the laundry processing apparatus.

The artificial intelligence model learned according to the flowchart of FIG. 12 may be activated when the laundry processing apparatus receives user image information during a washing operation. Specifically, in the processor of the present disclosure, when the first user is detected from the user image information during the operation of the laundry treatment device, and the behavior information of the first user is 'approach to the laundry treatment device', the artificial intelligence model is output One detergent remaining amount notification operation may be performed (S1310, S1320, S1331).

In another embodiment, the processor detects the first user from the user image information during the operation of the laundry processing device, and when the behavior information of the first user is 'approaching with an object', the artificial intelligence model This output laundry addition operation may be performed (S1330).

14 is a scenario in which user image information is obtained during the operation of the laundry treatment apparatus of the present disclosure.

Referring to FIG. 14 , the user 1410 may access the washing machine in the washing operation to insert additional laundry. The processor 180 of the washing machine may obtain the user image information 1400 and input the obtained user image information into the learned artificial intelligence model. Since the acquired user image information 1400 includes the characteristics of the user 1410 and the user's behavior information 1420 through the past usage history, the learned artificial intelligence model is the user 1410 and the additional laundry the user has. The laundry information of 'open additional laundry input container', which is an operation corresponding to 1420 , is output, and the processor may perform an additional laundry input container opening operation.

In addition, the output unit may display a notification related to the input of additional laundry using voice or menu information.

In one embodiment, the communication unit of the laundry processing apparatus of the present disclosure communicates with the server, and the processor records a usage history including laundry information corresponding to the user and the behavior information in the past and the user and the behavior information to the server It is possible to use the trained artificial intelligence model in such a way that it transmits to and receives the learned artificial intelligence model from the server.

The laundry processing apparatus of the present disclosure generates a plurality of users, behavior information of each of the plurality of users, and laundry information corresponding to the behavior information of each of the plurality of users as a usage history, and uses the usage history to learn an artificial intelligence model. It is possible to perform a customized washing operation corresponding to each user and behavior information of each user. And, since the laundry treatment apparatus uses data including user behavior information, it is possible to perform situation recognition according to the user behavior information.

In addition, by providing an individually customized operation for each user, even if the same behavior information is different for different users, the corresponding customized washing operation may also be different. Specifically, when the second user is detected from the user image information and the behavior information of the second user is 'approaching with an object', the processor may perform the second user-customized washing operation output by the artificial intelligence model. . In addition, the second customized washing operation may be different from the first customized washing operation.

The operations may be performed simultaneously and are not limited to the order in which they are performed. In addition, the present disclosure may consist of software, firmware, or a combination of software or firmware. And the differences related to these modifications and applications should be construed as being included in the scope of the present invention defined in the appended claims.

The present disclosure described above can be implemented as computer-readable code on a medium in which a program is recorded. The computer-readable medium includes all types of recording devices in which data readable by a computer system is stored. Examples of computer-readable media include Hard Disk Drive (HDD), Solid State Disk (SSD), Silicon Disk Drive (SDD), ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage device, etc. There is this. In addition, the computer may include a processor 180 of the terminal.

In addition, although the above description has been focused on services and embodiments, these are merely examples and do not limit the present invention, and those of ordinary skill in the art to which the present invention pertains will not deviate from the essential characteristics of the present services and embodiments. It can be seen that various modifications and applications not exemplified above are possible. For example, each component specifically shown in the embodiments can be implemented by modification. And the differences related to these modifications and applications should be construed as being included in the scope of the present invention defined in the appended claims.

Claims (10)

a washing unit performing a washing operation;
an output unit for displaying information corresponding to the washing operation;
a camera for sensing user image information;
A processor for inputting the user image information into an artificial intelligence model to obtain laundry information corresponding to 'user' and 'action information', and performing an operation corresponding to the laundry information,
laundry handling device. The method of claim 1
The artificial intelligence model is
Using, as learning data, a usage history including past user and past behavior information, and laundry information corresponding to the past user and past behavior information,
Comprising a machine learning algorithm trained to output 'washing information corresponding to the user and the behavior information' when the user image information is input,
laundry handling device. 3. The method of claim 2,
The processor detects the first user from the user image information, and when the behavior information of the first user is 'approaching to the laundry processing device', at least one of power-on and detergent remaining amount notification output by the artificial intelligence model performing an operation comprising
laundry handling device. 3. The method of claim 2
When the first user is detected from the user image information and the behavior information of the first user is 'approaching with an object', the processor performs a first user-customized washing operation output by the artificial intelligence model,
The first user-customized washing operation includes a washing operation set by the first user in the past while carrying the object,
laundry handling device. 5. The method of claim 4
the object is
at least one of laundry baskets, detergent baskets and other laundry;
The first user-customized laundry operation is different for each type of the laundry basket, the detergent basket and the other laundry
laundry handling device. 3. The method of claim 2,
When the first user is detected from the user image information during the operation of the laundry treatment device, and the behavior information of the first user is 'approaching the laundry treatment device', the processor is configured to notify the remaining detergent amount output by the artificial intelligence model performing an action,
laundry handling device. The method of claim 1
When the first user is detected from the user image information during the operation of the laundry processing device, and the behavior information of the first user is 'approaching with an object', the processor adds laundry output from the artificial intelligence model performing an action,
laundry handling device. 5. The method of claim 4,
The processor performs a second user-customized washing operation output by the artificial intelligence model when a second user is detected from the user image information and the behavior information of the second user is 'approaching with an object',
the second customized washing operation is different from the first customized washing operation;
laundry handling device. The method of claim 1
Further comprising a communication unit for communicating with the server,
The processor transmits to the server a usage history including the user and the behavior information in the past and laundry information corresponding to the user and the behavior information, and receives the artificial intelligence model learned from the server.
laundry handling device. sensing user image information;
obtaining laundry information corresponding to 'user' and 'action information' by inputting the user image information into an artificial intelligence model; and performing an operation corresponding to the laundry information,
A method of operation of a laundry treatment device.

Images