KR20240049992A - Information processing method, processing system, and processing apparatus for a home appliance - Google Patents

Abstract

The information processing system for home appliances according to the present invention includes at least one home appliance; A terminal capable of processing information of the at least one home appliance: an individual service area where individual services are provided; And it may include an information processing device that is connected to the terminal and the at least one home appliance through a network and provides an application service using the individual service. This system can identify user needs for individual services and provide corresponding application services.

Description

Information processing method, processing system, and processing apparatus for a home appliance}

The present invention relates to a home appliance information processing method, a home appliance information processing system, and a home appliance information processing device.

Home appliances mainly refer to devices installed in the user's home to help the user with housework. Home appliances can be fixed in the user's home and operate according to the user's commands. Users can schedule and operate home appliances. For example, if you are scheduled to return home at 5 o'clock, you can make a reservation in advance to have the air conditioner operate from 4:30 o'clock. According to this example, users who return home can enjoy a clean indoor environment.

Home appliances can access the Internet through a wired or wireless communication network, such as Wi-Fi. Users can collect information about the operation of home appliances through the Internet. Users can connect to the control server of home appliances through the Internet. In this way, various services are possible because users in remote locations and servers of home appliances can communicate with each other. For example, there is a way to obtain a desired cooking recipe by referring to the food ingredients in the refrigerator. For example, there is KR20190093814A 'Kitchen appliance system capable of supporting cooking according to recommended recipes and support method thereof'.

Through the above technologies, users can use home appliances more conveniently. Although the automatic operation of these various home appliances was expected to greatly change daily life, the reality is that it is not being utilized as much as expected. The inventor thought deeply about this problem and came up with the present invention.

The inventor found that even if the user operates the home appliance according to operable information, there is a major problem in that it does not operate in the state desired by the user in the end. For example, when operating a cooking appliance according to the cited invention, great inconvenience may be experienced if the recipe conditions are slightly different. Specifically, if the cooking appliance is operated according to the recommended recipe even though the amount of food ingredients is insufficient, the ingredients may burn and smoke may be generated. Because of this anxiety, users do not actively utilize technology despite its convenience.

The inventor found that because the operations of home appliances do not cooperate with each other, there are many cases where the device does not operate in the desired situation. For example, you can receive suggestions for cooking recipes using food ingredients stored in the refrigerator. At this time, depending on the cooking recipe, excessive oil vapor may completely contaminate the room. In this way, we came to the conclusion that it is difficult for consumers to use various new technologies in an environment where active collaboration between home appliances is absent.

In many other cases, it has been found that automatic operation and collaborative operation of home appliances can actually cause various problems such as excessive energy waste, unnecessary damage, and undesirable conditions.

KR20190093814A Kitchen appliance system capable of supporting cooking according to recommended recipes and support method thereof

The present invention proposes a home appliance information processing method, a home appliance information processing system, and a home appliance information processing device that can respond to a user's schedule.

The present invention proposes a home appliance information processing method, a home appliance information processing system, and a home appliance information processing device that can optimally operate home appliances even when the user is not aware of them.

The present invention proposes a home appliance information processing method, a home appliance information processing system, and a home appliance information processing device that further activate the usability of home appliances using external information.

For further problems of the present invention, refer to the contents disclosed in the specific details for carrying out the invention.

The information processing system for home appliances according to the present invention includes at least one home appliance; A terminal capable of processing information of the at least one home appliance: an individual service area where individual services are provided; And it may include an information processing device that is connected to the terminal and the at least one home appliance through a network and provides an application service using the individual service. This system can identify user needs for individual services and provide corresponding application services.

Optionally, the information processing device receives basic information about the user using at least one home appliance and the terminal using at least one of the home appliance and the terminal; Using the individual service, the requested information requested by the user can be confirmed from the basic information.

Optionally, confirming first information of a first home appliance among the at least one home appliance; And in order to provide the requested information, it may include transmitting second information generated with reference to the first information to at least the first home appliance. Accordingly, more accurate information can be provided in response to the user's request. You can accurately understand user requests and respond to those requests.

Optionally, if the user approves the second information, the second information may be transmitted to a second home appliance among the at least one home appliance. Accordingly, the user can conveniently control the second home appliance.

Optionally, the first home appliance may be a refrigerator, and the second home appliance may be a cooking appliance. Users can conveniently control refrigerators and cooking appliances. Users can conveniently control two collaborative home appliances at the same time.

Optionally, the first home appliance may be a robot vacuum cleaner, and the second home appliance may be an air purifier. Users can conveniently control robot vacuum cleaners and air purifiers. Users can conveniently control two collaborative home appliances at the same time.

Optionally, the second information may include operation information of the second home appliance, and/or operation sequence information of the first and second home appliances. Optionally, the first information may be stored in at least one of a memory of the information processing device and the first home appliance. Accordingly, collaboration between the two home appliances can become smoother.

The information processing device of the present invention includes an I/O module that communicates with external home appliances in response to a node request; And when there is a trigger signal including request information related to the user's cooking, it may include an operation decision module that checks first information on the first home appliance among the at least one home appliance. Optionally, the information processing device may include an I/O module that communicates with an external home appliance in response to a node request; And when there is a trigger signal including request information related to the user's cooking, an operation decision to transmit second information generated with reference to the first information to the first home appliance in order to provide the request information. Can contain modules. Optionally, the information processing device may include an I/O module that communicates with an external home appliance in response to a node request; And when there is a trigger signal containing request information related to the user's cooking, when it is determined to perform the second information, it may include an operation decision module that transmits the second information to the second home appliance. there is. Optionally, the second information may include operation sequence information of the first home appliance and the second home appliance. With the information processing device, home appliances can be used reliably and conveniently.

Optionally, the first information may be stored in at least one of the memory of the first home appliance and the information processing device.

Optionally, the second information may be transmitted to the user's terminal. Users can check the above information and conveniently use home appliances.

Optionally, the pair of the first home appliance and the second home appliance may include at least one of a refrigerator and an exposed cooking appliance, a refrigerator and an enclosed cooking appliance, a hood and an air purifier, and a robot vacuum cleaner and an air purifier. . According to this, users can conveniently use home appliances collaboratively in various environments related to cooking.

The information processing method of the present invention includes receiving a trigger signal from a node; The information processing device may use an individual service in an individual service area to identify cooking request information as the trigger signal. According to this, it is possible to accurately identify the information requested by the user.

Optionally, the information processing device may determine the status of food ingredients in the refrigerator and transmit first information, which is cooking information corresponding to the cooking request information, to the node. You can check the condition of the food inside the refrigerator and use it. Optionally, with the node's approval, the first information can be transmitted to the cooking appliance. According to this, the determined cooking method can be conveniently implemented.

Optionally, the cooking appliance may perform cooking according to the first information. Users can become more convenient.

Optionally, in the case where the cooking appliance is an exposed heating cooking appliance, when cooking starts, operation of the air purifier may be stopped and the hood may be controlled to operate. Users can perfectly utilize the operation of cooking appliances, air purifiers, and hoods. Even if the user does not know how to properly operate each home appliance, the home appliance can operate properly on its own.

Optionally, when the cooking appliance is a closed heating cooking appliance, when the door is opened after cooking, the operation of the air purifier can be stopped and the hood can be controlled to operate. Even if the user does not know how to properly operate each home appliance, the home appliance can operate properly on its own.

The information processing method of the present invention includes the information processing device recommending corresponding information including status information of a first home appliance in response to the request information and transmitting it to the node; And if there is approval from the node, it may include transmitting the corresponding information to a second home appliance. Users can conveniently use home appliances.

Optionally, the first home appliance may be a refrigerator, and the second home appliance may be an exposed cooking appliance. Optionally, the first home appliance may be a refrigerator, and the second home appliance may be a sealed cooking appliance. Optionally, the first home appliance may be a hood, and the second home appliance may be an air purifier. Optionally, the first home appliance may be a robot vacuum cleaner, and the second home appliance may be an air purifier. Cooperative operation of two or more home appliances can be implemented in different ways in response to each combination.

Optionally, the corresponding information may include operation sequence information of the first home appliance and the second home appliance. By applying the operation sequence of each home appliance, you can use the home appliance conveniently. Each home appliance can be operated efficiently.

According to the present invention, a user can change the reservation operation of a home appliance simply by informing the user of his or her schedule. According to this, there is an advantage in that the optimal operation of the home appliance is possible even when the user is not aware of the reserved operation.

According to the present invention, even if there is no scheduled operation of the home appliance, it is possible to operate the home appliance in an energy-saving environment in response to the user's needs.

According to the present invention, there is an advantage in that the user's life is enriched by artificial intelligence tasks that match the operation of home appliances and the user's schedule.

According to the present invention, at least two home appliances can perform cooperative operation. According to this, an optimal indoor environment can be established.

For further effects of the present invention, refer to the contents disclosed in the specific details for carrying out the invention.

1 is a diagram showing an information processing system for home appliances according to an embodiment.
Figure 2 is a configuration diagram of an information processing system for home appliances showing the cloud system in more detail.
Figure 3 is a diagram illustrating an information processing device for a home appliance in an embodiment.
Figure 4 is a diagram illustrating an embodiment of providing a learning model.
Figure 5 is a diagram showing the flow of an information processing method according to an embodiment.
Figure 6 is a diagram showing the interface of the information processing method of the embodiment.
Figure 7 is a diagram showing the interface of the information processing method of the embodiment.
Figure 8 is a flowchart of an information processing method for home appliances according to an embodiment.
Figure 9 is a flowchart of an information processing method for home appliances according to an embodiment.
Figure 10 is a flowchart of an information processing method for home appliances according to an embodiment.
11 is a flowchart of an information processing method for home appliances according to an embodiment.

Hereinafter, specific embodiments of the present invention will be described in detail with reference to the drawings. However, the spirit of the present invention is not limited to the embodiments presented below, and those skilled in the art who understand the spirit of the present invention will be able to see other embodiments included within the scope of the same spirit, including the addition of limitations on components or elements, It can be easily proposed by changes, deletions, additions, etc., but it is said that this is also included within the scope of the present invention. The present invention may have many embodiments in which the idea is implemented, and in each embodiment, any part may be replaced with a corresponding part or a related part of another embodiment. The present invention may be one of the examples presented below, or a combination of two or more.

In this application, the information processing method, information processing system, and information processing device examine the same inventive idea from different aspects. Any shortcomings in one embodiment can be supplemented by explanation of other embodiments. Any part of one embodiment may be expanded or changed according to the description of another embodiment.

In the description of the drawings, identical or similar components are assigned the same reference numbers regardless of reference numerals, and duplicate descriptions thereof may be omitted.

The suffixes “module” and “part” for components used in the following description are given or used interchangeably only for the ease of preparing the specification, and do not have distinct meanings or roles in themselves.

In describing the embodiments disclosed in this application, if it is determined that detailed descriptions of related known technologies may obscure the gist of the embodiments disclosed in this specification, the detailed descriptions will be omitted.

The attached drawings are only for easy understanding of the embodiments disclosed in this specification, and the technical idea disclosed in this specification is not limited by the attached drawings, and all changes, equivalents, and changes included in the spirit and technical scope of the present invention are not limited. It should be understood to include water or substitutes.

Terms containing ordinal numbers, such as first, second, etc., may be used to describe various components, but the components are not limited by the terms. The above terms are used only for the purpose of distinguishing one component from another.

When a component is said to be “connected” or “connected” to another component, it is understood that it may be directly connected or connected to the other component, but that other components may exist in between. It should be. On the other hand, when it is mentioned that a component is “directly connected” or “directly connected” to another component, it should be understood that there are no other components in between.

Singular expressions may include plural expressions, unless the context clearly dictates otherwise.

In this application, terms such as “comprise” or “have” are intended to designate the presence of features, numbers, steps, operations, components, parts, or combinations thereof described in the specification, but are not intended to indicate the presence of one or more other features. It should be understood that it does not exclude in advance the possibility of the existence or addition of elements, numbers, steps, operations, components, parts, or combinations thereof.

In this application, home appliances may also include devices of the same type, but used for commercial, public, and other purposes other than at home.

In this application, the service may include a processor that provides processing results for requests for a specific purpose. At this time, service may be an abbreviation for a service module. Here, the processor may include hardware, software running on specific hardware, or software running on general-purpose hardware. The service may refer to certain data.

In the description of the method of the present application, even if the first step (eg, S1) is located before the second step (eg, S2), the first step does not necessarily precede the second step. Depending on the content of the embodiment, the steps may change.

Figure 1 is a diagram showing an information processing system for home appliances according to an embodiment.

Referring to Figure 1, the information processing system for home appliances may include a user area (1) and an operator area (2). The user area 1 may be a dominant area for users who use home appliances. The operator area 2 may be the control area of an operator who provides home appliances, operates a system, or provides services.

The user area 1 may include a building 1a. Home appliances 1b can be placed in the building 1a. The building 1a may be connected to any structure to which a network is connected. The home appliance 1b may include home appliances such as massagers, water purifiers, vacuum cleaners, clothes care machines, refrigerators, washing machines, air purifiers, cookers, and air conditioners. It is not limited to this, and may include various devices such as automobiles, plant cultivators, and pumps. The user area 1 may include a terminal 1c. The user 1d can perform communication using the terminal 1c. The terminal 1c can be linked to any structure to which a network is connected. The terminal may be compatible with the home appliance. The terminal may include at least one. The terminal may include at least two for direct or indirect interaction between terminals.

The operator area 2 may include a server 2c. The server 2c can perform at least some functions of the information processing device of the present invention. The server 2c may be an operation server that operates the home appliance. In this case, the operation information of the home appliance can be obtained. The server 2c may be a communication server that communicates with the terminal. The server 2c may be a server of the manufacturer of the home appliance. The server 2c may be a server of an operator of the home appliance. In this case, management of the home appliance can be performed. For example, quality control can be performed. The server 2c may include a first server 2a that communicates with the home appliance 1b. The server 2c may include a second server 2b that communicates with the terminal 1c. The first and second servers 2a and 2b can communicate with each other. The first and second servers 2a and 2b may not be separated but may be configured as a single device. The first and second servers 2a and 2b may be implemented by at least one cloud system.

The first server 2a may receive usage information including the operating status of the home appliance 1b. For example, usage information such as operation time, operation mode, and failure status of the home appliance 1b can be received. The first server 2a can control the operation of the home appliance. The second server 2b can transmit the operating status of the home appliance 1b and necessary information to the terminal 1c. Here, the necessary information may include all information helpful to the user 1d in relation to the home appliance. The second server 2b can receive request information from the terminal 1c. Here, the request information may be control information of the home appliance (1b) desired by the user.

The first server 2a may transmit status information of the home appliance 1b to the second server 2b. The second server 2b may transmit control information of the home appliance 1b to the first server 2a. The second server 2a can provide services to users. Here, the service may include operating the home appliance in response to the user's needs.

The interaction between the server 2c, the home appliance 1b, the terminal 1c, and the user 1d will be briefly described.

The server can manage all home appliances that are the target of the service in order to perform the service. The home appliance is connected to the server through communication and can execute commands sent from the server. The service request may be received by the server through the home appliance. The terminal can apply for a service request. The terminal may refer to devices having a communication module, such as mobile phones, smartphones, laptops, and computers. The terminal can communicate directly with the server. The server can communicate with the home appliance.

The home appliance may have a unique device ID. The ID may be registered and managed on the server. When the home appliance accesses the server, the server can specify the home appliance through the device ID. The server may manage characteristic information of the home appliance along with the device ID.

The home appliance may be fixed within a building. Stable communication with the server may be possible through an AP within the building. The terminal may communicate indirectly with the home appliance through the server.

Users can request and receive services for home appliances registered on the server through the home appliances. However, it may not be easy to request the service from the server through a home appliance. In this case, the service can be requested from the server through a device carried by the user, such as a terminal. To achieve this, an application program capable of implementing this may be installed on the terminal.

Users can request service by accessing the server and entering the unique device ID of the home appliance for which they wish to receive service. Users may be given a user ID and password. Users can log in to the site operated by the server and register all of their home appliances. Home appliances registered to the server may be registered to the server along with the corresponding user ID. This can be done by transmitting the user ID and the unique device ID of the home appliance matched with the user ID to the server through the site. Through these processes, the server can store user information related to the user and home appliance information related to the home appliance. User information and home appliance information can be matched and stored on the server.

The user can register as a member of the site through an application installed on the terminal and be given a user ID and password. The user can recognize the application as the applicant's ThinQ app. Users can register all of their home appliances with the above application. The application may be provided for remote service of at least one home appliance.

Users can request services through the terminal and easily obtain information such as service performance and completion. The service processing process includes receiving a service request to a server through at least one of the site, the terminal, and the home appliance, processing the service request by the server, and processing the service according to the processing code. This may include executing a command for a service.

Figure 2 is a configuration diagram of an information processing system for home appliances showing the cloud system in more detail.

Referring to Figure 2, the information processing system of the embodiment. It can be divided into an individual service area (10) and an application service area (11). In the individual service area 10, basic information for operating the information processing method, information processing device, and information processing system of the embodiment can be processed. The basic information may be the first information for requesting an application service. For example, it may be the user's voice saying that they are going out. The voice is basic information, and the service of the individual service area 10 can understand the meaning of the voice.

The individual service area 10 may include at least one processor for individual services. The application service area 11 can provide application services to users by applying individual services supplied from the individual service area 10. The application service area 10 may include one or more processors for application services. The processor may include hardware and software running on the hardware. The servers 2a and 2b may be implemented on the cloud system 3. The servers 2a and 2b may be implemented as one component of the cloud system, but for ease of understanding and differences in operation, they are displayed separately as shown in the drawing.

In the application service area 11, a home appliance 1b and a server 2c can be placed. The home appliance may include a number of devices already described. The server may communicate with the home appliance. Application services provided in the application service area may include services directly requested by the user, services indirectly requested by the user, services that can be considered requested by the user, and services that can be assumed to have been requested by the user. The application service can be determined by the server. The server can control home appliances. The server can receive information about home appliances. The server can communicate with the user through a terminal or home appliance.

The individual service area 10 may include a cloud system 3 and an edge device. The edge device may be a user's terminal 1c as an example. Here, the terminal may mean at least two terminals beyond the terminal of a single individual. The cloud system 3 can have large capacity and high-speed information processing capabilities. The cloud system can perform a heavy individual service 352 that requires a lot of resources to operate the service. The edge device may have small capacity and low-speed information processing capabilities. The edge device can provide a lightweight individual service 351 that requires high-speed operation relative to accuracy.

The server 2c may initiate service in response to any one of several requests, such as a request from the terminal 1c, a request from the home appliance 1b, and fulfillment of preset requirements.

If there is a request for the service, the server 2c can perform information processing corresponding to the requested service. The information processing can be performed by referring to information previously stored in the memory 40. The server may call at least one service of the individual service area 10 to perform the requested service. The server can use a combination of two or more services from individual service areas. The server can use edge devices for individual services that require rapid performance. The server can use the cloud system 3 for individual services that require accurate performance.

The performance results of the requested service may be in various forms, such as reporting to the user, controlling home appliances, and information storage. The performance result of the requested service may correspond to the requested service.

The performance process and results of the requested service can be notified to the terminal 1c and the home appliance 1b.

A number of individual services may be provided in the cloud system. The individual service may be performed by various service nodes such as cloud virtual/physical machines, virtual/physical boxes, virtual/physical switches, and clusters.

The individual service area may include an external management service 31 that manages cloud and edge devices. The management service may include an interface. The management service can be provided to cloud and edge devices. The individual service area may include an expression service 32 that provides a method to increase recognition corresponding to the user. The individual service area may include a butler service 33 that identifies the user's needs. The individual service area may include a recognition enhancement service 34 to improve the performance of the butler service and the service. The individual service area can provide recognition service 35.

Each of the above individual services will be described in more detail. The individual services may include various services presented below.

The management service 31 may include an application programming interface (API). The API may be a connection between computers or computer programs. It is a type of software interface and can provide services to other types of software. The API can be applied to edge devices.

The management service 31 may include an API gateway. The API gateway may be a traffic manager that connects to the actual backend service or data and protects sensitive data by applying policies, authentication, and general access controls to API calls. Lee Sang-gi's API gateway can perform authentication/authorization, security, statistics, and monitoring. A REST API may be an API that adheres to the design principles of the REST (REpresentational State Transfer) architectural style. REST API can be said to comply with several principles applied in cloud services.

The expression service 32 may include text-to-speech (TTS), that is, voice synthesis. The TTS is a technology in which a machine automatically generates sound waves of speech sounds. For example, the speech sounds of a person selected as a model were recorded, divided into certain speech units, then coded and input into a speech synthesizer (speech computer, speech synthesizer). Then, according to instructions, only the necessary speech units were recombined to create speech sounds artificially. It is a technology that creates The TTS may have various implementation methods.

The expression service 32 may include AI sound. The AI sound refers to a technology that uses artificial intelligence (AI) to listen to sound and recognize the situation. For example, a scream or a sudden stop of a vehicle can be judged by listening to the sound.

The expression service 32 may include appearance technology. An example of the facial expression technology is technology that estimates feelings by looking at a face.

The Butler service (33) receives text, image information, sensor information, and situation information, which are the results of recognizing the user's voice input, as input data, and judges it based on the user's profile, context, and knowledge to provide natural control and response. It can be made possible. The Butler service may use recognition technology to analyze data received as input. The Butler service can be linked to external services by determining it as a knowledge base built on a Knowledge Management System (KMS). The Butler service 33 may include user behavior analysis, external knowledge connection, external knowledge analysis, situation judgment and response proposal, and intention execution judgment.

The Butler service 33 can provide control and response tailored to the conversation context and user situation by utilizing the functions of the AI module that existed independently. Users can feel customized services for themselves through the Butler service 33.

The Butler service 33 may include a service gateway that can access external knowledge. The service gateway is a gateway for accessing external services. The gateway may refer to a computer or software that enables communication between networks using different communication networks and protocols in a computer network. For example, it may be a network point that acts as an entrance from one network to another network.

The recognition enhancement service 34 may include a learning processor. The cognitive enhancement service 34 may include continuous learning. Continual learning is a method of making one model capable of handling multiple tasks by upgrading it little by little. In other words, it is a method of improving model performance by continuously learning new data/tasks each time new learning is done, without forgetting previously learned data.

The recognition enhancement service 34 may include a knowledge graph. The Knowledge Graph may refer to a collection of interconnected descriptions of entities such as entities, events, or concepts. The knowledge graph can store linking and semantic metadata within the context. This approach can provide a framework for data integration, unification, analysis, and sharing.

The recognition service 35 may include a speech service. The speech module may include various processors. The speech module can be used most accurately and conveniently by remote users. The speech module has developed a lot compared to other modules.

The speech service may include a speech signal processing (SSP) service. The voice signal processing service is a function for applications such as voice communication and voice recognition. The sound source input to the microphone is of poor quality because it contains various noises from the surrounding area. Since low-quality sound sources cannot be used as is in voice calls or voice recognition programs, the SSP engine can be used to remove noise and improve the quality of the sound source.

The speech may include a voice recognition (ASR: Automatic Speech Recognition) service. The voice recognition service can provide a service that receives the user's voice data and converts the voice into text. The sound ‘going out’ can be changed to the word ‘going out’.

The speech may include a Natural Language Processing (NLP) service. Natural language processing service can mean converting natural language, which is human language, into a semantic structure that machines can understand. The word 'going out' can mean understood as going out: going out of the house.

The speech may include a Hybrid Trigger Word Detection (HTWD) service. The HTWD service may include firstly performing embedded-based startup word detection and secondly performing server-based startup word detection to minimize misrecognition of startup words.

The speech may include a Trigger Word Detection (TWD) service. The TWD service can construct speech data sets and implement algorithms for trigger word detection (also known as keyword detection or wake word detection). Trigger word detection allows devices such as Amazon Alexa, Google Home, Apple Siri, and Baidu DuerOS to hear a specific word and trigger the device to wake up. TWD may mean performing maneuver word detection with a small capacity engine. For example, when a user says, ‘Enable going out mode,’ the word ‘going out’ can be searched as a trigger word.

The speech may include a Multi-device Wakeup Arbitration (MWA) service. The MWA service may be a service that selects the most appropriate device for the user when multiple devices recognize the same startup word at the same time. The MWA service measures the user's direction, distance, etc., selects the device that can best recognize the user's voice commands, and provides feedback to the user. For example, when a startup word is recognized for an air conditioner, a television, and an oven at the same time, a service for selecting a suitable device can be provided.

Among the speech services, the Speech Signal Processing (SSP) service and the Trigger Word Detection (TWD) service may be included in the edge device 1c. Because these two services are lightweight, they can fully run on the edge device's capacity. This is because the above two services can be conveniently used for edge device purposes.

The recognition service 35 can express a vision service. The vision service may include various sub-services. The vision service may correspond to services that users can perceive with their eyes.

The vision service may include food recognition service. The food recognition service can recognize the location of food stored in the refrigerator. For example, using an image of an egg, you can photograph the egg and find out the location of the egg from the photographed image. The food recognition service can recognize food in various ways, such as shape and barcode. Recognized foods can be managed on the server.

The vision service may include object/person recognition services. The person/object recognition service can distinguish the target person/object using the two-dimensional and three-dimensional shape of the object. The classified information can be used for various purposes such as identification, movement, and recognition.

The vision service may include a Driver State Monitoring (DSM) service. The driver state detection service can detect the user's face from video image data, track gaze, and track eye closure and eye blinking to provide the location of the face, head direction, facial feature points, gaze direction and angle, and degree of eye closure. The driver state detection service can detect the driver's drowsiness and issue a warning.

The vision service may include a behavior recognition service. The behavior recognition service may be a service that recognizes the user's behavior as an activation signal. For example, it may be a service that starts the terminal by the user shaking the terminal. The DSM service and behavior recognition service can be implemented in edge devices.

The recognition service 35 may include a biometric service. The biometric service may be an example of any service that determines the user's health information. For example, a diagnostic assistance service can be performed using heart rate, information, and sleep information transmitted using a smartwatch.

The recognition service 35 may include a sensor service. The sensor service can perform services such as recognizing the environment and biometrics through transmitted information.

The recognition service 35 may include an emotion service. The emotion service may include an emotion recognition (ER) service. Emotion recognition service is a service that recognizes emotions from the user's facial expression, voice, or text entered by the user. ER services that recognize emotions enable natural verbal and non-verbal communication between users and machines, helping people and machines communicate and interact smoothly.

In the individual service area 10, various other services such as gait tracking and behavior prediction can be applied. Each service in the individual service area 10 can provide other services by linking one service with another service. The services of the edge device 1c may be equipped with services that require speed. The services of the individual service area 10 can be linked to other external networks.

The cloud system may be equipped with services that require accuracy and complexity. The edge device and the cloud system can cooperate with each other. For example, after starting up with a word recognized in the TWD of the edge device 1c, the cloud system 3 can accurately determine using the HTWD. As another example, behavior recognition recognized in the vision service of the edge device (1c) can be used as information for other purposes in the object/person recognition service of the cloud system (3).

Each service in the individual service area 10 can operate as a learning model of artificial intelligence. The recognition enhancement service 34 can be constructed as a learning model. For example, at least one of the expression service 32, the butler service 33, the recognition enhancement service 34, and the recognition service 35 may be provided as a learning model using artificial intelligence. Each service in the individual service area 10 may be implemented as a table or an operation formula constructed as a mathematical formula. Each service in an individual service area may contain a mixture of learning models and calculation formulas using artificial intelligence.

Each service in the individual service area 10, that is, the service module, can return the processing result of the service request to the application service area.

In all of the following embodiments, there may be no specific mention of the service module of the individual service area called by the application service area. In this case, all individual service modules that can reply can reply to the service request, corresponding to the nature of the call by the application service in the application service area. There may be specific reference to the service module of the individual service area called by the application service area. In this case, a preferred example is described, and it does not exclude that service modules in other individual service areas reply.

Figure 3 is a diagram explaining an information processing device for a home appliance of an embodiment.

Referring to FIG. 3, the information processing device 12 of the embodiment includes an I/O module 20 that exchanges information with each external node, a memory 40 that stores information necessary for information processing, and the I/O It may include an operation determination module 30 that determines the operation of each node in response to the situation input by the module. The node may include home appliances and terminals. The nodes do not exclude terminals, cloud systems, and edge devices. Information flow between the I/O module, the memory, and the operation decision module may be performed by an internal bus. The I/O module, the memory, and the operation decision module may share information with each other.

The information processing device 12 can use individual services of the individual service area 10. The information processing device 12 can provide application services. The information processing device 12 may be a main processor that provides application services. The information processing device 12 may include a separate processor that provides application services. The information processing device 12 can be connected to the terminal 1c and the home appliances 1b1 and 1b2 through a wired or wireless network. The information processing device 12 may be implemented by the server 2c.

The information processing device 12 can start operation by a trigger signal. The trigger signal may mean a starting signal. The trigger signal may be basic information. The basic information may be a simple signal that is not interpreted in itself. The trigger signal may be a direct or indirect signal from the user. For example, the trigger signal may be a user's voice message. In this case, the meaning can be understood after the voice message is processed into natural language. The individual service used at this time may be the natural language processing service described above.

The trigger signal may be a direct or indirect signal generated from each node connected to the home appliance. The trigger signal may be the fulfillment of a reserved condition. For example, when waking up in the morning is scheduled for 6 o'clock, it may be a signal from the clock that has reached 6 o'clock. As another example, it may be a user detection signal when automatically detecting a user.

The information processing device 12 can request an individual service from the individual service area using basic information. Here, the individual service may include understanding the meaning of the trigger signal, which is the basic information. At this time, the speech service of the recognition service 35 can be used as an individual service. In the case of fulfillment of the reserved condition, the meaning has already been interpreted, so understanding the meaning may not be necessary even though it is a trigger signal.

The I/O module 20 can receive the trigger signal. The I/O module 20 may transmit a signal to each node in response to the trigger signal. The operating state of the home appliance can be known by the response signal from each node. The application service may include checking the operating status of home appliances. The above home appliance can be used by multiple people. The home appliance may have a reservation operation scheduled in advance. For example, the I/O module can determine information such as whether the home appliance is currently operating, waiting for operation, whether operation has ended, and what the specific contents of the operation standby state are. The information identified by the I/O module 20 may be stored in memory for each home appliance.

With reference to the information identified by the I/O module 20, the operation determination module 30 can determine the operation of the home appliance. The application service may include determining the operation of the home appliance. The operation decision module may determine an operation in response to the trigger signal. The operation determination module can determine an operation corresponding to the user's service request.

The operation decision module may include an operation sequence determination unit 31. The operation sequence determination unit 31 may determine the operation sequence of at least two home appliances that interoperate with each other. Here, the operation sequence may be different from the operation sequence of existing home appliances confirmed by the I/O module. The operation decision module may include an operation mode determination unit 31. The operation mode determination unit 31 may determine the operation mode of at least one home appliance. Here, the operation mode may be different from the operation mode of the home appliance confirmed by the I/O module. The operation determination module may include an operation time determination unit 33. The operation time determination unit 33 may determine the operation time of at least one home appliance. Here, the operation time may be different from the operation time of the home appliance confirmed by the I/O module.

The operation decision module 30 can operate with artificial intelligence. The operation decision module 30 can operate using the individual service of the individual service area 10. For example, it can be operated using the learning model of the recognition enhancement service 34.

Figure 4 is a diagram explaining an embodiment of providing a learning model.

Referring to FIG. 4, the AI (Artificial Intelligence) server 2d may refer to a device that trains an artificial neural network using a machine learning algorithm or uses a learned artificial neural network. Here, the AI server 2d may be composed of a plurality of servers to perform distributed processing, and may be defined as a 5G network. At this time, the AI server 2d may be included as a part of the display device 25 and may perform at least part of the AI processing. The AI server 2d may be part of the server 2c.

The AI server 2d may include a communication unit 21, a memory 23, a learning processor 24, and a response processor 26. The communication unit 21 can transmit and receive data with an external device such as the display device 25.

The memory 23 may include a model storage unit 23-1. The model storage unit 23-1 may store a model being trained or learned through the learning processor 24 (or artificial neural network, 23-2).

The learning processor 24 can train the artificial neural network 23-2 using training data. The learning model may be used while mounted on the AI server 2d of the artificial neural network, or may be mounted and used on an external device such as the display device 25.

The learning model may be implemented in hardware, software, or a combination of hardware and software. When part or all of the learning model is implemented in software, one or more instructions constituting the learning model may be stored in the memory 63.

The response processor 26 may infer a result value for new input data using a learning model and generate a response or control command based on the inferred result value.

The learning model implemented by the AI server can be provided for each service provided in the individual service area. Each learning model can provide a response for each service. The operation decision module 30 can receive a response using each learning model.

Figure 5 is a diagram showing the flow of an information processing method according to an embodiment.

Referring to FIG. 5, the information processing device 12 can receive the trigger signal from the node (S1). The node may be a terminal or a home appliance. The information processing device can request an individual service from the individual service area (S2). Here, the individual service may include understanding the meaning of the trigger signal. For example, the speech service of the recognition service 35 can be used.

The individual service area can perform result notification as a result of responding to the request of the information processing device (S3). The result notification may include at least one of the meaning of the trigger signal, a list of required operations corresponding to the trigger signal, and a list of operations required for the node.

The information processing device may perform functions such as selecting a required operation of a home appliance in response to a trigger signal, and selecting an operation required by a user in response to a trigger signal. The information processing device can provide application services. The information processing device can notify the node of the meaning of the trigger signal, the necessary operation corresponding to the trigger signal, the operation required from the node, and the confirmation result as a first result reply (S4).

The information processing device may transmit a first control signal to the first home appliance (S5). The first home appliance may transmit an Eck signal and a status report signal including the execution result of the first control signal (S6). The information processing device can transmit a second control signal to the second home appliance (S7). The second home appliance may transmit an Eck signal and a status report signal including the execution result of the second control signal (S8).

The information processing device may notify the node of the performance results of the first and second home appliances as a second result reply to the node (S9). The node may include more home appliances in addition to the first and second home appliances. The node may include any one of the first and second home appliances. At least one step of the information processing method may be omitted in the information processing method of another embodiment. At least one step of the information processing method of another embodiment may be added to the information processing method.

An information processing method for home appliances of the embodiment will be described.

Figure 6 is a diagram showing the interface of the information processing method of the embodiment. Terms used in the interface in the drawings are directly quoted from the description of the embodiment for convenience of understanding. Other terms and methods with similar meaning may be used. Figure 6 illustrates cooking by a user.

Referring to FIG. 6, the user 1d can transmit that he or she is cooking food using the terminal 1c. The trigger signal may be “Tell me the breakfast menu,” or “Send the recipe to my mobile phone.” The user 1d can transmit that he or she is cooking using the home appliance 1b. The trigger signal may be a voice saying “Tell me the breakfast menu,” or “Send the recipe to my cell phone,” or a video of the user approaching the refrigerator. The method of transmitting the trigger signal is exemplified by voice or video. In addition, it can be delivered to the information processing device 10 in various ways, such as text messages and agreed actions. If a video of the user approaching the refrigerator after waking up is learned by the AI server, the user approaching the refrigerator can be recognized as a trigger signal. This step may correspond to the notification step (S1) in FIG. 5.

The user's message can be understood to mean that the user is cooking it. The intention that the user cooks can be interpreted by at least one service among the recognition service 35, butler service 33, and recognition enhancement service 34 provided as a service in the individual service area 10. The interpreted results may include the intent that the user is cooking, and the actions that the home appliance should perform in response to the user's cooking. This step may correspond to the inquiry step (S2) and result notification step (S3) in FIG. 5.

In response to the above interpreted result, a reply may be provided from the information processing device to the terminal 1c in the form of “Yes, you are cooking.” The interpreted results can be transmitted to home appliances. The home appliance that has received the message can reply to the user with “Yes, you are cooking.” The home appliance may include at least one. The user (1d) who received the reply can confirm that the information processing system has operated correctly. This step may correspond to the first result response step (S4) of FIG. 5.

In response to the interpreted results, the information processing device can determine the current state of the home appliance by transmitting a control signal to the device. The home appliance may transmit an ec/status report in response to the control signal. This step may correspond to the control signal and ec/status report (S5 to S8) of FIG. 5. If there is information that the user should pay attention to in the check/status reports (S6) (S8) of the home appliance, the second result response step (S9) of FIG. 5 can be performed. For example, if there is no signal because the home appliance is not operating normally, that effect can be sent as a second result reply. The second result reply may be omitted.

In response to the interpreted results, the information processing device can provide an application service. The information processing device can determine control signals for performing application services. The information processing device can transmit and control the control signal to the home appliance. In this case, it may be the case that the check/status report (S6) (S8) of the home appliance has been performed normally. The application service may include at least one of the operation sequence, operation mode, and operation time of the home appliance.

In response to the analyzed result, the information processing device may determine at least one of the operation sequence, operation mode, and operation time of at least one home appliance. The information processing device can control the home appliance using at least one of the determined operation sequence, operation mode, and operation time. The information processing device can notify the control result of the home appliance to the terminal 1c. This step can be called the third result reply.

Examples of each reply in the above third result reply are given. For the refrigerator, the information processing device may say, “This is the breakfast menu you have reserved and the location of the ingredients.” In another case, it could be “The recipe was sent to ThinQ.”

The third result reply may also be sent through the terminal as above. Alternatively, in the third result reply, the corresponding home appliance may transmit the message. This is because the user may be in front of the refrigerator while notifying cooking. The message can be delivered directly to the user. The message may be of various types, such as a stored voice, an alarm, and a stored sign.

The third result reply step may have more diverse cases depending on the information processing method of the home appliance. The user can know that the application service has been properly performed through the third result reply.

In the information processing method of the embodiment, the information processing method of FIG. 6 can be sequentially performed for the basic information, “Tell me the breakfast menu” and “Send the recipe to my mobile phone.” The information processing device of the embodiment may first respond to the first, second, and third results for “Tell me the breakfast menu.” In this case, the third result reply may be “Here are the breakfast menu you reserved and the location of the ingredients.” Afterwards, you can continue to reply to the 1st, 2nd, and 3rd results for “Send the recipe to my cell phone.” In this case, the third result reply might be “Recipe sent to ThinQ.”

The information processing method of the embodiment may additionally transmit the recipe to at least one of the terminal and the cooking appliance as a third result reply of "Send the recipe to the mobile phone." In this way, the information processing method of the embodiment can use home appliances to benefit the user through the cooperation of various home appliances. When the cooking device receives the recipe, there may be a message “You can cook conveniently with the light wave oven” as the third result reply.

If the data storage capacity of the home appliance is large, the information processing method of the embodiment may be performed directly between the home appliance and the user. In this case, the home appliance can later report performance information to the server.

Figure 7 is a diagram showing the interface of the information processing method of the embodiment. Terms used in the interface in the drawings are directly quoted from the description of the embodiment for convenience of understanding. Other terms and methods with similar meaning may be used. Figure 7 shows a case where cooking is performed using a cooking device.

Other contents are the same as the description of FIG. 6 above and the differences are explained.

First, the basic information may be voice information asking ‘How is the cooking going?’ The third result reply indicating the application service corresponding to the basic information may be various depending on the interpreted result. For example, a cooking device can transmit the message “Adjust the temperature using the chicken stew recipe.” The hood may send a message saying “The hood is operating to remove harmful gases in the kitchen.” The robot vacuum cleaner may send a message saying “After cooking is completed, we will proceed to floor mopping mode.” The air purifier may send a message saying “Air purifier while cooking.” You can send a message saying, “We will temporarily suspend filtering.”

The third result reply may also be sent through the terminal as above. Alternatively, in the third result reply, the corresponding home appliance may transmit the message. This is because the user may be at home while cooking. The message can be delivered directly to the user. The message may be of various types, such as stored voice, alarm, and stored message.

Below, various embodiments of information processing methods for home appliances will be described.

Figure 8 is a flowchart of an information processing method for home appliances according to an embodiment.

Referring to FIG. 8, the user 1d can transmit that he or she is cooking using the terminal 1c (S11). Users can transmit their voice.

The intention to go out can be understood by at least one service among the recognition service 35, butler service 33, and recognition enhancement service 34, which are provided as services in the individual service area 10. For example, through the recognition service 35, the meaning of cooking can be understood through a learning model. Afterwards, application services can be provided. For example, the interpreted result may include the intention that the user is cooking and the actions that the home appliance should perform in response to the user cooking (S12) (S13).

In response to the interpreted result, the I/O module 20 may perform a first result response step (S14) to confirm to the user whether the cooking is correct.

In response to the analyzed result, the I/O module 20 can determine the current state of the home appliance by transmitting a control signal to the home appliance. The identified results may be delivered to the user as a second result including the status of the home appliance (S15).

In response to the analyzed result, the operation decision module 30 may determine at least one of the operation order, operation mode, and operation time of at least one home appliance. The information processing device can control the home appliance using at least one of the determined operation sequence, operation mode, and operation time. The information processing device can notify the terminal 1c of the control result of the home appliance as a third result (S16). The home appliance may directly send a third result reply to the user.

While performing the information processing method, the I/O module and the operation decision module may use individual services in the individual service area. The user can know the progress and results of the application service through the first, second, and third result replies.

Figure 9 is a flowchart of an information processing method for home appliances according to an embodiment. This embodiment may be an embodiment after the user's cooking intent has been identified. For example, it may be an information processing method after the user's cooking is identified by the information processing method of FIG. 5.

Referring to Figure 9, the I/O module 20 can determine the current state of the home appliance (S21) (S21a) (S21b) by transmitting the first control signal to the home appliance (1b). The home appliance can transmit an ACK message and a status report message (S22) (S22a) (S22b). The home appliance can transmit an ACK message and status report signal to the I/O module 20. Through the bus inside the information processing device, each module of the information processing device can share information. The status report message may be transmitted to the operation decision module.

The operation mode determination unit 32 may receive the status report message. The operation decision module may refer to the status report message and determine the operation sequence, operation mode, and operation time of the home appliance. Depending on the type of the home appliance and/or the status report message, the home appliance may be determined differently. At this time, at least two home appliances can work together.

The above information processing method will be explained with an example.

Take a refrigerator as an example. The basic information of the user's “Tell me the breakfast menu” can be recognized as the purpose of starting cooking. The I/O module 20 can transmit a first control signal to the refrigerator (S21a) and receive a status report message about the food ingredient information and location of the refrigerator (S22a). In addition, the operation mode determination unit 32 can inquire with the memory 40 to receive various information such as food ingredient information, location, and expiration date stored in the refrigerator. If there is a discrepancy between the two pieces of information, you can request confirmation or correction.

Information about food ingredients in the refrigerator can be obtained in advance. For example, food ingredients stored in the refrigerator and food inside the refrigerator through information such as camera information taken inside the refrigerator, image information taken when putting food ingredients into the refrigerator, user's handwriting input, and barcode of food ingredients. You can know the location of the material. In addition, information such as a list of foods and expiration dates can also be known. These food ingredient information can be input by terminals, refrigerators, and other devices when storing food ingredients and stored on the server.

The operation determination module 30 can determine whether food can be cooked according to the food ingredient information. At this time, you can decide on the food menu and recipe. The operation decision module 30 can read and inform the user of menus and recipes previously saved. The operation decision module 30 can transmit menu information to the refrigerator or terminal (S23) and receive a response of approval, denial, or restart (S24).

In the case of the approval, the operation sequence determination unit 31, the operation mode determination unit 32, and the operation time determination unit 33 may transmit information to control at least two home appliances including at least a cooker ( S25a)(25b)(25c). In the above case, the user can cook on his own regardless of recommendations. In the case of the above restart, other menus and recipes can be recommended.

In the case of the approval, the operation sequence determination unit 31 performs the first process among the first process of removing food ingredients from the refrigerator as the first home appliance and the second process of cooking in the cooking appliance as the second home appliance. You can guide them to do it first. The operation mode determination unit 32 may perform a mode for guiding the withdrawal of food ingredients for a refrigerator, which is a first home appliance, and transmit and store menus and recipes for a cooking appliance, which is a second home appliance. . The operation time determination unit 33 can convert the decision information of the operation sequence determination unit 31 into time and guide it to each home appliance. For example, in order to maintain the freshness of food ingredients, you can be guided to retrieve food ingredients from the refrigerator within 10 minutes and operate the cooking appliance within 20 minutes.

Lastly, operation information of the refrigerator and cooker can be provided to the terminal (S26).

As described above, since the refrigerator and cooking appliance can operate cooperatively, the user can conveniently cook. The number of home appliances involved in the cooperative operation is not limited to two, but may be three or more.

Take a robot vacuum cleaner as an example. As the node, the air quality sensor can recognize that there is a lot of oil vapor and recommend the operation of the robot vacuum cleaner. This is because oil vapor spreads on the floor and makes it slippery. At this time, the basic information may be the air quality of the air quality sensor. The above individual service can be recognized as the purpose of floor cleaning. The I/O module 20 can transmit a first control signal to the robot vacuum cleaner (S21a) and receive ec and status report messages of the robot vacuum cleaner (S22a). Here, the status report message may include various information such as location information, dust amount information, and charging information of the robot vacuum cleaner. In addition, the operation mode determination unit 32 can inquire with the memory 40 to receive various information such as whether the robot vacuum cleaner is suitable for operation. The operation mode determination unit can be determined by referring to the two pieces of information together. If there is a discrepancy between the two pieces of information, you can request confirmation or correction.

Information on the robot vacuum cleaner and indoor air may be recorded in advance as past information. For example, there may be information that the indoor air had a lot of oil vapor from the past hour or that oil vapor would be spread on the indoor floor due to lack of ventilation.

The operation determination module 30 can determine the operation of the robot vacuum cleaner. At this time, the path and time of the robot vacuum cleaner can be determined. The operation decision module 30 can transmit cleaning information to the robot vacuum cleaner or terminal (S23) and receive a response of approval, denial, or restart (S24).

In the case of the approval, the operation sequence determination unit 31, the operation mode determination unit 32, and the operation time determination unit 33 may transmit information to control at least two home appliances including at least a cooker ( S25a)(25b)(25c). In the case of the above denial, it may be terminated. In the case of the above restart, another cleaning method may be recommended.

In the case of the approval, the operation sequence determination unit 31 performs the first process first among the first process of operating the robot vacuum cleaner as the first home appliance and the second process of operating the air purifier as the second home appliance. We can guide you as much as possible. This is because a large amount of flying dust is generated while the robot vacuum cleaner is first operating, which may have a negative effect on the filter of the air purifier. The operation mode determination unit 32 allows the robot vacuum cleaner, which is the first home appliance, to strongly clean areas with a lot of oil vapor, and the air purifier, which is the second home appliance, can perform an operation in response to flying dust emitted from the robot vacuum cleaner. . The operation time determination unit 33 can convert the decision information of the operation sequence determination unit 31 into time and guide it to the robot vacuum cleaner and air purifier.

Lastly, operation information of the robot vacuum cleaner and air purifier can be provided to the terminal (S26).

As mentioned above, since robot vacuum cleaners and air purifiers can operate cooperatively, it is not only convenient for users but also avoids wasting unnecessary energy and resources.

Figure 10 is a flowchart of an information processing method for home appliances according to an embodiment. This embodiment may be an embodiment after the user's intent is understood. For example, it may be an information processing method after the user's cooking is identified by the information processing method of FIG. 5.

Referring to Figure 10, the I/O module 20 can determine the current state of the home appliance (1b1) by transmitting the first control signal to the home appliance (1b1) (S31). The home appliance can transmit an ACK message and a status report message (S32). The I/O module 20 can determine the current state of the home appliance (1b2) by transmitting the second control signal to the home appliance (1b2) (S33). The home appliance can transmit an ACK message and a status report message (S34). Each home appliance can transmit an ACK message and status report signal to the I/O module 20. Through the bus inside the information processing device, each module of the information processing device can share information. The status report message may be transmitted to the operation decision module.

The operation mode determination unit 32 may receive the status report message. The operation decision module may refer to the status report message to determine and notify the operation sequence, operation mode, and operation time of the home appliance. Depending on the type of the home appliance and/or the status report message, the home appliance may be determined differently. At this time, at least two home appliances can work together. The drawing illustrates the operation sequence and output of operation mode signals for the first and second home appliances (S35a) (S35b) (S36a) (S36b).

The above information processing method will be explained with an example.

The user's intention may be cooking using an exposed heating cooking device. The exposed heating cooking appliances may include induction, range, and highlight. The first home appliance may be a hood, and the second home appliance may be an air purifier. When the exposed heating cooker is operating, the hood may be operating and the air purifier may not be operating. This is because smoke generated from exposed heating cooking appliances has a negative effect on the air purifier. In other words, the lifespan of the filter that filters ultra-fine filters (PM2.5) in an air purifier may rapidly decrease.

Therefore, the hood operates ahead of the air purifier, and the hood can operate at the highest speed. The time of the hood and air purifier can be managed using indoor space information and air quality sensors.

The user's intention may be cooking using a sealed cooking device. The sealed cooking appliance may include an oven or the like. The first home appliance may be a hood, and the second home appliance may be an air purifier. The fact that the door is opened after the operation of the closed cooking appliance is completed can be used as basic information. If there is a signal of the basic information, the air purifier may not operate. With the basic information, the hood can operate quickly. This is because smoke generated from sealed cooking appliances has a negative effect on the air purifier. In other words, the lifespan of the filter that filters ultra-fine filters (PM2.5) in an air purifier may rapidly decrease.

Therefore, the hood operates ahead of the air purifier, and the hood can operate at the highest speed. The time of the hood and air purifier can be managed using indoor space information and air quality sensors.

The node providing the basic information may be an air quality sensor. The first home appliance may be an air purifier, and the second home appliance may be a hood. The air quality sensor can recognize that almost all post-cooking substances such as smoke have been emitted and odor particles remain. This signal can be known to the information processing device as basic information. If there is a signal of the basic information, the air purifier can operate. If the basic information is available, the hood can stop operation. This is because the air purifier has a rapid removal performance for odor particles.

Therefore, the air purifier operates ahead of the hood, and the air purifier can operate at the highest speed. The operating time of the hood and air purifier can be managed using indoor space information and air quality sensors.

There is an advantage in that the home appliances can be used in a more desirable environment without the user's recognition or instructions through cooperative operation of at least two home appliances. The cooperative operation shown in FIGS. 9 and 10 may be performed by combining two or more of the exemplary uses.

Figure 11 is a flowchart of an information processing method for home appliances according to an embodiment. This embodiment is a diagram explaining an information processing method for multiple home appliances.

Referring to Figure 11, an application program is installed on the user's home appliance, and the home appliance can be connected to the server through a network (S41). As an example, the application may be a ThinQ app. The application program can be installed by the server 2c.

In this state, the user can input food ingredient information in various ways. Food ingredients can be stored in the refrigerator. Information on food ingredients can be stored on the server (S42).

When the user starts cooking, a recommendation for a dish or recipe can be requested (S43). Here, the request can use various initial information such as terminal instructions, user's voice, and user's behavior. The meaning of initial information can be understood by utilizing each individual service in the individual service area. The recommendation here is that you can use ingredients that are in the refrigerator.

In the approval stage for the recommendation of a dish or recipe (S44), if approved, the recipe can be transmitted to the cooking device (S45a). In the approval stage for the recommendation of a dish or recipe (S44), if there is no approval, the user can cook the desired dish himself (S45b).

After the recipe is transmitted to the cooking device, the properties of the cooking device are determined (S46a). In the cooking appliance property determination step (S46a), it may be an exposed heating cooking appliance. In this case, when the cooking device detects the start of cooking (S47a), the hood can be strongly operated and the air purifier can be turned off (S48a). This is because the air purifier's filter is weak to food vapor. In the cooking device property determination step (S46a), it may be a sealed heating cooking device. In this case, when cooking starts (S46b) and the door of the sealed heating cooker is opened (S47b), the hood can be strongly operated and the air purifier can be turned off (S48a). This is because oil vapor that comes out in a confined space at once has a negative effect on the air purifier filter.

After cooking is completed (S49), the air quality can be measured (S51). The measured air quality is transmitted to the server, that is, the information processing device, and it can be determined that air purification is necessary. At this time, the operation of the kitchen hood can be stopped (S52) and the air purifier can be operated (S53). This operation can have the effect of reducing disruption to life due to noise from the hood and removing odor particles.

Afterwards, after a predetermined period of time has elapsed after cooking, cleaning of the oil vapor that has settled on the bottom can begin. The information processing device can recommend the operation of the robot vacuum cleaner to the user (S54). If the user approves it (S55), the robot vacuum cleaner can operate and the air purifier can stop (S56). This is because the filter of the air purifier may be damaged by flying dust from the robot vacuum cleaner. After a predetermined time has elapsed and the robot vacuum cleaner has finished cleaning (S57), the air purifier can be operated (S58).

According to the cooperative operation of the various home appliances mentioned above, users can become more convenient, prevent waste of energy, and keep the indoor environment cleaner.

According to the present invention, consumption of the remote control function of home appliances can be activated.

According to the present invention, consumers can use at least two home appliances more conveniently.

1: User area
1a: house
1b: Home appliances
1c: Edge device
2: Operator area
2a: first server
2b: second server
2c: server
2d: AI server
10: Individual service area
11: Application service area
12: Information processing device
20: I/O module
30: Operation decision module

Claims (20)

At least one household appliance;
A terminal capable of processing information of the at least one home appliance:
Individual service areas where individual services are provided; and
An information processing device connected to the terminal and the at least one home appliance through a network and providing an application service using the individual service,
The information processing device,
Receiving basic user information from a user using the at least one home appliance using at least one of the home appliance and the terminal;
Using the individual service, confirming the requested information requested by the user from the basic information;
Confirming first information of a first home appliance among the at least one home appliance; and
In order to provide the requested information, transmitting second information generated with reference to the first information to at least the first home appliance,
Information processing system for home appliances. According to claim 1,
If the user approves the second information, the second information is transmitted to a second home appliance among the at least one home appliance.
Information processing system for home appliances. According to claim 2,
The information processing system for home appliances, wherein the first home appliance is a refrigerator and the second home appliance is a cooking appliance. According to claim 2,
An information processing system for home appliances wherein the first home appliance is a robot vacuum cleaner and the second home appliance is an air purifier. According to claim 1,
In the second information,
An information processing system for home appliances that includes at least one of operation information of the second home appliance and operation sequence information of the first and second home appliances. According to claim 1,
The first information is stored in at least one of the memory of the information processing device and the first home appliance,
Information processing system for home appliances. I/O module that communicates with external home appliances in response to node requests; and
When there is a trigger signal containing requested information related to the user's cooking,
Confirming first information of a first home appliance among the at least one home appliance,
Sending second information generated with reference to the first information to the first home appliance to provide the requested information, and
When execution of the second information is determined, transmitting the second information to a second home appliance,
The second information includes operation sequence information of the first home appliance and the second home appliance.
An information processing device including an action decision module. According to claim 7,
The information processing device stores the first information in at least one of the first home appliance and a memory of the information processing device. According to claim 7,
An information processing device that transmits the second information to the user's terminal. According to claim 7,
The pair of the first home appliance and the second home appliance,
An information processing device including at least one of a refrigerator and an exposed cooking appliance, a refrigerator and an enclosed cooking appliance, a hood and an air purifier, and a robot vacuum cleaner and an air purifier. Receiving a trigger signal from a node;
the information processing device detects cooking request information from the trigger signal using an individual service in an individual service area;
The information processing device determines the status of food ingredients in the refrigerator and transmits first information, which is cooking information corresponding to the cooking request information, to the node; and
An information processing method for a home appliance, including transmitting first information to the cooking appliance upon approval of the node. According to claim 11,
An information processing method for a home appliance in which the cooking appliance performs cooking according to first information. According to claim 12,
When the cooking appliance is an exposed heating cooking appliance, an information processing method for a home appliance that controls the air purifier to stop operation and the hood to operate when cooking starts. According to claim 12,
An information processing method for a home appliance that controls the air purifier to stop operation and the hood to operate when the door is opened after cooking when the cooking appliance is a sealed heating cooking appliance. Receiving a trigger signal from a node;
the information processing device detects requested information from the trigger signal using an individual service in an individual service area;
In response to the request information, the information processing device recommends corresponding information including status information of the first home appliance and transmits it to the node; and
An information processing method for a home appliance, including transmitting the corresponding information to a second home appliance if there is approval from the node. According to claim 15,
An information processing method for a home appliance, wherein the first home appliance is a refrigerator and the second home appliance is an exposed cooking appliance. According to claim 15,
An information processing method for a home appliance, wherein the first home appliance is a refrigerator and the second home appliance is a sealed cooking device. According to claim 15,
An information processing method for a home appliance, wherein the first home appliance is a hood and the second home appliance is an air purifier. According to claim 15,
An information processing method for a home appliance, wherein the first home appliance is a robot vacuum cleaner, and the second home appliance is an air purifier. According to claim 15,
The information processing method for a home appliance, wherein the corresponding information includes operation sequence information of the first home appliance and the second home appliance.

Images