WO2020017686A1

WO2020017686A1 - Artificial intelligence server and artificial intelligence device

Info

Publication number: WO2020017686A1
Application number: PCT/KR2018/008976
Authority: WO
Inventors: 한종우
Original assignee: 엘지전자 주식회사
Priority date: 2018-07-18
Filing date: 2018-08-07
Publication date: 2020-01-23
Also published as: KR20200013149A; KR102172772B1

Abstract

An artificial intelligence server is disclosed. An artificial intelligence server, according to an embodiment of the present invention, comprises: a communication unit for communicating with an external device; and an artificial intelligence unit for, on the basis of characteristic information of a first artificial intelligence device and characteristic information of a second artificial intelligence device, acquiring a recognition model of the second artificial intelligence device that reflects personalized information of the first artificial intelligence device so as to match the characteristic information of the second artificial intelligence device.

Description

AI Servers and AI Devices

The present invention relates to an artificial intelligence server and an artificial intelligence device for obtaining a recognition model reflecting personalized information in a first artificial intelligence device to correspond to characteristic information of a second artificial intelligence device.

Artificial intelligence is a branch of computer science and information technology that studies how to enable computers to do things like thinking, learning, and self-development that human intelligence can do. It means to be able to imitate.

In addition, artificial intelligence does not exist by itself, but is directly or indirectly related to other fields of computer science. Particularly in modern times, attempts are being actively made to introduce artificial intelligence elements in various fields of information technology and use them to solve problems in those fields.

Meanwhile, technologies for recognizing and learning the surrounding situation using artificial intelligence and providing information desired by a user in a desired form or performing a desired operation or function have been actively studied.

An electronic device providing such various operations and functions may be referred to as an artificial intelligence device.

Artificial intelligence devices, on the other hand, come equipped with trained recognition models. However, the learning environment before the product may be different from the environment in which the artificial intelligence device is actually used.

Therefore, the artificial intelligence device modifies the recognition model by recognizing and learning a new environment after sale, and this process may be called personalization.

For example, when the artificial intelligence device is a voice recognition air conditioner and the recognition model of the artificial intelligence device is a speech recognition model, the artificial intelligence device may perform speaker adaptation by learning a speech habit of a speaker and modifying the speech recognition model. .

For another example, if the artificial intelligence device is a robot cleaner and the recognition model of the artificial intelligence device is a terrain recognition model, the artificial intelligence device may perform adaptation to the space by modifying the terrain recognition model according to the space and obstacles in the home. Can be.

On the other hand, personalization requires a huge amount of data for learning, so personalization is a time-consuming process.

However, when a new artificial intelligence device is added while a personalized artificial intelligence device already exists in the same use environment, if the personalization information of the personalized artificial intelligence device is available to the new artificial intelligence device, Personalization can be shortened in time.

However, since the characteristics of the products are different for each product, there is a problem that it is difficult to apply personalized information of the personalized AI device to a new AI device.

The present invention is to solve the above-described problems, an object of the present invention, an artificial intelligence server and artificial intelligence for obtaining a recognition model reflecting the personalized information in the first artificial intelligence device corresponding to the characteristic information of the second artificial intelligence device To provide an intelligent device.

According to an embodiment of the present disclosure, an artificial intelligence server may include a communication unit configured to communicate with an external device, and based on characteristic information of a first artificial intelligence device and characteristic information of the second artificial intelligence device. And an artificial intelligence unit for obtaining a recognition model of the second artificial intelligence device reflecting personalization information to correspond to the characteristic information of the second artificial intelligence device.

In this case, the personalization information of the first artificial intelligence device is information obtained by learning an input signal from a learning model of the first artificial intelligence device and personalized the artificial intelligence unit, and the artificial intelligence unit includes characteristic information and the first information of the first artificial intelligence device. Based on the difference in the characteristic information of the artificial intelligence device, a recognition model of the second artificial intelligence device reflecting personalization information of the first artificial intelligence device to correspond to the characteristic information of the second artificial intelligence device may be obtained. .

In this case, the recognition model of the second artificial intelligence device may include a learning model that outputs an output value obtained by converting a characteristic of the input signal when the input signal is input, and divides the output value into a command and a non-command using a division boundary. It may include a separator.

In this case, the artificial intelligence unit may input the input device of the first artificial intelligence device and the second artificial intelligence device based on a difference between the characteristic information of the first artificial intelligence device and the characteristic information of the second artificial intelligence device. Obtain first modification information for applying a difference between devices to the input signal, apply the first modification information to an input signal output from an input device of the second artificial intelligence device, and apply the first modification information The recognition model for inputting an input signal to which the information is applied may be acquired.

On the other hand, the artificial intelligence unit, based on the difference between the characteristic information of the first artificial intelligence device and the characteristic information of the second artificial intelligence device, the input device of the first artificial intelligence device and the input device of the second artificial intelligence device The second modification information for applying the difference to the learning model may be obtained, and the recognition model including the modified learning model may be obtained by applying the second modification information.

On the other hand, the artificial intelligence unit, based on the difference between the characteristic information of the first artificial intelligence device and the characteristic information of the second artificial intelligence device, the input device of the first artificial intelligence device and the input device of the second artificial intelligence device And obtaining third modification information for applying the difference to the output value of the learning model, and obtaining the recognition model for applying the third modification information to the output value of the learning model.

On the other hand, the artificial intelligence unit, based on the difference between the characteristic information of the first artificial intelligence device and the characteristic information of the second artificial intelligence device, the input device of the first artificial intelligence device and the input device of the second artificial intelligence device And obtaining fourth modification information for applying the difference between the separators, and obtaining the recognition model in which the division boundary reflecting the personalization information is corrected using the fourth modification information.

Meanwhile, when the recognition model of the second artificial intelligence device is acquired, the artificial intelligence unit may update the acquired recognition model to the second artificial intelligence device.

On the other hand, the artificial intelligence device according to the embodiment of the present invention, a communication unit for communicating with an external device, an input unit for receiving an input signal, and receives the characteristic information and personalization information of another artificial intelligence device, the characteristics of the other artificial intelligence device And an artificial intelligence unit configured to obtain a recognition model of the artificial intelligence device reflecting personalization information of the other artificial intelligence device corresponding to the characteristic information of the artificial intelligence device based on the information and the characteristic information of the artificial intelligence device.

In this case, the personalization information of the other artificial intelligence device is information obtained by personalizing the input signal by learning from the learning model of the other artificial intelligence device, and the artificial intelligence unit includes characteristic information of the other artificial intelligence device and the artificial intelligence device. The recognition model of the artificial intelligence device may be acquired by reflecting personalization information of the other artificial intelligence device to correspond to the characteristic information of the artificial intelligence device, based on the difference of the characteristic information of.

In this case, the recognition model of the artificial intelligence device may include a learning model that outputs an output value obtained by converting a characteristic of the input signal when the input signal is input, and a classification that divides the output value into a command and a non-command using a division boundary. It may include a group.

The artificial intelligence unit may be configured to determine a difference between an input device of the other artificial intelligence device and an input device of the artificial intelligence device based on a difference between the characteristic information of the other artificial intelligence device and the characteristic information of the artificial intelligence device. Acquiring first correction information for applying to the controller, applying the first correction information to an input signal output from the input unit, and inputting the input signal to which the first correction information is applied to the learning model. A model can be obtained.

The artificial intelligence unit may be configured to determine a difference between an input device of the other artificial intelligence device and an input device of the artificial intelligence device based on a difference between the characteristic information of the other artificial intelligence device and the characteristic information of the artificial intelligence device. The second model may acquire second modification information for applying to the second modification information, and obtain the recognition model including the modified learning model by applying the second modification information.

The artificial intelligence unit may be configured to determine a difference between an input device of the other artificial intelligence device and an input device of the artificial intelligence device based on a difference between the characteristic information of the other artificial intelligence device and the characteristic information of the artificial intelligence device. The third modification information may be acquired to apply the output value of the learning model, and the recognition model may be obtained that applies the third modification information to the output value of the learning model.

On the other hand, the artificial intelligence unit, based on the difference between the characteristic information of the other artificial intelligence device and the characteristic information of the artificial intelligence device, the difference between the input device of the other artificial intelligence device and the input device of the artificial intelligence device to the separator; Obtaining fourth modification information for application, and obtaining the recognition model in which the division boundary reflecting the personalization information is corrected using the fourth modification information.

Meanwhile, a method of operating an artificial intelligence server according to an exemplary embodiment of the present disclosure may include obtaining characteristic information of a first artificial intelligence device and characteristic information of a second artificial intelligence device, and characteristic information of the first artificial intelligence device and the first information. Obtaining a recognition model of the second artificial intelligence device reflecting personalization information of the first artificial intelligence device to correspond to the characteristic information of the second artificial intelligence device based on the characteristic information of the artificial intelligence device;

In this case, the personalization information of the first artificial intelligence device is information obtained by personalizing the input signal by learning from a learning model of the first artificial intelligence device, and obtaining the recognition model of the second artificial intelligence device comprises: The second artificial intelligence reflecting personalization information of the first artificial intelligence device to correspond to the characteristic information of the second artificial intelligence device based on a difference between the characteristic information of the first artificial intelligence device and the characteristic information of the second artificial intelligence device; A recognition model of the intelligent device can be obtained.

In this case, the acquiring of the recognition model of the second artificial intelligence device may include inputting the first artificial intelligence device based on a difference between the characteristic information of the first artificial intelligence device and the characteristic information of the second artificial intelligence device. Acquiring first modification information for applying a difference between an apparatus and an input device of the second artificial intelligence device to the input signal, and applying the first correction information to the input signal output from the input device of the second artificial intelligence device; And applying the first modification information and obtaining the recognition model for inputting the input signal to which the first modification information is applied to the learning model.

The acquiring of the recognition model of the second AI device may include: inputting the first AI device based on a difference between the property information of the first AI device and the property information of the second AI device. Acquiring second modification information for applying a difference between the input device of the second artificial intelligence device and the learning model; and the recognition including a learning model modified by applying the second modification information. Obtaining a model.

The acquiring of the recognition model of the second AI device may include: inputting the first AI device based on a difference between the property information of the first AI device and the property information of the second AI device. Acquiring third correction information for applying a difference between an input device of the second artificial intelligence device and an output value of the learning model, and applying the third modification information to an output value of the learning model. It may include obtaining a recognition model.

The acquiring of the recognition model of the second AI device may include: inputting the first AI device based on a difference between the property information of the first AI device and the property information of the second AI device. Acquiring fourth modification information for applying a difference between an input device of the second artificial intelligence device and the separator, and correcting the division boundary reflecting the personalization information by using the fourth modification information. It may include the step of obtaining the recognition model.

According to the present invention, despite the difference in characteristics between the first AI device and the second AI device, since the learning result of the first AI device can be adopted in the second AI device, There is an advantage that can significantly reduce the time required for personalization.

1 is a block diagram illustrating an artificial intelligence device according to the present invention.

2 is a diagram illustrating a plurality of artificial intelligence devices and an artificial intelligence server according to an exemplary embodiment of the present invention.

3 is a block diagram illustrating a configuration of the artificial intelligence server 600 when the recognition model is mounted on the artificial intelligence server 600 according to an exemplary embodiment of the present invention.

4 is a diagram illustrating a recognition model 700 mounted in the artificial intelligence server 600 according to an embodiment of the present invention.

5 is a diagram illustrating a personalization process of the recognition model 700 according to an exemplary embodiment of the present invention.

FIG. 6 is a diagram illustrating a problem that may occur when personalization information of a first artificial intelligence device is applied to a second artificial intelligence device.

FIG. 7 illustrates a method of canceling a difference in characteristics between the first AI device 100 and the second AI device 200 by applying correction information to a voice signal according to a first embodiment of the present invention. It is for the drawing.

FIG. 8 is a diagram illustrating a process of converting a voice signal output from the input device of the second artificial intelligence device to be similar to the voice signal output from the first artificial intelligence device according to the first embodiment of the present invention.

FIG. 9 illustrates the first AI device 100 and the second AI device 200 by applying correction information to a learning model or applying correction information to an output value of the learning model according to the second embodiment of the present disclosure. It is a figure for demonstrating the method of canceling the characteristic difference of ().

FIG. 10 is a diagram illustrating a process of converting an output value output from a learning model of a second artificial intelligence device to be similar to an output value output from a learning model of a first artificial intelligence device according to a second embodiment of the present disclosure. .

FIG. 11 illustrates that the first AI device 100 and the second AI device 200 are applied by applying correction information to a separator that reflects personalization information of the first AI device, according to the third embodiment of the present disclosure. A diagram for explaining a method of canceling the difference in characteristics.

12 is a diagram illustrating a process in which personalization information of a first artificial intelligence device is reflected to correspond to characteristic information of a second artificial intelligence device by setting a new division boundary according to a third embodiment of the present invention.

13 is a view for explaining a method of operating an artificial intelligence server according to an embodiment of the present invention.

FIG. 14 is a diagram for describing a method of operating a second AI device when the first AI device and the second AI device directly communicate with each other to update a recognition model.

FIG. 15 is a diagram for describing a method of operating an artificial intelligence system including a first artificial intelligence device, a second artificial intelligence device, and a server, according to an exemplary embodiment.

Hereinafter, embodiments of the present disclosure will be described in detail with reference to the accompanying drawings, and the same or similar components will be given the same reference numerals regardless of the reference numerals, and redundant description thereof will be omitted. The suffixes "module" and "unit" for components used in the following description are given or mixed in consideration of ease of specification, and do not have distinct meanings or roles. In addition, in the following description of the embodiments disclosed herein, when it is determined that the detailed description of the related known technology may obscure the gist of the embodiments disclosed herein, the detailed description thereof will be omitted. In addition, the accompanying drawings are only for easily understanding the embodiments disclosed in the present specification, the technical idea disclosed in the specification by the accompanying drawings are not limited, and all changes included in the spirit and scope of the present invention. It should be understood to include equivalents and substitutes.

Terms including ordinal numbers such as first and second may be used to describe various components, but the components are not limited by the terms. The terms are used only for the purpose of distinguishing one component from another.

When a component is said to be "connected" or "connected" to another component, it may be directly connected to or connected to that other component, but it may be understood that another component may be present in the middle. Should be. On the other hand, when a component is said to be "directly connected" or "directly connected" to another component, it should be understood that there is no other component in between.

Singular expressions include plural expressions unless the context clearly indicates otherwise. In this application, the terms "comprises" or "having" are intended to indicate that there is a feature, number, step, operation, component, part, or combination thereof described in the specification, and one or more other features. It is to be understood that the present invention does not exclude the possibility of the presence or the addition of numbers, steps, operations, components, components, or a combination thereof.

The artificial intelligence device 100 may include a wireless communication unit 110, an input unit 120, an artificial intelligence unit 130, a detection unit 140, an output unit 150, an interface unit 160, a memory 170, and a control unit ( 180 and the power supply unit 190 may be included.

The components shown in FIG. 1 are not essential to implementing an AI device, so an AI device described herein may have more or fewer components than the components listed above.

More specifically, the wireless communication unit 110 of the components, between the artificial intelligence device 100 and the wireless communication system, between the artificial intelligence device 100 and another artificial intelligence device 100, or the artificial intelligence device 100 ) And one or more modules that enable wireless communication between the external server and the external server. In addition, the wireless communication unit 110 may include one or more modules for connecting the artificial intelligence device 100 to one or more networks.

The wireless communication unit 110 may include at least one of the broadcast receiving module 111, the mobile communication module 112, the wireless internet module 113, the short range communication module 114, and the location information module 115. .

The input unit 120 may include a camera 121 or an image input unit for inputting an image signal, a microphone 122 for inputting an audio signal, an audio input unit, or a user input unit 123 for receiving information from a user. , Touch keys, mechanical keys, and the like. The voice data or the image data collected by the input unit 120 may be analyzed and processed as a control command of the user.

The artificial intelligence unit 130 performs a role of processing information based on artificial intelligence technology, and includes one or more modules that perform at least one of learning information, inferring information, perceiving information, and processing natural language. It may include.

The artificial intelligence unit 130 uses a machine learning technology to generate a large amount of information (big data, big data, etc.) stored in the artificial intelligence device, environment information around the artificial intelligence device, and information stored in an external storage that can be communicated with. at least one of data, learning, inference, and processing. The artificial intelligence unit 130 predicts (or infers) an operation of at least one AI device executable by using the information learned using the machine learning technique, and among the at least one predicted operations. The AI device can be controlled to perform the most feasible operation.

Machine learning technology is a technology that collects and learns a large amount of information based on at least one algorithm, and determines and predicts information based on the learned information. Learning information is an operation of grasping characteristics, rules, and judgment criteria of information, quantifying the relationship between information, and predicting new data using the quantized pattern.

The algorithms used by these machine learning techniques can be algorithms based on statistics, for example, decision trees using tree structures as predictive models, artificial trees that mimic the neural network structure and function of organisms. Neural networks, genetic programming based on living evolutionary algorithms, clustering that distributes observed examples into subsets of clusters, and Monte Carlo, which randomly computes function values through randomized random numbers Monte carlo method.

As a field of machine learning technology, deep learning technology is a technology that performs at least one of learning, determining, and processing information by using an artificial neural network algorithm. The artificial neural network may have a structure that connects layers to layers and transfers data between layers. Such deep learning technology can learn a huge amount of information through an artificial neural network using a graphic processing unit (GPU) optimized for parallel computing.

On the other hand, the artificial intelligence unit 130 collects (detects, monitors, monitors, monitors, etc.) signals, data, information, etc. input or output from the components of the AI device in order to collect a large amount of information for applying the machine learning technology. Extraction, detection, reception). In addition, the artificial intelligence unit 130 may collect (detect, monitor, extract, detect, receive) data, information, and the like stored in an external storage (eg, a cloud server) connected through communication. More specifically, the collection of information may be understood as a term including an operation of sensing information through a sensor, extracting information stored in the memory 170, or receiving information from an external storage through communication.

The artificial intelligence unit 130 may detect information within the artificial intelligence device, surrounding environment information surrounding the artificial intelligence device, and user information through the sensing unit 140. In addition, the artificial intelligence unit 130 may receive a broadcast signal and / or broadcast related information, a wireless signal, wireless data, and the like through the wireless communication unit 110. In addition, the artificial intelligence unit 130 may receive image information (or signal), audio information (or signal), data, or information input from a user from the input unit.

The artificial intelligence unit 130 collects a large amount of information in real time on the background, learns it, and stores the processed information (for example, knowledge graph, command policy, personalization database, conversation engine, etc.) in an appropriate form. Can be stored at 170.

And, based on the information learned using the machine learning technology, the artificial intelligence unit 130, if the operation of the artificial intelligence device is predicted, to execute the predicted operation, to control the components of the artificial intelligence device, The control command for executing the predicted operation may be transmitted to the controller 180. The controller 180 may execute the predicted operation by controlling the artificial intelligence device based on the control command.

Meanwhile, when a specific operation is performed, the artificial intelligence unit 130 analyzes historical information indicating performance of a specific operation through machine learning technology, and updates the previously learned information based on the analysis information. Can be. Thus, the artificial intelligence unit 130 may improve the accuracy of the information prediction.

Meanwhile, in the present specification, the artificial intelligence unit 130 and the controller 180 may be understood as the same component. In this case, a function performed by the controller 180 described herein may be expressed as being performed by the artificial intelligence unit 130, and the controller 180 may be named as the artificial intelligence unit 130 or vice versa. The intelligent unit 130 may be referred to as the controller 180.

Alternatively, in the present specification, the artificial intelligence unit 130 and the controller 180 may be understood as separate components. In this case, the artificial intelligence unit 130 and the controller 180 may perform various controls on the artificial intelligence device through data exchange with each other. The controller 180 may perform at least one function on the artificial intelligence device or control at least one of the components of the artificial intelligence device based on the result derived from the artificial intelligence unit 130. Furthermore, the artificial intelligence unit 130 may also be operated under the control of the controller 180.

The sensing unit 140 may include one or more sensors for sensing at least one of information in the artificial intelligence device, surrounding environment information surrounding the artificial intelligence device, and user information.

For example, the sensing unit 140 may include a proximity sensor 141, an illumination sensor 142, an illumination sensor, a touch sensor, an acceleration sensor, a magnetic sensor, and gravity. G-sensor, Gyroscope Sensor, Motion Sensor, RGB Sensor, Infrared Sensor, Infrared Sensor, Finger Scan Sensor, Ultrasonic Sensor Optical sensors (e.g. cameras 121), microphones (see 122), battery gauges, environmental sensors (e.g. barometers, hygrometers, thermometers, radiation detection sensors, Thermal sensors, gas sensors, etc.), chemical sensors (eg, electronic noses, healthcare sensors, biometric sensors, etc.). On the other hand, the artificial intelligence device disclosed herein may utilize a combination of information sensed by at least two or more of these sensors.

The output unit 150 is used to generate an output related to sight, hearing, or tactile sense, and includes at least one of a display unit 151, an audio output unit 152, a hap tip module 153, and an optical output unit 154. can do. The display unit 151 forms a layer structure with or is integrally formed with the touch sensor, thereby implementing a touch screen. The touch screen may function as a user input unit 123 that provides an input interface between the artificial intelligence device 100 and the user, and may also provide an output interface between the artificial intelligence device 100 and the user.

The interface unit 160 serves as a path to various types of external devices connected to the artificial intelligence device 100. The interface unit 160 connects a device equipped with a wired / wireless headset port, an external charger port, a wired / wireless data port, a memory card port, and an identification module. It may include at least one of a port, an audio input / output (I / O) port, a video input / output (I / O) port, and an earphone port. In the artificial intelligence device 100, in response to an external device being connected to the interface unit 160, appropriate control associated with the connected external device may be performed.

In addition, the memory 170 stores data supporting various functions of the artificial intelligence device 100. Memory 170 is a plurality of application programs (application program or application) running in the artificial intelligence device 100, the data for the operation of the artificial intelligence device 100, instructions, the artificial intelligence unit 130 Data for operation of (eg, at least one algorithm information for machine learning, etc.). At least some of these applications may be downloaded from an external server through wireless communication. In addition, at least some of these applications may exist on the AI device 100 from the time of shipment for the basic functions of the AI device 100 (for example, call forwarding, call forwarding, message reception, and call forwarding). have. Meanwhile, the application program may be stored in the memory 170 and installed on the artificial intelligence device 100 to be driven by the controller 180 to perform an operation (or function) of the artificial intelligence device.

In addition to the operation related to the application program, the controller 180 typically controls the overall operation of the artificial intelligence device 100. The controller 180 may provide or process information or a function appropriate to a user by processing signals, data, information, and the like, which are input or output through the above-described components, or driving an application program stored in the memory 170.

In addition, the controller 180 may control at least some of the components described with reference to FIG. 1A in order to drive an application program stored in the memory 170. In addition, the controller 180 may operate at least two or more of the components included in the artificial intelligence device 100 in combination with each other to drive the application program.

The power supply unit 190 receives power from an external power source or an internal power source under the control of the controller 180 to supply power to each component included in the artificial intelligence device 100. The power supply unit 190 includes a battery, which may be a built-in battery or a replaceable battery.

Hereinafter, the components listed above will be described in detail with reference to FIG. 1A before looking at various embodiments implemented through the artificial intelligence device 100 described above.

First, referring to the wireless communication unit 110, the broadcast receiving module 111 of the wireless communication unit 110 receives a broadcast signal and / or broadcast related information from an external broadcast management server through a broadcast channel. The broadcast channel may include a satellite channel and a terrestrial channel. Two or more broadcast receiving modules may be provided to the mobile terminal 100 for simultaneous broadcast reception or broadcast channel switching for at least two broadcast channels.

The broadcast management server may mean a server that generates and transmits a broadcast signal and / or broadcast related information or a server that receives a previously generated broadcast signal and / or broadcast related information and transmits the same to a terminal. The broadcast signal may include not only a TV broadcast signal, a radio broadcast signal, and a data broadcast signal, but also a broadcast signal having a data broadcast signal combined with a TV broadcast signal or a radio broadcast signal.

The broadcast signal may be encoded according to at least one of technical standards for transmitting / receiving a digital broadcast signal (or a broadcast method, for example, ISO, IEC, DVB, ATSC, etc.), and the broadcast receiving module 111 may The digital broadcast signal may be received using a method suitable for the technical standard set by the technical standards.

The broadcast associated information may mean information related to a broadcast channel, a broadcast program, or a broadcast service provider. The broadcast related information may also be provided through a mobile communication network. In this case, it may be received by the mobile communication module 112.

The broadcast related information may exist in various forms such as an electronic program guide (EPG) of digital multimedia broadcasting (DMB) or an electronic service guide (ESG) of digital video broadcast-handheld (DVB-H). The broadcast signal and / or broadcast related information received through the broadcast receiving module 111 may be stored in the memory 160.

The mobile communication module 112 may include technical standards or communication schemes (eg, Global System for Mobile communication (GSM), Code Division Multi Access (CDMA), Code Division Multi Access 2000 (CDMA2000), EV, etc.) for mobile communication. Enhanced Voice-Data Optimized or Enhanced Voice-Data Only (DO), Wideband CDMA (WCDMA), High Speed Downlink Packet Access (HSDPA), High Speed Uplink Packet Access (HSUPA), Long Term Evolution (LTE), LTE-A (Long Term Evolution-Advanced) and the like to transmit and receive a radio signal with at least one of a base station, an external terminal, a server on a mobile communication network.

The wireless signal may include various types of data according to transmission and reception of a voice call signal, a video call signal, or a text / multimedia message.

The wireless internet module 113 refers to a module for wireless internet access and may be embedded or external to the artificial intelligence device 100. The wireless internet module 113 is configured to transmit and receive wireless signals in a communication network according to wireless internet technologies.

Wireless Internet technologies include, for example, Wireless LAN (WLAN), Wireless-Fidelity (Wi-Fi), Wireless Fidelity (Wi-Fi) Direct, Digital Living Network Alliance (DLNA), Wireless Broadband (WiBro), and WiMAX (World Interoperability for Microwave Access (HSDPA), High Speed Downlink Packet Access (HSDPA), High Speed Uplink Packet Access (HSUPA), Long Term Evolution (LTE), Long Term Evolution-Advanced (LTE-A), and the like. 113) transmits and receives data according to at least one wireless Internet technology in a range including the Internet technologies not listed above.

In view of the fact that the wireless Internet access by WiBro, HSDPA, HSUPA, GSM, CDMA, WCDMA, LTE, LTE-A, etc. is made through a mobile communication network, the wireless Internet module 113 for performing a wireless Internet access through the mobile communication network 113 ) May be understood as a kind of mobile communication module 112.

The short range communication module 114 is for short range communication, and includes Bluetooth ™, Radio Frequency Identification (RFID), Infrared Data Association (IrDA), Ultra Wideband (UWB), ZigBee, and NFC. (Near Field Communication), Wi-Fi (Wireless-Fidelity), Wi-Fi Direct, Wireless USB (Wireless Universal Serial Bus) by using at least one of the technologies, it is possible to support near field communication. The short-range communication module 114 may be configured between an artificial intelligence device 100 and a wireless communication system, between an artificial intelligence device 100 and another artificial intelligence device 100, or artificially via a wireless area network. Wireless communication between the intelligent device 100 and the network in which the other artificial intelligence device 100 or the external server is located may be supported. The short range wireless communication network may be short range wireless personal area networks.

Here, the other AI device 100 is a wearable device capable of exchanging (or interworking) data with the AI device 100 according to the present invention, for example, a smartwatch, It may be a smart glass, a head mounted display (HMD). The short range communication module 114 may detect (or recognize) a wearable device that can communicate with the artificial intelligence device 100 around the artificial intelligence device 100. Further, when the detected wearable device is a device that is authenticated to communicate with the artificial intelligence device 100 according to the present invention, the controller 180 may communicate at least a portion of data processed by the artificial intelligence device 100 with the local area communication. The module 114 may transmit to the wearable device. Therefore, the user of the wearable device may use data processed by the artificial intelligence device 100 through the wearable device. For example, according to this, when a call is received by the AI device 100, the user performs a phone call through the wearable device, or when a message is received by the AI device 100, the wearable device transmits the call. It is possible to confirm the received message.

The location information module 115 is a module for obtaining a location (or current location) of an artificial intelligence device, and a representative example thereof is a Global Positioning System (GPS) module or a Wireless Fidelity (WiFi) module. For example, if the artificial intelligence device utilizes a GPS module, the artificial intelligence device may acquire the location of the artificial intelligence device using a signal transmitted from a GPS satellite.

As another example, when the artificial intelligence device utilizes the Wi-Fi module, the artificial intelligence device may acquire the location of the artificial intelligence device based on the information of the wireless access point (AP) transmitting or receiving the wireless signal with the Wi-Fi module. have. If necessary, the location information module 115 may perform any function of other modules of the wireless communication unit 110 to substitute or additionally obtain data regarding the location of the artificial intelligence device. The location information module 115 is a module used to obtain a location (or a current location) of the artificial intelligence device, and is not limited to a module that directly calculates or obtains the location of the artificial intelligence device.

Next, the input unit 120 is for inputting image information (or signal), audio information (or signal), data, or information input from a user, and for inputting image information, the artificial intelligence device 100 One or a plurality of cameras 121 may be provided. The camera 121 processes image frames such as still images or moving images obtained by the image sensor in the video call mode or the photographing mode. The processed image frame may be displayed on the display unit 151 or stored in the memory 170. On the other hand, the plurality of cameras 121 provided in the artificial intelligence device 100 may be arranged to form a matrix structure, and through the camera 121 forming a matrix structure as described above, the artificial intelligence device 100 may have various angles or A plurality of image information having a focus may be input. In addition, the plurality of cameras 121 may be arranged in a stereo structure to acquire a left image and a right image for implementing a stereoscopic image.

The microphone 122 processes external sound signals into electrical voice data. The processed voice data may be variously utilized according to a function (or an application program being executed) performed by the artificial intelligence device 100. Meanwhile, various noise reduction algorithms may be implemented in the microphone 122 to remove noise generated in the process of receiving an external sound signal.

The user input unit 123 is for receiving information from a user. When information is input through the user input unit 123, the controller 180 may control an operation of the artificial intelligence device 100 to correspond to the input information. have. The user input unit 123 may be a mechanical input unit (or a mechanical key, for example, buttons, dome switches, and jog wheels located at the front and rear or side surfaces of the artificial intelligence device 100). , Jog switch, etc.) and touch input means. As an example, the touch input means may include a virtual key, a soft key, or a visual key displayed on the touch screen through a software process, or a portion other than the touch screen. The virtual key or the visual key may be displayed on the touch screen while having various forms, for example, a graphic or text. ), An icon, a video, or a combination thereof.

Meanwhile, the sensing unit 140 senses at least one of information in the artificial intelligence device, surrounding environment information surrounding the artificial intelligence device, and user information, and generates a sensing signal corresponding thereto. The controller 180 may control driving or operation of the artificial intelligence device 100 or perform data processing, function or operation related to an application program installed in the artificial intelligence device 100 based on the sensing signal. Representative sensors among various sensors that may be included in the sensing unit 140 will be described in more detail.

First, the proximity sensor 141 refers to a sensor that detects the presence or absence of an object approaching a predetermined detection surface or an object present in the vicinity without using a mechanical contact by using an electromagnetic force or infrared rays. The proximity sensor 141 may be disposed in the inner region of the artificial intelligence device covered by the touch screen as described above or near the touch screen.

Examples of the proximity sensor 141 include a transmission photoelectric sensor, a direct reflection photoelectric sensor, a mirror reflection photoelectric sensor, a high frequency oscillation proximity sensor, a capacitive proximity sensor, a magnetic proximity sensor, and an infrared proximity sensor. In the case where the touch screen is capacitive, the proximity sensor 141 may be configured to detect the proximity of the object with the change of the electric field according to the proximity of the conductive object. In this case, the touch screen (or touch sensor) itself may be classified as a proximity sensor.

On the other hand, for convenience of description, the action of allowing the object to be recognized without being in contact with the touch screen so that the object is located on the touch screen is referred to as "proximity touch", and the touch The act of actually touching an object on the screen is called a "contact touch." The position where an object is in close proximity touch on the touch screen means a position where the object is perpendicular to the touch screen when the object is in close proximity touch. The proximity sensor 141 may detect a proximity touch and a proximity touch pattern (for example, a proximity touch distance, a proximity touch direction, a proximity touch speed, a proximity touch time, a proximity touch position, and a proximity touch movement state). have.

Meanwhile, the controller 180 processes data (or information) corresponding to the proximity touch operation and the proximity touch pattern detected through the proximity sensor 141 as described above, and further, provides visual information corresponding to the processed data. It can be output on the touch screen. Furthermore, the controller 180 may control the AI device 100 to process different operations or data (or information) according to whether the touch on the same point on the touch screen is a proximity touch or a touch touch. have.

The touch sensor applies a touch (or touch input) applied to the touch screen (or display unit 151) using at least one of various touch methods such as a resistive film method, a capacitive method, an infrared method, an ultrasonic method, and a magnetic field method. Detect.

As an example, the touch sensor may be configured to convert a change in pressure applied to a specific portion of the touch screen or capacitance generated at the specific portion into an electrical input signal. The touch sensor may be configured to detect a position, an area, a pressure at the touch, a capacitance at the touch, and the like, when the touch object applying the touch on the touch screen is touched on the touch sensor. Here, the touch object is an object applying a touch to the touch sensor and may be, for example, a finger, a touch pen or a stylus pen, a pointer, or the like.

As such, when there is a touch input to the touch sensor, the corresponding signal (s) is sent to the touch controller. The touch controller processes the signal (s) and then transmits the corresponding data to the controller 180. As a result, the controller 180 can determine which area of the display unit 151 is touched. Here, the touch controller may be a separate component from the controller 180 or may be the controller 180 itself.

The controller 180 may perform different control or perform the same control according to the type of the touch object, which touches the touch screen (or a touch key provided in addition to the touch screen). Whether to perform different control or the same control according to the type of the touch object may be determined according to the operation state of the artificial intelligence device 100 or an application program being executed.

Meanwhile, the touch sensor and the proximity sensor described above may be independently or combined, and may be a short (or tap) touch, a long touch, a multi touch, a drag touch on a touch screen. ), Flick touch, pinch-in touch, pinch-out touch, swipe touch, hovering touch, etc. A touch can be sensed.

The ultrasonic sensor may recognize location information of a sensing object using ultrasonic waves. On the other hand, the controller 180 can calculate the position of the wave generation source through the information detected from the optical sensor and the plurality of ultrasonic sensors. The position of the wave source can be calculated using the property that light is much faster than ultrasonic waves, i.e., the time that the light reaches the optical sensor is much faster than the time when the ultrasonic wave reaches the ultrasonic sensor. More specifically, the position of the wave generation source may be calculated using a time difference from the time when the ultrasonic wave reaches the light as the reference signal.

On the other hand, the camera 121, which has been described as the configuration of the input unit 120, includes at least one of a camera sensor (eg, CCD, CMOS, etc.), a photo sensor (or an image sensor), and a laser sensor.

The camera 121 and the laser sensor may be combined with each other to detect a touch of a sensing object on a 3D stereoscopic image. The photo sensor may be stacked on the display element, which is configured to scan the movement of the sensing object in proximity to the touch screen. More specifically, the photo sensor mounts a photo diode and a transistor (TR) in a row / column and scans contents mounted on the photo sensor by using an electrical signal that varies according to the amount of light applied to the photo diode. That is, the photo sensor calculates coordinates of the sensing object according to the amount of change of light, and thus the position information of the sensing object can be obtained.

The display unit 151 displays (outputs) information processed by the artificial intelligence device 100. For example, the display unit 151 may display execution screen information of an application program driven by the artificial intelligence device 100, or UI (User Interface) or GUI (Graphic User Interface) information according to the execution screen information. have.

In addition, the display unit 151 may be configured as a stereoscopic display unit for displaying a stereoscopic image. The stereoscopic display unit may be a three-dimensional display method such as a stereoscopic method (glasses method), an auto stereoscopic method (glasses-free method), a projection method (holographic method).

Generally, a 3D stereoscopic image is composed of a left image (left eye image) and a right image (right eye image). A top-down method in which the left and right images are arranged up and down in one frame according to the way in which the left and right images are combined into a 3D stereoscopic image, and the left and right images are in the left and right in one frame. L-to-R (left-to-right, side by side) method that is arranged as a checker board method to arrange the pieces of the left and right images in the form of tiles, and the left and right images in columns Or an interlaced method of alternately arranging rows, and a time sequential (frame by frame) method of alternately displaying left and right images by time.

In addition, the 3D thumbnail image may generate a left image thumbnail and a right image thumbnail from the left image and the right image of the original image frame, respectively, and may be generated as one image as they are combined. In general, a thumbnail refers to a reduced image or a reduced still image. The left image thumbnail and the right image thumbnail generated as described above may be displayed with a three-dimensional space by displaying a left and right distance difference on the screen by a depth corresponding to the parallax of the left image and the right image.

The left image and the right image necessary for implementing the 3D stereoscopic image may be displayed on the stereoscopic display by the stereoscopic processing unit. The stereoscopic processor is configured to receive a 3D image (an image of a reference time point and an image of an extended time point) and set a left image and a right image therefrom, or to receive a 2D image and convert it to a left image and a right image.

The sound output unit 152 may output audio data received from the wireless communication unit 110 or stored in the memory 170 in a call signal reception, a call mode or a recording mode, a voice recognition mode, a broadcast reception mode, and the like. The sound output unit 152 may also output a sound signal related to a function (eg, a call signal reception sound, a message reception sound, etc.) performed by the artificial intelligence device 100. The sound output unit 152 may include a receiver, a speaker, a buzzer, and the like.

The haptic module 153 generates various tactile effects that a user can feel. A representative example of the tactile effect generated by the haptic module 153 may be vibration. The intensity and pattern of vibration generated by the haptic module 153 may be controlled by the user's selection or the setting of the controller. For example, the haptic module 153 may output different synthesized vibrations or sequentially output them.

In addition to the vibration, the haptic module 153 may be used for stimulation such as a pin array vertically moving with respect to the contact skin surface, a jetting force or suction force of air through the injection or inlet, grazing to the skin surface, contact of an electrode, and electrostatic force. Various tactile effects can be generated, such as the effects of the heat-absorption and the reproduction of the sense of cold using the element capable of generating heat.

The haptic module 153 may not only deliver a tactile effect through direct contact, but also may allow a user to feel the tactile effect through a muscle sense such as a finger or an arm. The haptic module 153 may be provided with two or more according to the configuration aspect of the artificial intelligence device 100.

The light output unit 154 outputs a signal for notifying occurrence of an event by using light of a light source of the artificial intelligence device 100. Examples of events generated in the artificial intelligence device 100 may be message reception, call signal reception, missed call, alarm, schedule notification, email reception, information reception through an application, and the like.

The signal output from the light output unit 154 is implemented as the artificial intelligence device emits light of a single color or a plurality of colors to the front or the rear. The signal output may be terminated by the artificial intelligence device detecting the user's event confirmation.

The interface unit 160 serves as a path to all external devices connected to the artificial intelligence device 100. The interface unit 160 receives data from an external device, receives power, transfers the power to each component inside the artificial intelligence device 100, or transmits data within the artificial intelligence device 100 to an external device. For example, a wired / wireless headset port, an external charger port, a wired / wireless data port, a memory card port, or a port that connects a device equipped with an identification module. The port, an audio input / output (I / O) port, a video input / output (I / O) port, an earphone port, and the like may be included in the interface unit 160.

On the other hand, the identification module is a chip that stores a variety of information for authenticating the use authority of the artificial intelligence device 100, a user identification module (UIM), subscriber identity module (SIM), universal user A universal subscriber identity module (USIM) or the like. A device equipped with an identification module (hereinafter referred to as an 'identification device') may be manufactured in the form of a smart card. Therefore, the identification device may be connected to the terminal 100 through the interface unit 160.

In addition, the interface unit 160 may be a passage for supplying power from the cradle to the artificial intelligence device 100 when the artificial intelligence device 100 is connected to an external cradle, or by a user in the cradle. Various command signals inputted may be a passage through which the artificial intelligence device 100 is transmitted. Various command signals or power input from the cradle may operate as signals for recognizing that the artificial intelligence device 100 is correctly mounted on the cradle.

The memory 170 may store a program for the operation of the controller 180 and may temporarily store input / output data (for example, a phone book, a message, a still image, a video, etc.). The memory 170 may store data relating to various patterns of vibration and sound output when a touch input on the touch screen is performed.

The memory 170 may include a flash memory type, a hard disk type, a solid state disk type, an SSD type, a silicon disk drive type, and a multimedia card micro type. ), Card type memory (e.g. SD or XD memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read It may include at least one type of storage medium of -only memory (PROM), programmable read-only memory (PROM), magnetic memory, magnetic disk and optical disk. The artificial intelligence device 100 may be operated in connection with a web storage that performs a storage function of the memory 170 on the Internet.

On the other hand, as described above, the controller 180 controls the operation related to the application program, and generally the overall operation of the artificial intelligence device 100. For example, if the state of the artificial intelligence device satisfies a set condition, the controller 180 may execute or release a lock state that restricts input of a user's control command to applications.

In addition, the controller 180 may perform control and processing related to a voice call, data communication, video call, or the like, or may perform pattern recognition processing for recognizing handwriting or drawing input performed on a touch screen as text and images, respectively. Can be. Furthermore, the controller 180 may control any one or a plurality of components described above in order to implement various embodiments described below on the artificial intelligence device 100 according to the present invention.

The power supply unit 190 receives an external power source and an internal power source under the control of the controller 180 to supply power for operation of each component. The power supply unit 190 includes a battery, and the battery may be a built-in battery configured to be rechargeable, and may be detachably coupled to the terminal body for charging.

In addition, the power supply unit 190 may be provided with a connection port, the connection port may be configured as an example of the interface 160 that is electrically connected to the external charger for supplying power for charging the battery.

As another example, the power supply unit 190 may be configured to charge the battery in a wireless manner without using the connection port. In this case, the power supply unit 190 may use at least one of an inductive coupling based on a magnetic induction phenomenon or a magnetic resonance coupling based on an electromagnetic resonance phenomenon from an external wireless power transmitter. Power can be delivered.

Meanwhile, various embodiments of the present disclosure may be implemented in a recording medium readable by a computer or a similar device using, for example, software, hardware, or a combination thereof.

Meanwhile, the description of the artificial intelligence device 100 described with reference to FIG. 1 may be applied to other

artificial intelligence devices

200, 300, 400, and 500 as described below.

The plurality of

artificial intelligence devices

100, 200, 300, 400, and 500 may communicate with the artificial intelligence server 600. In more detail, each of the plurality of

artificial intelligence devices

100, 200, 300, 400, and 500 may include a communication unit, and the communication unit may provide an interface for connecting an electronic device to a wired / wireless network including an internet network. have. The communication unit may transmit or receive data with the server through the connected network or another network linked to the connected network.

Meanwhile, the plurality of

artificial intelligence devices

100, 200, 300, 400, and 500 may learn a voice signal or perform a function corresponding to the voice data in various ways.

For example, when a recognition model is mounted on the server 600, and the plurality of

AI devices

100, 200, 300, 400, and 500 receive an input signal and transmit it to the server 600, the server 600 inputs the input signal. Personalize the recognition model by using the signal, or output the recognition result for the input signal to the plurality of artificial intelligence devices (100, 200, 300, 400, 500), the plurality of artificial intelligence devices (100, 200, 300) , 400 and 500 may be implemented by generating a control command corresponding to the recognition result and performing control.

In another example, when the recognition model is mounted on the server 600 and the plurality of

artificial intelligence devices

100, 200, 300, 400, and 500 receive an input signal and transmit the input signal to the server 600, the server 600 Personalize the recognition model using the input signal, output the recognition result for the input signal, and transmit a control command corresponding to the recognition result to the plurality of

artificial intelligence devices

100, 200, 300, 400, and 500. Can be implemented.

In another example, a recognition model is mounted on the plurality of

artificial intelligence devices

100, 200, 300, 400, and 500, and the plurality of

artificial intelligence devices

100, 200, 300, 400, 500 receive and recognize an input signal. Personalize the model or output the recognition result for the input signal to the server 600, and the server 600 sends a control command corresponding to the recognition result of the plurality of artificial intelligence devices (100, 200, 300, 400, 500) It can be implemented by transmitting to.

In addition, the plurality of

artificial intelligence devices

100, 200, 300, 400, and 500 may independently perform artificial intelligence functions regardless of the server 600.

For example, the recognition model is mounted on the plurality of

artificial intelligence devices

100, 200, 300, 400, and 500, and the plurality of

artificial intelligence devices

100, 200, 300, 400, 500 receive an input signal to recognize the recognition model. Can be personalized or output the recognition result for the input signal, and generate a control command corresponding to the recognition result.

The communication unit 610 may communicate with an external device. In detail, the communication unit 610 may communicate with a plurality of

artificial intelligence devices

100, 200, 300, 400, and 500.

The artificial intelligence unit 630 may receive an input signal from the plurality of

artificial intelligence devices

100, 200, 300, 400, and 500 through the communication unit 610.

In addition, the artificial intelligence unit 630 outputs a recognition result for an input signal using the recognition model 700, and transmits the output recognition result to the plurality of

artificial intelligence devices

100, 200, 300, 400, and 500. The control command corresponding to the output recognition result may be transmitted to the plurality of

artificial intelligence devices

100, 200, 300, 400, and 500.

In addition, the artificial intelligence unit 630 may store personalized information, which is personalized by learning an input signal from a learning model of the artificial intelligence device, in the storage 620.

The storage unit 620 may be a database used in the recognition model 700, personalization information of the plurality of

artificial intelligence devices

100, 200, 300, 400, and 500, and a plurality of

artificial intelligence devices

100, 200, 300, and 400. , At least one of feature information of 500).

Hereinafter, it is assumed that the recognition model 700 is a speech recognition model.

In this case, the input signal may be a voice signal of the user, and the voice signal of the user may be input to the recognition model 700 as an input value.

In this case, the recognition model 700 may extract a recognition result by analyzing a voice signal and extracting a feature, wherein the recognition result indicates whether the received voice signal is a command or a non-command, or a command among a plurality of commands. Can be.

Herein, the command may be pre-registered to perform a function of the AI device, and the non-command may be irrelevant to the performance of the function of the AI device.

Voice recognition refers to converting a speech signal into a string or identifying linguistic semantic content by interpreting the speech signal and combining it with a patterned database.

The recognition model 700 analyzes the received speech signal, extracts a feature, and measures similarity with a previously collected speech model database to convert the most similar into a text or a command.

In addition, the recognition model 700 may increase the recognition rate of the user's voice signal by learning the user's speech characteristics from the voice signal.

The recognition model 700 may include a learning model 710 and a separator 720.

When the voice signal is input, the learning model 710 may analyze the input voice signal to extract a feature of the voice signal and output an output value obtained by converting the feature of the voice signal.

In detail, when a voice signal is input, the learning model 710 may extract a feature of the input signal and output the extracted signal in the form of a vector.

FIG. 5A illustrates the vector in a 2D form. When the input signal is transformed by the training model 710, the instruction 810 is collected at a specific position, and the non-instruction 820 is positioned at the periphery.

That is, the learning model 710 may operate as a mapping function that maps an input signal from an input signal domain to a feature domain.

In this case, the divider 720 may divide the output value into an instruction 810 and a non-instruction 820 using the division boundary 830.

Meanwhile, the division boundary shown in FIG. 5A may be a division boundary of an unpersonalized state. Accordingly, the learning model 710 may be configured as a deep learning model using a neural network or a model using other hand-craft features to learn speech characteristics of a specific user.

In more detail, the learning model 710 may personalize the recognition model 700 to a specific user by separately collecting command / non-instruction samples of a specific user or generating a sample by assuming a specific user in advance.

In FIG. 5B, personalization is performed and a new division boundary 860 is set. The personalized recognition model 700 may divide the output value into a command 840 of a specific user and a non-command 850 of a specific user.

As such, the personalized information may be referred to as personalized information by learning an input signal in a learning model of a specific artificial intelligence device.

3 to 5 illustrate that the recognition model 700 is mounted and operated in a server, the present invention is not limited thereto. The recognition model 700 may be applied to a plurality of

artificial intelligence devices

100, 200, 300, 400, and 500. 3 and 5 may be performed by the plurality of

artificial intelligence devices

100, 200, 300, 400, and 500.

If a new second AI device 200 is added while the first AI device 100 personalized already exists in the same use environment, the personalization information of the first AI device 100 already personalized is removed. If the 2 AI device 200 can be used, personalization of the second AI device can be shortened in time.

However, there may be a difference between the characteristic information of the first artificial intelligence device and the characteristic information of the second artificial intelligence device.

Specifically, in the case of the voice recognition model, the voice input device of the first artificial intelligence device and the voice input device of the second artificial intelligence device may be different. Accordingly, although the voice signal input to the input device is the same, a difference may occur in the input signal output from the input device to the recognition model 700.

For example, the microphone of the first artificial intelligence device and the microphone of the second artificial intelligence device may have different amplitudes, average values, echoes, noise sensitivity, and the like. For example, the first AI device and the second AI device may have different microphone installation positions, the number of microphones, a space in which the first AI device and the second AI device are installed, and a preprocessing technique for the micro-input signal. Can be.

Therefore, even if the personalization information of the first artificial intelligence device 100 is applied to the second artificial intelligence device 200 as it is, the instruction 840 and the non-instruction 850 of the first artificial intelligence device 100 in FIG. The distribution and the distribution of the instruction 870 and the non-instruction 880 of the second artificial intelligence device 200 in FIG. 6 may be different.

In this case, a voice signal recognized as a command in the first artificial intelligence device 100 may also be recognized as a non-command by the second artificial intelligence device 200, and conversely, as a command in the second artificial intelligence device 200. The voice signal may also be recognized as a non-command by the first artificial intelligence device 100. Therefore, personalization in the second artificial intelligence device 200 will fail.

Therefore, in order to apply the personalized information of the first artificial intelligence device 100 to the second artificial intelligence device 200, between the input device of the first artificial intelligence device 100 and the input device of the second artificial intelligence device 200. There is a need for a technique that can offset the difference in characteristics.

The artificial intelligence unit 630 of the artificial intelligence server 600 may generate personalization information of the first artificial intelligence device based on the characteristic information of the first artificial intelligence device and the characteristic information of the second artificial intelligence device. A recognition model of the second artificial intelligence device reflected to correspond to the characteristic information of may be obtained.

In more detail, the characteristic information of the first artificial intelligence device and the characteristic information of the second artificial intelligence device are stored in the storage unit 620 of the artificial intelligence server 600, or from the first artificial intelligence device and the second artificial intelligence device, respectively. Can be received.

In addition, the personalization information of the first artificial intelligence device may be stored in the storage unit 620 of the artificial intelligence server 600 or may be received from the first artificial intelligence device.

The artificial intelligence unit 630 of the artificial intelligence server 600 may generate personalization information of the first artificial intelligence device based on a difference between the characteristic information of the first artificial intelligence device and the characteristic information of the second artificial intelligence device. The recognition model of the second artificial intelligence device reflected to correspond to the characteristic information of the artificial intelligence device may be obtained.

In this case, the artificial intelligence unit 630 inputs a difference between the input device of the first artificial intelligence device and the second artificial intelligence device based on the difference between the characteristic information of the first artificial intelligence device and the characteristic information of the second artificial intelligence device. First correction information for applying to a signal may be obtained.

Here, the first modification information is information for converting a voice signal output from the input device of the second artificial intelligence device together with the voice signal output from the input device of the first artificial intelligence device, and the characteristic information of the first artificial intelligence device. And the difference between the characteristic information of the second artificial intelligence device.

In this case, the first modification information may be a mapping function 730 that may cancel a difference between the characteristic information of the input apparatus of the first artificial intelligence device and the characteristic information of the input apparatus of the second artificial intelligence device.

In this case, the first modification information may be applied to the voice signal output from the input device of the second artificial intelligence device.

The voice signal to which the first correction information is applied may be input to the learning model 710, and the output value of the learning model 710 may be input to the separator 720 reflecting the division boundary of the first artificial intelligence device. That is, the recognition model 700 of the second artificial intelligence device reflecting the personalization information of the first artificial intelligence device includes a mapping function for applying the first modification information to the speech signal output from the input device of the second artificial intelligence device. mapfunction) 730, a learning model 710 for inputting a speech signal to which the first modification information is applied, and outputting a result of converting the input speech signal, and a learning model by reflecting personalization information of the first AI device. It may include a separator 720 for dividing the output value of the 710 into an instruction and a non-instruction.

In FIG. 8, for the same voice 1110 spoken by a user, a voice signal (FIG. 8A) converted into an electrical signal at an input device of a first artificial intelligence device and an input signal of a second artificial intelligence device are converted into an electric signal. Voice signal (FIG. 8B) is shown.

According to the characteristic difference between the input devices, a difference occurs between the voice signal output from the input device of the first artificial intelligence device (FIG. 8A) and the voice signal output from the input device of the second artificial intelligence device (FIG. 8B).

By applying the first correction information to the voice signal (FIG. 8B) output from the input device of the second artificial intelligence device, the voice signal to which the first correction information is applied (FIG. 8C) is transferred from the input device of the first artificial intelligence device. It may be converted similarly to the output audio signal (FIG. 8A).

Accordingly, the output value output from the learning model of the first artificial intelligence device and the output value output from the learning model of the second artificial intelligence device are similar to each other, so that the personalization information of the first artificial intelligence device can be used as it is in the second artificial intelligence device. Can be.

First, an implicit implementation will be described with reference to FIG. 9A.

The artificial intelligence unit 630 of the artificial intelligence server 600, based on the difference between the characteristic information of the first artificial intelligence device and the characteristic information of the second artificial intelligence device, the personalization information of the first artificial intelligence device second artificial intelligence A recognition model of the second artificial intelligence device reflected to correspond to the characteristic information of the intelligent device may be obtained.

In this case, the artificial intelligence unit 630 learns the difference between the input device of the first artificial intelligence device and the second artificial intelligence device based on the difference between the characteristic information of the first artificial intelligence device and the characteristic information of the second artificial intelligence device. Second modification information for applying to the model may be obtained.

Here, the second modification information is information for converting the output value output from the learning model of the second artificial intelligence device with the output value output from the learning model of the first artificial intelligence device, and includes the characteristic information and the first information of the first artificial intelligence device. 2 may be determined based on the difference in the characteristic information of the artificial intelligence device.

That is, the learning model 710 operates as a mapping function that maps an input signal from an input signal domain to a feature domain, and the second modification information uses a mapping function to map characteristic information of the second artificial intelligence device. It may be information to correct in consideration of the.

In this case, the learning model 710 of the second artificial intelligence device may be modified to reflect the second modification information, and the voice signal output from the input device of the second artificial intelligence device is input to the modified learning model 710. Can be.

The output value of the modified learning model 710 may be input to the separator 720 reflecting the division boundary of the first artificial intelligence device.

That is, the recognition model 700 of the second artificial intelligence device reflecting the personalization information of the first artificial intelligence device is inputted with a voice signal output from the input device of the second artificial intelligence device and modified to reflect the second modification information. It may include a separator 720 for dividing the output value of the learning model 710 into instructions and non-instructions by reflecting the training model 710 and personalization information of the first artificial intelligence device.

Next, an explicit implementation will be described with reference to FIG. 9B.

In this case, the artificial intelligence unit 630 learns the difference between the input device of the first artificial intelligence device and the second artificial intelligence device based on the difference between the characteristic information of the first artificial intelligence device and the characteristic information of the second artificial intelligence device. Third modification information for applying to an output value of the model may be obtained.

The third modification information is information for converting an output value output from the learning model of the second artificial intelligence device with an output value output from the learning model of the first artificial intelligence device. 2 may be determined based on the difference in the characteristic information of the artificial intelligence device.

In this case, the third modification information may be a mapping function 740 that may cancel a difference between the characteristic information of the input apparatus of the first artificial intelligence device and the characteristic information of the input apparatus of the second artificial intelligence device.

In this case, a voice signal output from the input device of the second artificial intelligence device may be input to the learning model 710, and third modified information may be applied to the output value of the learning model 710.

The output value to which the third modification information is applied may be input to the separator 720 reflecting the division boundary of the first artificial intelligence device.

In other words, the learning model 710 operates as a mapping function that maps an input signal from an input signal domain to a feature domain, and the third correction information uses the second artificial information to output a result value output from the mapping function. The information may be modified once again in consideration of the characteristic information of the intelligent device.

That is, the recognition model 700 of the second artificial intelligence device reflecting the personalization information of the first artificial intelligence device includes a learning model 710 and a learning model in which a voice signal output from an input device of the second artificial intelligence device is input. Map function 740 to apply the third modification information to the output value of the < RTI ID = 0.0 >) < / RTI > It may include a separator 720 for separating.

FIG. 10A illustrates an output value of the learning model of the first AI device, and FIG. 10B illustrates an output value of the learning model of the second AI device for the same voice of the same user.

Due to the difference in characteristics between the first AI device and the second AI device, the distribution of the instruction 1310 and the non-instruction 1320 of FIG. 10A and the distribution of the instruction 1330 and the non-instruction 1340 of FIG. 10B are different. Will be displayed.

In this case, in the explicit implementation described with reference to FIG. 9B, as illustrated in FIG. 10C, third modification information may be applied to an output value of the learning model of the second artificial intelligence device. In this case, the output value of the learning model of the second artificial intelligence device may be mapped once again to represent a distribution as illustrated in FIG. 10D. In this case, the instruction 1330 enters into the division boundary 1350 of the first artificial intelligence device.

That is, in the process of mapping the input signal from the input signal domain to the feature domain, by performing additional mapping reflecting characteristic differences between devices, personalization information of the first AI device may be applied to the second AI device.

Meanwhile, in the implementation of the implementation described in FIG. 9A, by modifying the mapping function of the second artificial intelligence device itself, the output value of the training model 710 is as shown in FIG. 10D without the process of FIG. 10C. It can represent the same distribution.

That is, the device may modify the mapping function for mapping the input signal from the input signal domain to the feature domain to reflect the characteristic difference, thereby applying personalization information of the first AI device to the second AI device.

In this case, the artificial intelligence unit 630 classifies the difference between the input device of the first artificial intelligence device and the second artificial intelligence device based on the difference between the characteristic information of the first artificial intelligence device and the characteristic information of the second artificial intelligence device. Fourth modification information for applying to the device may be obtained.

Here, the fourth modification information may be information for correcting the division boundary in consideration of the difference in the characteristic information between the artificial intelligence devices in the state in which the division boundary of the first artificial intelligence device is reflected.

That is, the artificial intelligence unit 730 may change the division boundary of the second artificial intelligence device by reflecting the personalization information of the first artificial intelligence device, that is, the division boundary of the first artificial intelligence device. Also, the artificial intelligence unit 730 may again modify the changed division boundary based on the fourth modification information.

The recognition model 700 of the second artificial intelligence device reflecting the personalization information of the first artificial intelligence device includes a learning model 710 and a first artificial intelligence device to which a voice signal output from an input device of the second artificial intelligence device is input. Reflecting the personalization information of the may include a separator 720 further modified according to the fourth modification information.

12A illustrates an output value and a division boundary of the learning model of the first AI device, and FIG. 12B illustrates an output value and the division boundary of the learning model of the second AI device with respect to the same voice of the same user.

Due to the difference in characteristics between the first AI device and the second AI device, the distribution of the instruction 1510 and the non-instruction 1520 of FIG. 12A and the distribution of the instruction 1540 and the non-instruction 1550 of FIG. 12B are different. Will be displayed.

In this case, the artificial intelligence unit 630 first reflects the division boundary 1530 of the first artificial intelligence device and modifies the division boundary 1530 of the first artificial intelligence device according to the fourth modification information ( 1560 can be set. In this case, the instruction 1540 enters into the division boundary 1560 of the second artificial intelligence device.

That is, the personalized information of the first artificial intelligence device can be applied to the second artificial intelligence device by modifying the division boundary to reflect the characteristic difference between the devices.

The artificial intelligence server 600 may receive an update request of the recognition model in operation S1410.

Specifically, when the second artificial intelligence device approaches the first artificial intelligence device, the first artificial intelligence device detects the approach of the second artificial intelligence device by receiving a Wi-Fi signal, a Bluetooth signal, an infrared signal, etc. of the second artificial intelligence device. can do. In this case, the first AI device may request the server 600 to update the recognition model of the second AI device.

In the same principle, it is also possible for the second AI device to request the server to update the recognition model of the second AI device.

Meanwhile, the server 600 may acquire a recognition model of the second artificial intelligence device reflecting personalization information of the first artificial intelligence device to correspond to the characteristic information of the second artificial intelligence device (S1430).

Specifically, when the recognition model of the first artificial intelligence device is mounted on the first artificial intelligence device, the first artificial intelligence device may transmit personalization information of the first artificial intelligence device to the server along with a request for updating the recognition model. .

In this case, the server 600 reflects the personalization information of the first artificial intelligence device in the recognition model of the second artificial intelligence device and according to the characteristic information of the first artificial intelligence device and the characteristic information of the second artificial intelligence device. The recognition model of the intelligent device can be modified.

On the other hand, when the recognition model of the first artificial intelligence device and the recognition model of the second artificial intelligence device is mounted on the server, the server reflects the personalization information of the first artificial intelligence device possessed in the recognition model of the second artificial intelligence device The recognition model of the second artificial intelligence device may be modified according to the characteristic information of the first artificial intelligence device and the characteristic information of the second artificial intelligence device.

Meanwhile, when the recognition model of the second artificial intelligence device is mounted on the second artificial intelligence device, the server 600 may update the recognition model of the new second artificial intelligence device to the second artificial intelligence device (S1450).

When the second artificial intelligence device approaches the first artificial intelligence device, the first artificial intelligence device may transmit personalization information and characteristic information of the first artificial intelligence device to the second artificial intelligence device.

In this case, the second artificial intelligence device may receive personalization information and characteristic information from the first artificial intelligence device (S1510).

In operation S1530, the second AI device may acquire a recognition model of the second AI device in which personalization information of the first AI device is reflected to correspond to characteristic information of the second AI device.

In this case, the second AI device may update the recognition model of the new second AI device.

In addition, the second artificial intelligence device may modify the recognition model by learning an input signal received from the second artificial intelligence device (S1550). That is, in addition to reflecting personalized information of the first AI device, personalization that is suitable for the characteristics of the second AI device may be continuously performed through additional learning.

In addition, the operations and functions of the second AI device in the case where the first AI device and the second AI device communicate directly to update the recognition model, all operations and functions of the AI server described with reference to FIGS. This can be applied.

The first artificial intelligence device 100 may detect the second artificial intelligence device 200 (S1805).

Specifically, when the second artificial intelligence device approaches the first artificial intelligence device, the first artificial intelligence device detects the approach of the second artificial intelligence device by receiving a Wi-Fi signal, a Bluetooth signal, an infrared signal, etc. of the second artificial intelligence device. can do.

As another example, the first artificial intelligence device 100 may detect the second artificial intelligence device 200 as an input of a user's touch, gesture, voice, or the like is received.

Meanwhile, the first artificial intelligence device 100 may be an in-house master device.

In this case, the first AI device 100 may determine whether an update of the recognition model of the second AI device 200 is required (S1810), and if it is necessary, determine whether or not an update authority is held (S1815).

On the other hand, if the update authority does not hold the first AI device 100 may request the update authority from the user (S1820).

In this case, the first AI device 100 “new device, model name BBB was detected”, “the device can be quickly personalized with the information of the existing AAA model. Do you want to apply it? ”.

Meanwhile, when a user input for granting an update right is received, the first AI device 100 may start an update procedure.

In addition, when the first AI device 100 has the update right, the update procedure may be started.

Meanwhile, the first AI device 100 may determine whether the update can be performed using the self-recognition model in operation S1830.

For example, when the first artificial intelligence device 100 and the second artificial intelligence device 200 have the same product or have an input device having the same characteristics, the first artificial intelligence device 100 and the second artificial intelligence device 200 have their own database. 2 The recognition model of the artificial intelligence device 200 may be updated.

Therefore, when the update is possible using the self recognition model, the first AI device 100 may transmit the recognition model of the first AI device to the second AI device 200 (S1835).

On the other hand, when the update of the recognition model is completed in the second AI device, the first AI device may output an update completion notification of the second AI device (S1865).

On the other hand, when the first AI device 100 cannot update using the self recognition model (S1830), the first AI device 100 may request the server 600 to update the recognition model (S1830). S1840).

When the update request of the recognition model is received, the server 600 may search for the recognition model for the second artificial intelligence device 200 (S1845).

On the other hand, if there is a recognition model for the second AI device 200 (S1850), the AI server 600 updates the new recognition model in the second AI device 200 (S1860), the first artificial The update completion notification may be transmitted to the intelligent device 100.

On the other hand, when there is no recognition model for the second AI device 200 (S1850), the artificial intelligence server 600 may obtain a new recognition model for the second AI device 200 (S1855). .

In this case, the AI server 600 reflects the personalization information of the first AI device to correspond to the property information of the second AI device based on the property information of the first AI device and the property information of the second AI device. A recognition model of the second artificial intelligence device may be generated.

In this case, the artificial intelligence server 600 may update the new recognition model to the second artificial intelligence device 200 (S1860) and may transmit an update completion notification to the first artificial intelligence device 100.

In the above description, the speech recognition model has been described as an example, but the present invention is not limited thereto. The present invention can be applied to all artificial intelligence devices capable of learning and personalizing a use environment.

For example, in the case of the robot cleaner, the input device for recognizing the use environment may be a camera. In addition, the brightness of the image, the viewing angle, the noise, and the sensitivity of the camera may vary according to the camera module, the lens, the sensor size, the image processing engine, and so on.

Even in such a case, the recognition model of the second AI device may include the characteristic information of the first AI device and the second AI device (camera module, lens, sensor size, image processing engine, image brightness, viewing angle, noise, sensitivity, etc.). ), The personalization information of the first artificial intelligence device may be reflected to correspond to the characteristic information of the second artificial intelligence device.

Accordingly, the second AI device may modify the terrain recognition model according to the space and obstacles in the home by reflecting personalization information of the first AI device.

In the above description, the difference of the characteristic information between the input apparatuses has been described as an example, but the present invention is not limited thereto. The present invention may be applied based on the characteristic difference of the configuration other than the input apparatus.

For example, the robot cleaner may include a sensor for measuring a load on the motor of the robot cleaner.

In this case, the robot cleaner may determine the load on the motor using the sensor. Accordingly, the robot cleaner may determine whether the robot cleaner is on the carpet or the floor of the general floor, and perform a turbo mode operation according to the determination result.

However, there may be a difference in load depending on the type of rug, and personalization is required for each home for accurate operation.

In addition, since the load varies depending on the motor, when the motor is different, the previously learned personalization information may be different, and relearning is required.

However, according to the present invention, personalization information of the first artificial intelligence device can be applied to the second artificial intelligence device based on not only the difference between the input devices, but also the difference in the other configuration (motor performance) of the artificial intelligence device.

As described above, according to the present invention, despite the difference in characteristics between the first AI device and the second AI device, since the learning result of the first AI device can be adopted in the second AI device, the second AI device can be employed. There is an advantage that can significantly reduce the time required for personalization of the device.

The controller or artificial intelligence unit may be used interchangeably with terms such as a central processing unit, a microprocessor, and a processor.

The present invention described above can be embodied as computer readable codes on a medium on which a program is recorded. The computer-readable medium includes all kinds of recording devices in which data that can be read by a computer system is stored. Examples of computer-readable media include hard disk drives (HDDs), solid state disks (SSDs), silicon disk drives (SDDs), ROMs, RAMs, CD-ROMs, magnetic tapes, floppy disks, optical data storage devices, and the like. There is this. In addition, the computer may include the controller 180 of the terminal. Accordingly, the above detailed description should not be construed as limiting in all aspects and should be considered as illustrative. The scope of the present invention should be determined by reasonable interpretation of the appended claims, and all changes within the equivalent scope of the present invention are included in the scope of the present invention.

Claims

Communication unit for communicating with an external device; And

The second artificial intelligence device reflecting personalization information of the first artificial intelligence device to correspond to the characteristic information of the second artificial intelligence device based on the characteristic information of the first artificial intelligence device and the characteristic information of the second artificial intelligence device; An artificial intelligence unit for obtaining a recognition model of

AI server.
The method of claim 1,

Personalization information of the first artificial intelligence device,

Personalized information obtained by learning an input signal from a learning model of the first artificial intelligence device,

The artificial intelligence unit,

The first reflecting personalization information of the first artificial intelligence device to correspond to the characteristic information of the second artificial intelligence device based on a difference between the characteristic information of the first artificial intelligence device and the characteristic information of the second artificial intelligence device; 2 Acquiring Recognition Models of Artificial Intelligence Devices

AI server.
The method of claim 2,

The recognition model of the second artificial intelligence device,

A learning model outputting an output value obtained by converting a feature of the input signal when the input signal is input; And

A delimiter for dividing the output value into a command and a non-command using a division boundary;

AI server.
The method of claim 3, wherein

The artificial intelligence unit,

The difference between the input device of the first artificial intelligence device and the input device of the second artificial intelligence device is based on a difference between the property information of the first artificial intelligence device and the property information of the second artificial intelligence device. Obtain first modification information for applying to

Applying the first correction information to an input signal output from an input device of the second artificial intelligence device, and acquiring the recognition model for inputting the input signal to which the first modification information is applied to the learning model;

AI server.
The method of claim 3, wherein

The artificial intelligence unit,

The learning model determines a difference between an input device of the first AI device and an input device of the second AI device based on a difference between the property information of the first AI device and the property information of the second AI device. Obtain second modification information for applying to

Obtaining the recognition model including the modified learning model by applying the second modification information

AI server.
The method of claim 3, wherein

The artificial intelligence unit,

The learning model determines a difference between an input device of the first AI device and an input device of the second AI device based on a difference between the property information of the first AI device and the property information of the second AI device. Obtain third modification information for applying to an output value of,

Obtaining the recognition model applying the third modification information to an output value of the learning model

AI server.
The method of claim 3, wherein

The artificial intelligence unit,

The difference between the input device of the first AI device and the input device of the second AI device is based on a difference between the property information of the first AI device and the property information of the second AI device. Obtain fourth modification information for application,

Acquiring the recognition model in which the division boundary reflecting the personalization information is corrected using the fourth modification information.

AI server.
The method of claim 1,

The artificial intelligence unit,

When the recognition model of the second artificial intelligence device is obtained, updating the acquired recognition model to the second artificial intelligence device

AI server.
Obtaining characteristic information of the first artificial intelligence device and characteristic information of the second artificial intelligence device; And

The second artificial intelligence device reflecting personalization information of the first artificial intelligence device corresponding to the characteristic information of the second artificial intelligence device based on the characteristic information of the first artificial intelligence device and the characteristic information of the second artificial intelligence device; Obtaining a recognition model of the

How AI Servers Work.
The method of claim 9,

Personalization information of the first artificial intelligence device,

Personalized information obtained by learning an input signal from a learning model of the first artificial intelligence device,

Acquiring the recognition model of the second artificial intelligence device,

The first reflecting personalization information of the first artificial intelligence device to correspond to the characteristic information of the second artificial intelligence device based on a difference between the characteristic information of the first artificial intelligence device and the characteristic information of the second artificial intelligence device; 2 Acquiring Recognition Models of Artificial Intelligence Devices

How AI Servers Work.
The method of claim 10,

The recognition model of the second artificial intelligence device,

A learning model outputting an output value obtained by converting a feature of the input signal when the input signal is input; And

A delimiter for dividing the output value into a command and a non-command using a division boundary;

How AI Servers Work.
The method of claim 11,

Acquiring the recognition model of the second artificial intelligence device,

The difference between the input device of the first artificial intelligence device and the input device of the second artificial intelligence device is based on a difference between the property information of the first artificial intelligence device and the property information of the second artificial intelligence device. Obtaining first modification information for applying to the; And

Applying the first correction information to an input signal output from an input device of the second artificial intelligence device, and acquiring the recognition model for inputting the input signal to which the first modification information is applied to the learning model; Containing

How AI Servers Work.
The method of claim 11,

Acquiring the recognition model of the second artificial intelligence device,

The learning model determines a difference between an input device of the first AI device and an input device of the second AI device based on a difference between the property information of the first AI device and the property information of the second AI device. Obtaining second modification information for applying to the; And

Obtaining the recognition model including the modified learning model by applying the second modification information;

How AI Servers Work.
The method of claim 11,

Acquiring the recognition model of the second artificial intelligence device,

The learning model determines a difference between an input device of the first AI device and an input device of the second AI device based on a difference between the property information of the first AI device and the property information of the second AI device. Obtaining third modification information for applying to an output value of the; And

Obtaining the recognition model applying the third correction information to an output value of the learning model.

How AI Servers Work.
The method of claim 11,

Acquiring the recognition model of the second artificial intelligence device,

The difference between the input device of the first artificial intelligence device and the input device of the second artificial intelligence device is based on a difference between the property information of the first artificial intelligence device and the property information of the second artificial intelligence device. Obtaining fourth modification information for application; And

Obtaining the recognition model in which the division boundary reflecting the personalization information is corrected using the fourth modification information.

How AI Servers Work.