WO2020209693A1 - Dispositif électronique pour mise à jour d'un modèle d'intelligence artificielle, serveur, et procédé de fonctionnement associé - Google Patents

Dispositif électronique pour mise à jour d'un modèle d'intelligence artificielle, serveur, et procédé de fonctionnement associé Download PDF

Info

Publication number
WO2020209693A1
WO2020209693A1 PCT/KR2020/004965 KR2020004965W WO2020209693A1 WO 2020209693 A1 WO2020209693 A1 WO 2020209693A1 KR 2020004965 W KR2020004965 W KR 2020004965W WO 2020209693 A1 WO2020209693 A1 WO 2020209693A1
Authority
WO
WIPO (PCT)
Prior art keywords
electronic device
gradient
information
model
local model
Prior art date
Application number
PCT/KR2020/004965
Other languages
English (en)
Korean (ko)
Inventor
러셀 제라치제임스
시비첸
로러아옹후스
스미스베리
헐리네일
동루이하이
Original Assignee
삼성전자 주식회사
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from KR1020190120851A external-priority patent/KR20200120469A/ko
Application filed by 삼성전자 주식회사 filed Critical 삼성전자 주식회사
Priority to EP20788289.5A priority Critical patent/EP3816917A4/fr
Priority to CN202080010910.1A priority patent/CN113366520A/zh
Publication of WO2020209693A1 publication Critical patent/WO2020209693A1/fr

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q30/00—Commerce
    • G06Q30/02—Marketing; Price estimation or determination; Fundraising
    • G06Q30/0241—Advertisements
    • G06Q30/0251—Targeted advertisements
    • G06Q30/0254—Targeted advertisements based on statistics
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q30/00—Commerce
    • G06Q30/02—Marketing; Price estimation or determination; Fundraising
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q30/00—Commerce
    • G06Q30/02—Marketing; Price estimation or determination; Fundraising
    • G06Q30/0201—Market modelling; Market analysis; Collecting market data
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q30/00—Commerce
    • G06Q30/02—Marketing; Price estimation or determination; Fundraising
    • G06Q30/0201—Market modelling; Market analysis; Collecting market data
    • G06Q30/0202—Market predictions or forecasting for commercial activities
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q30/00—Commerce
    • G06Q30/02—Marketing; Price estimation or determination; Fundraising
    • G06Q30/0241—Advertisements
    • G06Q30/0251—Targeted advertisements
    • G06Q30/0269—Targeted advertisements based on user profile or attribute

Definitions

  • the present disclosure relates to an electronic device, a server, and a method of operating the same for updating an artificial intelligence model stored in the electronic device.
  • the artificial intelligence model is continuously updated according to the changed situation information. Accordingly, the artificial intelligence model may be continuously updated based on various types of situation information collected in real time so that an appropriate operation can be performed by the artificial intelligence model according to the situation information.
  • the update of the artificial intelligence model may be performed in a server with better performance than the electronic device possessed by the user in terms of the amount of computation and data used for learning.
  • the artificial intelligence model stored in the electronic device may be additionally updated based on the result of updating the artificial intelligence model performed by the server. Accordingly, the electronic device may transmit context information of the electronic device to the server in order to update the artificial intelligence model.
  • the context information of the electronic device may include personal information, biometric information, secret information, financial information, etc., including the user's sensitive information, if the context information is leaked to the outside during the transmission process, serious damage to the user There is a problem that can occur.
  • the problem to be solved by the present disclosure is to solve the above-described problem, and to provide an electronic device, a server, and an operating method thereof for updating an artificial intelligence model stored in the electronic device.
  • FIG. 1 is a diagram illustrating a system for performing various operations using a local model of an electronic device according to an embodiment.
  • FIG. 2 is a diagram illustrating a system for performing various operations using a local model according to an embodiment.
  • FIG. 3 is a diagram illustrating a system for performing object recognition using a local model according to an exemplary embodiment.
  • FIG. 4 is a diagram illustrating a system for performing speech recognition using a local model according to an embodiment.
  • FIG. 5 is a diagram illustrating an example in which a local model and a global model are updated according to an embodiment.
  • FIG. 6 is a diagram illustrating an example of a local model or a global model according to an embodiment.
  • FIG. 7 is a diagram for describing an operation of an electronic device according to an exemplary embodiment.
  • FIG. 8 is a diagram illustrating an example in which a local model is updated according to the passage of time according to an embodiment.
  • FIG. 9 is a block diagram illustrating an internal configuration of an electronic device according to an exemplary embodiment.
  • FIG. 10 is a block diagram illustrating an internal configuration of an electronic device according to an embodiment.
  • FIG. 11 is a block diagram showing an internal configuration of a server according to an embodiment.
  • FIG. 12 is a flowchart illustrating a method of updating a local model according to an embodiment.
  • FIG. 13 is a flowchart illustrating a method of updating a global model in a server according to an embodiment.
  • FIG. 14 is a diagram illustrating a system for updating a local model and a global model according to an embodiment.
  • 15 is a diagram illustrating a system for updating a local model and a global model through an intermediary device according to an embodiment.
  • 16 is a flowchart illustrating a method of updating a global model and a local model according to a gradient according to an embodiment.
  • 17 is a flowchart illustrating a method of updating a global model, a first local model of an electronic device, and a second local model of an external device according to a gradient, according to an exemplary embodiment.
  • FIG. 18 is a flowchart illustrating a method of updating a global model, a first local model of an electronic device, and a second local model of an external device according to a gradient, according to an exemplary embodiment.
  • a first aspect of the present disclosure includes: detecting that information about a situation of the electronic device is changed, which is used to update a local model stored in the electronic device; Determining gradient information for updating the local model, based on the information on the changed situation; Updating the local model based on the determined gradient; Transmitting the gradient to a server; Receiving information on the updated global model based on the gradient from the server; And updating the local model based on the received information.
  • a method of updating an artificial intelligence model may be provided.
  • a second aspect of the present disclosure includes: a memory storing a local model; Detecting that information about the situation of the electronic device used to update the local model is changed, based on the information about the changed situation, determine a gradient for updating the local model, and based on the determined gradient At least one processor for updating the local model; And a communication unit configured to transmit the gradient to a server and receive information on a global model updated based on the gradient from the server, wherein the at least one processor is configured to generate the local model based on the received information.
  • An electronic device that updates and updates an artificial intelligence model may be provided.
  • a third aspect of the present disclosure may provide a recording medium in which a program for performing the method of the first aspect is stored.
  • the processor may be composed of one or a plurality of processors.
  • one or more processors may be a general-purpose processor such as a CPU, AP, or Digital Signal Processor (DSP), a graphics-only processor such as a GPU, a Vision Processing Unit (VPU), or an artificial intelligence-only processor such as an NPU.
  • DSP Digital Signal Processor
  • GPU Graphics Processing Unit
  • NPU Vision Processing Unit
  • One or more processors control to process input data according to a predefined operation rule or an artificial intelligence model stored in the memory.
  • the dedicated artificial intelligence processor may be designed with a hardware structure specialized for processing a specific artificial intelligence model.
  • a predefined motion rule or an artificial intelligence model is characterized by being created through learning.
  • learning means that a basic artificial intelligence model is learned using a plurality of learning data by a learning algorithm, so that a predefined motion rule or an artificial intelligence model set to perform a desired characteristic (or purpose) is created. Means Jim.
  • Such learning may be performed in the device itself on which the artificial intelligence according to the present disclosure is performed, or may be performed through a separate server and/or system. Examples of the learning algorithm include supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but are not limited to the above-described examples.
  • the artificial intelligence model may be composed of a plurality of neural network layers.
  • Each of the plurality of neural network layers has a plurality of nodes and weight values, and performs a neural network operation through an operation result of a previous layer and a plurality of weights.
  • the plurality of weights of the plurality of neural network layers can be optimized by the learning result of the artificial intelligence model. For example, a plurality of weights may be updated to reduce or minimize a loss value or a cost value acquired from an artificial intelligence model during a learning process. In addition, in order to minimize the loss value or the cost value, a plurality of weights may be updated in a direction to minimize the loss value or the gradient related to the cost value.
  • the artificial neural network may include a deep neural network (DNN), for example, CNN (Convolutional Neural Network), DNN (Deep Neural Network), RNN (Recurrent Neural Network), RBM (Restricted Boltzmann Machine), DBN (Deep Belief Network), BRDNN (Bidirectional Recurrent Deep Neural Network), or deep Q-Networks (Deep Q-Networks), and the like, but is not limited to the above-described example.
  • DNN deep neural network
  • CNN Convolutional Neural Network
  • DNN Deep Neural Network
  • RNN Recurrent Neural Network
  • RBM Restricted Boltzmann Machine
  • DBN Deep Belief Network
  • BRDNN Bidirectional Recurrent Deep Neural Network
  • Deep Q-Networks Deep Q-Networks
  • FIG. 1 is a diagram illustrating a system for performing various operations using a local model of an electronic device according to an embodiment.
  • a system may include an electronic device 1000 and a server 2000.
  • the electronic device 1000 may be configured to output a result suitable for the situation of the electronic device 1000 based on information about the situation of the electronic device 1000 collected by the electronic device 1000. ), and various operations can be performed based on the updated local model.
  • the local model of the electronic device 1000 may be continuously updated in the electronic device 1000 based on information on the situation of the electronic device 1000 and may be used to perform various operations. It is an artificial intelligence model.
  • the electronic device 1000 may update a local model by optimizing a plurality of weights included in a plurality of neural network layers constituting the local model based on information about a situation. For example, the electronic device 1000 may update the local model so that a difference between prediction information that can be output through a local model and the prediction information and corresponding observation information can be minimized based on the information on the situation. I can.
  • the observation information is information indicating a correct answer to the prediction information, and may be determined based on various types of information related to the output result of the local model as well as information on the situation of the electronic device 1000 described above. have.
  • the information about the situation of the electronic device 1000 includes situation information collected by the electronic device 1000, information about the state of the electronic device 1000, information about the user of the electronic device 1000, and information about the electronic device 1000. It may include information about the surrounding environment.
  • the context information collected by the electronic device 1000 may include various pieces of information that can be used to update the local model so that a suitable result according to the current situation of the electronic device 1000 is output.
  • the context information may include information about a state of the electronic device 1000, information about a user of the electronic device 1000, information about a surrounding environment of the electronic device 1000, and the like.
  • observation information corresponding to prediction information that may be output to a local model may be obtained based on the situation information.
  • the electronic device 1000 may update the local model by modifying values constituting the local model so that a difference between the observation information and the prediction information is minimized.
  • the information on the state of the electronic device 1000 may include information on an operation state of the electronic device 1000, information on a network connection state in the electronic device 1000, and an OS installed in the electronic device 1000 (operating system) or information about the state of the application, information about the location of the electronic device 1000, information about the moving state of the electronic device 1000, information about the hardware state of the electronic device 1000, etc.
  • Various information on the state of the device 1000 may be included.
  • Information about a user of the electronic device 1000 is, for example, information about a user's characteristics, information about a user's personal information, information about an account of a service to which the user has subscribed, and It may include various types of user's privacy information, such as information on a pattern.
  • the information on the user is not limited to the above example, and may include various types of information on the user.
  • the information on the surrounding environment of the electronic device 1000 may include information on objects existing around the electronic device 1000, information on illumination of the surrounding environment of the electronic device 1000, etc. 1000) can contain various information about the surrounding environment.
  • operations that may be performed based on the local model may include various operations that may be performed by the electronic device 1000.
  • the operation may include an operation of recommending an item, an operation of recognizing an object around the electronic device 1000, an operation of performing voice recognition on an audio signal collected by the electronic device 1000, and the like.
  • the above-described example is not limited to, and operations that may be performed based on the local model according to an embodiment may include various operations that may be performed by the electronic device 1000 using the local model.
  • the electronic device 1000 not only performs an operation for performing various operations using the local model, but also determines the accuracy of the operation using the local model based on information about the situation of the electronic device 1000. Learning to improve can be performed.
  • the electronic device 1000 may perform learning on a local model, so that the electronic device 1000 may derive an operation result optimized for a usage pattern of the electronic device 1000 by using the learned local model. have.
  • there is at least one local model corresponding to each operation and each operation may be performed according to the at least one local model. For example, the electronic device 1000 acquires at least one local model corresponding to an operation (ex.
  • the operation may be performed.
  • the above-described example is not limited, and a plurality of different operations may be performed according to a single local model.
  • the electronic device 1000 may obtain the at least one local model described above from a memory provided in the electronic device 1000 or may receive it from the outside.
  • the electronic device 1000 may obtain a local model by reading a memory.
  • the electronic device 1000 may receive a global model from the external server 2000 and obtain the received global model as a local model of the electronic device 1000.
  • the embodiment is not limited to the above example, and the electronic device 1000 may obtain the local model through various methods.
  • the electronic device 1000 when a local model for performing a predetermined operation is not stored in the electronic device 1000, the electronic device 1000 stores information on a global model for performing the predetermined operation. Can be received from Also, the electronic device 1000 may acquire and store a local model based on information on the global model received from the server 2000.
  • the electronic device 1000 requests a global model for performing the predetermined operation from the server 2000, receives a global model from the server 2000, and transmits the received global model to the electronic device ( 1000) can be saved as a local model. Thereafter, based on the context information collected by the electronic device 1000, the local model stored in the electronic device 1000 may be continuously updated.
  • the local model stored in the electronic device 1000 may be continuously updated based on various data required for updating the local model as well as the context information collected by the electronic device 1000.
  • the electronic device 1000 is a device that performs various operations using a local model using artificial intelligence technology, and may be implemented in various forms.
  • the electronic device 1000 described in this specification includes a digital camera, a smart phone, a laptop computer, a tablet PC, an electronic book terminal, a digital broadcasting terminal, and personal digital assistants (PDAs). , Portable Multimedia Player (PMP), navigation, MP3 player, artificial intelligence speaker, robot cleaner, home appliance, etc., but the present invention is not limited thereto.
  • PMP Portable Multimedia Player
  • MP3 player Portable Multimedia Player
  • artificial intelligence speaker robot cleaner
  • home appliance etc.
  • the electronic device 1000 described in the present specification may be a device that can be worn by a user.
  • Wearable devices include accessory type devices (e.g. watches, rings, wristbands, ankle bands, necklaces, glasses, contact lenses), head-mounted-devices (HMD), textile or clothing integrated devices (e.g., Electronic clothing), a body-attached device (eg, a skin pad), or a living body implantable device (eg, an implantable circuit), but is not limited thereto.
  • updating of the local model may be performed in two steps.
  • the updating of the local model may include a first step of updating the local model based on information about a situation collected by the electronic device 1000.
  • the step of updating the local model includes a second step of updating the local model again based on the information received from the server 2000 as the gradient for the updated local model is transmitted to the server 2000. I can.
  • the gradient of the local model may include information for updating the local model.
  • the gradient may include information related to a case in which a difference between the above-described observation information and prediction information is minimized.
  • the electronic device 1000 may obtain a loss function representing a difference between observation information and the prediction information, and obtain a gradient including information related to a point at which a value of the loss function is minimum.
  • the minimum value of the loss function can be detected by finding a point at which the slope of the loss function becomes minimum.
  • the slope value of the loss function according to an embodiment may be obtained from a partial derivative value for each variable included in the prediction information.
  • the electronic device 1000 may determine prediction information of a local model corresponding to a point at which a value of a loss function is minimum, using a gradient.
  • the electronic device 1000 modifies parameters (eg, weights, biases, etc. for each node) constituting the local model so that the determined prediction information can be output by the local model. Can be updated.
  • the server 2000 updates the global model stored in the server 2000 based on the gradient transmitted from the electronic device 1000, and transmits information on the updated global model to the electronic device 1000.
  • the server 2000 may update the global model so that prediction information corresponding to slope information of a loss function included in the gradient may be output by the global model.
  • the observation information is information indicating a correct answer to the prediction information of the global model, and may be obtained based on various types of information related to the output result of the global model.
  • the electronic device 1000 may update the local model again based on the updated global model information received from the server 2000.
  • the global model may be updated according to the relatively excellent learning ability and computational ability of the server 2000 based on various information collected by the server 2000.
  • the global model may be continuously learned and stored based on various pieces of information collected by various methods in the server 2000.
  • the global model is not limited to the above example, and the global model is learned through various methods and may be stored in advance in the server 2000, and the global model stored in the server 2000 is updated with a local model according to an embodiment. Can be used.
  • Various information collected by the server 2000 described above may include, for example, information collected by external devices other than the electronic device 1000, information collected on the Internet, and the like. It is not limited to the above example, and various types of information collected by the server 2000 may include various types of information collected by various methods.
  • a global model according to an embodiment may have better performance than a local model in performing an operation according to an embodiment.
  • the electronic device 1000 according to an embodiment may update the local model again based on the global model updated by the server 2000 having more information and having relatively better learning performance.
  • the server 2000 may update the global model by using not only data received from the electronic device 1000 but also data received from other external devices 3000-1 and 3000-2.
  • the global model of the server 2000 according to an embodiment may be updated as information on the situation of the server 2000 changes. Therefore, according to an embodiment, even when data (ex. the gradient of the local model updated by the electronic device 1000 or another external device) is not received from the outside, based on the information on the situation of the server 2000 , The global model of the server 2000 may be updated. Accordingly, the electronic device 1000 according to an embodiment may provide a better service to a user by using a local model updated using information on the global model.
  • the global model updated in the server 2000 considers not only the electronic device 1000 but also context information of other external devices. Considering that it is an updated artificial intelligence model, it can be updated.
  • the local model may be updated so that a result that does not contradict the situation of the electronic device 1000 is output by the local model. Therefore, even if the local model according to an embodiment is updated again based on the information on the global model, a suitable operation can be performed according to individual context information of the electronic device 1000, which is different from that of other external devices. , Can be updated.
  • the electronic device 1000 may prevent the risk of leaking user sensitive information included in the context information to the outside by transmitting the gradient to the server 2000 instead of the context information. Also, as the amount of data transmitted to the server 2000 decreases, the time required for updating the local model may be reduced.
  • the local model may be updated based on current situation information collected by the electronic device 1000 so that a result suitable for the current situation of the electronic device 1000 is output by the local model.
  • the context information collected by the electronic device 1000 may include various pieces of information that can be used to update the local model so that a suitable result according to the current situation of the electronic device 1000 is output.
  • the context information is based on information about the state of the electronic device 1000, information about the user of the electronic device 1000, information about the surrounding environment of the electronic device 1000, and results output by the local model. It may include user feedback information, etc.
  • a user may provide feedback information on an operation performed by a local model.
  • the feedback information may include, for example, information input by a user indicating whether an item recommended based on a local model, an object recognition result, a voice recognition result, and the like are appropriate.
  • the feedback information may further include rating information on the recommended item input by the user.
  • the above-described example is not limited, and the feedback information may include various types of information including a user's opinion on an operation performed by the local model.
  • Feedback information may be collected as context information collected by the electronic device 1000 that can be used to update a local model, and is based on context information of the electronic device 1000 including feedback information.
  • the local model can be updated. For example, when feedback information is obtained from a user, it may be determined that context information has changed, and a local model may be updated according to the feedback information.
  • the situation information collected by the electronic device 1000 is not limited to the above example, and may include various types of situation information that may be used to update the local model.
  • the context information of the electronic device 1000 may be continuously collected by the electronic device 1000. According to an embodiment, the electronic device 1000 may continuously check the status of the electronic device 1000 and detect whether the status information of the electronic device 1000 has changed.
  • the local model may be updated based on the changed information.
  • a difference of a specific type of information about the situation of the electronic device 1000 collected by the electronic device 1000 by a reference value or more It may be changed so that it is changed, or a case in which new type of information about the situation of the electronic device 1000 is obtained as situation information may be included.
  • a new application is installed on the electronic device 1000, a new user using the electronic device 1000 is identified, an electronic device that has not been previously identified This may include a case where a new motion pattern of the device 1000 is identified, a case where a new type of object is recognized around the electronic device 1000, a case where the surrounding environment of the electronic device 1000 is changed, and the like. If it is not limited to the above-described example, and when it is detected that information on the situation of the electronic device 1000 is changed, various types of information about the user or the electronic device 1000 that can be used to update the local model May include cases of change.
  • the case in which the context information of the electronic device 1000 is changed may include a case in which the user's feedback information for an operation provided based on a local model is changed according to an embodiment.
  • the electronic device 1000 when it is detected that information about the situation of the electronic device 1000 is changed, it may be determined whether the electronic device 1000 performs an operation of updating a local model. For example, whether the electronic device 1000 is currently in an idle state, whether the memory space of the electronic device 1000 is sufficient for local model update, whether the battery of the electronic device 1000 is being charged, and the current time is late at night. Based on at least one of whether it is time or not, the local model may be updated.
  • the electronic device 1000 may update the local model based on the changed context information according to various preset conditions.
  • the electronic device 1000 determines that the state of the electronic device 1000 is suitable for updating the local model later. When it is determined, an operation of updating the local model can be performed.
  • the electronic device 1000 may transmit a gradient used to update the local model to the server 2000.
  • the gradient when the local model is updated based on the context information, in the process of deriving the weight and bias information of each node included in the local model, based on the changed context information, at least that can be generated It can contain one value.
  • the gradient may include information related to a point at which a loss function representing a difference between observation information and prediction information of a local model is minimized, based on information about a changed situation.
  • the server 2000 may update a global model provided in the server 2000 according to the gradient.
  • the global model corresponds to the local model of the electronic device 1000, but, unlike the local model, is provided in the server 2000 and is also updated by a gradient received from an external device other than the electronic device 1000. There are differences in what can be done.
  • the server 2000 may transmit the updated global model gradient to the electronic device 1000, and the electronic device 1000 updates the local model again based on the gradient received from the server 2000. can do.
  • the global model includes an artificial intelligence model corresponding to the local model of the electronic device 1000, but the local model and the global model are operations in which each model is updated in a device having different performance and specifications. As this is performed, some data of various types of data, such as weight values and bias values, constituting each model may be different from each other.
  • the global model may be updated based on a gradient received from the electronic device 1000.
  • the server 2000 considers that the gradient is obtained based on the new context information collected by the electronic device 1000, and configures the global model so that the new context information can be applied to the global model through the gradient. Can be updated.
  • the gradient for the updated global model may be transmitted to the electronic device 1000 and used to update the local model of the electronic device 1000.
  • a method of updating the global model and a method of updating the local model again based on a gradient for the updated global model will be described in more detail with reference to FIGS. 13, 16, and 17.
  • the electronic device 1000 after the local model is updated again based on the updated global model information, the electronic device 1000 performs a predetermined operation (eg, item recommendation, according to the finally updated local model). Object recognition, voice recognition) can be performed.
  • a predetermined operation eg, item recommendation, according to the finally updated local model.
  • Object recognition, voice recognition can be performed.
  • the electronic device 1000 may perform an operation according to the local model. For example, after the local model is updated based on information received from the server 2000 according to an embodiment, the electronic device 1000 performs an operation according to the updated local model according to the on-device AI technology. can do. For example, the electronic device 1000 continuously updates the local model according to an embodiment through data transmission and reception with the server 2000, but while the local model is not updated according to an embodiment , Can operate according to on-device AI technology.
  • the electronic device 1000 performs a predetermined operation based on a local model (ex. item recommendation, object recognition, voice recognition) on the electronic device 1000 without transmitting and receiving data with the server 2000. And print the results.
  • the electronic device 1000 may perform an operation using a local model based on various pieces of information collected by the electronic device 1000 in real time without the need to use big data stored in the server 2000. have.
  • the electronic device 1000 may make a self-determination based on the learned local model and various types of data collected by itself according to an embodiment.
  • the electronic device 1000 by the on-device AI technology does not transmit the collected data to the outside, but operates by itself, so there is an advantage in terms of user's personal information protection and data processing speed.
  • the electronic device according to the local model of the electronic device 1000 1000 may operate without connection to the server 2000 using on-device AI technology.
  • the electronic device 1000 is not limited to operating according to the on-device AI technology, and may perform an operation according to an embodiment through data transmission/reception with the server 2000 or an external device.
  • the electronic device 1000 may request the server 2000 to perform the operation instead of performing the operation according to the embodiment using a local model.
  • the server 2000 according to an embodiment may perform an operation according to an embodiment based on a global model according to a received request and transmit the result to the electronic device 1000.
  • the electronic device 1000 may perform an operation according to an embodiment by combining the above-described on-device AI technology and a method through data transmission/reception with the server 2000.
  • the operation through the server 2000 is more advantageous in terms of data processing speed, or data that does not include the user's personal information is transferred to the server 2000.
  • the method through the server 2000 is more advantageous than the on-device AI technology, such as the case of transmission, the operation according to an embodiment may be performed according to the method through the server 2000.
  • FIG. 2 is a diagram illustrating a system for performing various operations using a local model according to an embodiment.
  • the item recommendation system may include an electronic device 1000, a server 2000, an external device 1 (3000-1), and an external device 2 (3000-2). I can.
  • the electronic device 1000 and server 2000 of FIG. 2 correspond to the electronic device 1000 and server 2000 of FIG. 1.
  • the external device 1 (3000-1) and the external device 2 (3000-2) are the same as the electronic device 1000, each of the external device 1 (3000-1) and the external device 2 ( 3000-2), local models 1 and 2 can be used to perform various operations.
  • Each of the local models 1 and 2 stored in each of the external device 1 (3000-1) and the external device 2 (3000-2) is, according to an embodiment, each of the external device 1 (3000-1), and It may be updated based on the different context information collected by the device 2 (3000-2). Accordingly, each of the local models 1 and 2 may be an artificial intelligence model including different data suitable for the situation of each of the external device 1 (3000-1) and the external device 2 (3000-2).
  • users of the external device 1 (3000-1), the external device 2 (3000-2), and the electronic device 1000 may be different from each other.
  • the present invention is not limited thereto, and the external device 1 (3000-1), the external device 2 (3000-2), and the electronic device 1000 may be used by the same user.
  • the server 2000 may be configured from at least one of the external device 1 (3000-1) and the external device 2 (3000-2), and at least one of local models 1 and 2 updated in each device. At least one gradient obtained based on the model may be received.
  • the local model updated in each device may be updated as context information collected by each of the external device 1 (3000-1) and the external device 2 (3000-2) is changed.
  • the server 2000 in the same manner as in the case of updating the global model based on the gradient received from the electronic device 1000 described above, the server 2000 may be configured with external device 1 (3000-1) and external device 2 (3000). According to at least one gradient received from -2), the global model of the server 2000 may be updated.
  • the server 2000 is the electronic device 1000 From, the obtained gradient based on the updated local model may be received.
  • the server 2000 generates a global model updated according to the gradient received from at least one of the external device 1 (3000-1) and the external device 2 (300-2) based on the gradient received from the electronic device 1000, You can update again.
  • the server 2000 may transmit information on the updated global model to the electronic device 1000.
  • the electronic device 1000 may provide a better service to the user using the local model.
  • the server 2000 may update the global model whenever a gradient is received from the electronic device 1000, the external device 1 3000-1, and the external device 2 3000-2.
  • the server 2000 may transmit information on the updated global model to the device that transmitted the gradient to the server 2000.
  • the global model of the server 2000 may be updated as information on the situation of the server 2000 changes. Accordingly, according to an embodiment, even when data (ex. the gradient of the local model updated by the electronic device 1000 or other external devices 3000-1 and 3000-2) is not received from the outside, the server 2000 ), the global model of the server 2000 may be updated.
  • the global model may be continuously updated based on gradients continuously received from a plurality of devices, or based on a change in information about a situation of the server 2000.
  • the global model is not limited to the above example, and may be continuously updated according to various methods based on various types of data.
  • the local model may not be updated for a considerable period of time as the context information used to update the local model is not changed. Therefore, the difference between the local model and the global model may increase.
  • the server 2000 includes a gradient of a current global model and a gradient of the global model most recently transmitted from the server 2000 to the electronic device 1000. It can be determined whether or not there is a difference greater than or equal to the reference value. The determination may be periodically performed by the server 2000.
  • the server 2000 may transmit the gradient for the recently updated global model to the electronic device 1000.
  • the electronic device 1000 may update the local model based on information on the global model received from the server 2000, for example, a gradient of the global model.
  • the electronic device 1000 may receive the gradient for the most recently updated global model from the server 2000 without transmitting the gradient from the electronic device 1000 to the server 2000. have.
  • the electronic device 1000 according to an exemplary embodiment receives the gradient for the most recently updated global model from the server 2000 even if the context information collected by the electronic device 1000 is not changed, so that the global model is Based on the local model update operation can be continuously performed.
  • the server 2000 may transmit a gradient for the currently updated global model to the electronic device 1000 under various conditions. For example, when the server 2000 receives a gradient from at least one external device other than the electronic device 1000 by a reference number or more and updates the global model by a reference number or more, the server 2000 For example, the gradient may be transmitted to the electronic device 1000.
  • FIG. 3 is a diagram illustrating a system for performing object recognition using a local model according to an exemplary embodiment.
  • a system for performing an object recognition operation using a local model may include a server 2000 and at least one electronic device 3001, 3002, and 3003. At least one electronic device 3001, 3002, and 3003 illustrated in FIG. 3 may correspond to the electronic device 1000 illustrated in FIGS. 1 and 2.
  • Each of the electronic devices 3001, 3002, and 3003 may be a home appliance that is placed in a different home environment and can be used by different users. As each of the electronic devices 3001, 3002, and 3003 are placed in different home environments, recognizable objects may be different from each other.
  • Each of the electronic devices 3001, 3002, and 3003 according to an embodiment includes local object recognition models 1, 2, and 3, respectively, for performing object recognition, and each of the electronic devices 3001, 3002, and 3003 Object recognition can be performed using local object recognition models 1, 2, and 3.
  • each of the electronic devices 3001, 3002, and 3003 is based on the information on the surrounding environment of each of the electronic devices 3001, 3002, and 3003, according to the local object recognition models 1, 2, and 3 for object recognition. At least one of the local models can be updated. Accordingly, in the local object recognition models 1, 2, and 3 of each electronic device 3001, 3002, and 3003 according to an embodiment, an object recognition operation suitable for different surrounding environments of each electronic device 3001, 3002, and 3003 is performed. It can be updated so that it can be performed.
  • information on the surrounding environment used to update the local object recognition model includes state information, appearance information, shape information, etc. about objects that can be recognized by each electronic device (3001, 3002, 3003). can do.
  • the information on the surrounding environment is not limited to the above example, and may include various pieces of information that can be used to update a local object recognition model for object recognition.
  • the context information collected in relation to the local object recognition model for object recognition among the context information collected by the electronic device 1000 is Various information related to object recognition may be included, such as information and information on an object newly recognized by the electronic device 1000 among objects existing around the electronic device 1000.
  • the server 2000 recognizes the global object according to the gradient of the updated local object recognition model in each electronic device 3001, 3002, and 3003. You can update the model.
  • FIG. 4 is a diagram illustrating a system for performing speech recognition using a local model according to an embodiment.
  • a system for performing an operation for speech recognition using a local model may include a server 2000 and at least one electronic device 4001, 4002, and 4003. At least one electronic device 4001, 4002, and 4003 illustrated in FIG. 3 may correspond to the electronic device 1000 illustrated in FIGS. 1 and 2.
  • the server 2000 may include a global speech processing model that can be used to perform an operation for speech recognition among global models.
  • each electronic device 4001, 4002, 4003 according to an embodiment is a local model that can be used to perform an operation for speech recognition, and may include local speech processing models 1, 2, and 3, respectively. have.
  • Each of the electronic devices 4001, 4002, and 4003 may have different surrounding environments and may be electronic devices that may be used by different users. As each of the electronic devices 4001, 4002, and 4003 are placed in different surrounding environments, speech recognition may be performed based on noise information of different surrounding environments.
  • information collected in relation to a local model for voice recognition is information about noise generated in the surrounding environment of the electronic device 1000 .
  • Various information related to speech recognition may be included, such as information about a user's voice that can be recognized by the device 1000.
  • the electronic devices 4001, 4002, and 4003 may be located in a house, a company, or a bus, respectively. As the electronic devices 4001, 4002, and 4003 are placed in different surrounding environments, noise characteristics included in an audio signal to be recognized for speech may be different from each other.
  • each of the electronic devices 4001, 4002, and 4003 may more effectively perform speech recognition according to a local speech processing model by using noise information of a surrounding environment.
  • each electronic device 4001, 4002, 4003 recognizes the user's voice included in the audio signal based on the voice characteristic of the user who wants to perform speech recognition and the noise characteristic of the surrounding environment. Can be performed more effectively.
  • each electronic device 4001, 4002, 4003 acquires relationship information between the user's voice and at least one type of noise, and based on the obtained information, speech recognition is performed.
  • the local speech processing model can be updated so that it can be performed effectively.
  • each of the electronic devices 4001, 4002, and 4003 may update a local model for speech recognition according to information about noise in the surrounding environment of each of the electronic devices 4001, 4002, and 4003. . Accordingly, in the local speech processing model of each electronic device 4001, 4002, 4003 according to an embodiment, a speech recognition operation suitable for noise in different surrounding environments of each electronic device 4001, 4002, 4003 can be performed. So that it can be updated.
  • the information on noise of the surrounding environment used to update the local speech processing model is information on the characteristic of at least one type of noise that can be detected by each electronic device 4001, 4002, 4003, Relationship information between the user's voice and at least one type of noise may be included.
  • the above-described example is not limited, and the information on the noise of the surrounding environment may include various types of information that can be used to update a local speech processing model for speech recognition.
  • the local voice processing model according to the embodiment may be updated. For example, when a new type of noise appears, a case where new relationship information is acquired among the relationship information between the user's voice and at least one type of noise, etc., it may be determined that the situation information according to an embodiment is changed. I can.
  • the server 2000 updates the global speech processing model according to the updated local model in each electronic device 4001, 4002, and 4003. can do.
  • FIG. 5 is a diagram illustrating an example in which a local model and a global model are updated according to an embodiment.
  • the electronic device 1000 may include a local model 501, and the server 2000 may include a global model 502.
  • the horizontal arrow relates to an embodiment of performing an operation through each artificial intelligence model
  • the vertical arrow is an embodiment of updating each artificial intelligence model and Related.
  • the electronic device 1000 updates the local model 501 based on various information for updating the local model 501 collected by the electronic device 1000, and the updated local model 501 The gradient can be obtained based on.
  • the information for updating the local model 501 may include changed context information on the electronic device 1000.
  • the update information may include changed context information based on feedback information on a result of an operation performed by the local model 501.
  • the server 2000 updates the global model 502 based on information for updating the global model 502 collected by the server 2000, and the updated global model 502 You can obtain a gradient for.
  • information for updating the global model 502 is obtained as the server 2000 updates a local model stored in the electronic device 1000 or another external device through the network 100. It can be obtained by receiving the gradient.
  • the server 2000 may update the global model 502 based on the received gradient.
  • the server 2000 may transmit the updated gradient of the global model 502 to the electronic device 1000 or another external device that has transmitted the gradient of the local model to the server 2000.
  • the local model 501 and the global model 502 may output output information corresponding to the input information as input information is respectively input.
  • the electronic device 1000 when operating according to an on-device method, may perform an item recommendation operation, an object recognition operation, a voice recognition operation, or the like according to the local model 501.
  • the input information input to the local model 501 may include information necessary to perform the above-described item recommendation operation, object recognition operation, and voice recognition operation.
  • the input information on the item recommendation operation may include information on the type of an item to be recommended.
  • the input information on the object recognition operation may include sensor information (eg, image information, infrared detection information, etc.) currently detected by the electronic device 1000.
  • the input information on the speech recognition operation may include information on an audio signal currently received by the electronic device 1000.
  • the above-described input information may include context information used to update the local model. The input information is not limited to the above example, and the input information may include various pieces of information for performing an operation in the electronic device 1000.
  • the output information output from the local model 501 may include information on a result of performing the above-described item recommendation operation, object recognition operation, and voice recognition operation.
  • the output information on the item recommendation operation may include information on the recommended item and ranking information on the recommended item.
  • the output information on the object recognition operation may include identification information, status information, and the like about an object recognized based on sensor information.
  • the output information for the speech recognition operation may include text information obtained as a result of speech recognition of an audio signal input as input information.
  • the output information is not limited to the above-described example, and the output information may include various pieces of information indicating a result of performing an operation according to the local model 501 in the electronic device 1000.
  • the server 2000 when the electronic device 1000 operates according to a cloud method rather than an on-device method, the server 2000 performs various operations using the global model 502 according to the request of the electronic device 1000. , For example, an item recommendation operation, an object recognition operation, a voice recognition operation, and the like may be performed.
  • the above-described operation is performed while transmitting input information to be input to the local model 501 to the server 2000 instead of inputting to the local model 501 It is possible to request the server 2000 to perform.
  • the server 2000 may perform an operation using the global model 502 according to the request of the electronic device 1000 and transmit the result to the electronic device 1000.
  • FIG. 6 is a diagram illustrating an example of a local model or a global model according to an embodiment.
  • a local model or a global model may be configured as a neural network model that mimics a manner in which the human brain recognizes patterns.
  • the electronic device 1000 or the server 2000 may provide various services to a user using a local model or a global model composed of a neural network model.
  • the neural network model is a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a bidirectional recurrent deep neural network (BRDNN), and a limited Boltzmann. It may be one of a Restricted Boltzmann Machine (RBM), a Deep Belief Network (DBN), or a Deep Q-Networks. Further, the present invention is not limited to the above example, and the local model or the global model according to an embodiment may be one of various types of neural networks.
  • a neural network model may include at least one layer including at least one node.
  • the neural network model may include layer 1 as an input layer and layer 2 as an output layer.
  • the neural network model according to an embodiment may further include at least one hidden layer (not shown) between the input layer and the output layer.
  • a neural network including an input layer and an output layer excluding a hidden layer will be described as an example.
  • At least one input value may be input to layer 1 of the neural network model.
  • values of I1 and I2 may be input to the nodes N11 and N12 of layer 1, respectively.
  • nodes N11, N12, N21, and N22 included in layers 1 and 2 of the neural network model may be processed.
  • a value output from the node of each layer may be used as an input value of the next layer.
  • a predetermined value may be input to the nodes N21 and N22 of the layer 2 based on the values obtained as the nodes N11 and N12 of the layer 1 are processed.
  • the value output from layer 2 may be output as an output value in the neural network model.
  • values O1 and O2 output from layer 2 may be output as output values of the neural network model.
  • the electronic device 1000 may input a value of input information into an input layer of a local model. Also, the electronic device 1000 may provide various services (eg, item recommendation, object recognition, and voice recognition) to a user based on a value output from the output layer of the local model.
  • various services eg, item recommendation, object recognition, and voice recognition
  • At least one edge data may be obtained from one node.
  • Edge data is data that can be obtained by applying at least one weight to one value output from one node.
  • Edge data may be obtained as many as the number of weights applied to one value.
  • a value output from each node of layer 1 may be converted into at least one edge data and input to a node of a next layer.
  • edge data to which different weights W11 and W12 are applied to a value output from node N11 may be input to nodes N21 and N22 of the next layer, respectively.
  • edge data to which different weights W21 and W22 are applied to a value output from node N12 may be input to nodes N21 and N22 of the next layer, respectively.
  • the local model when the local model is updated, at least one weight value applicable to each node may be changed.
  • the gradient according to an embodiment may include information obtained in an operation process for deriving an optimal weight value applied to each node.
  • a loss function or a cost function may be used.
  • the loss value that can be determined through the loss function may be determined based on a difference between prediction information that can be output by a local model and observation information indicating a correct answer to the prediction information.
  • the observation information may be determined as corresponding information of prediction information that can be output by a local model based on the currently collected situation information.
  • the local model may be updated by replacing the weight value W11 of the local model with the weight value W11' when the loss value is minimized.
  • a mean squared error function a cross entropy error function, or the like may be used.
  • the above-described example is not limited, and various types of functions may be used as the loss function.
  • Equation 1 The mean square error function and the cross entropy error function according to an embodiment may be expressed as Equation 1 below.
  • the first is a mean square error function
  • the second is a cross entropy error function.
  • y and t represent values of prediction information and observation information, respectively.
  • the electronic device 1000 minimizes the loss value obtained by the loss function (ex. the weight value of each node of the local model, and Bias value) can be determined.
  • the electronic device 1000 uses a first prediction value among a plurality of prediction values of the prediction information as a variable, and other prediction values as a constant in a graph of a loss function that has a minimum slope.
  • the loss value ie, the result of the loss function
  • the minimum value of the slope may be obtained as the first gradient.
  • the electronic device 1000 may obtain the gradient including the first to nth gradients as the updated local model gradient.
  • the electronic device 1000 may update the local model based on a parameter corresponding to at least one predicted value.
  • the gradient may include various types of values obtained in a process of deriving various types of information constituting the updated local model according to the update of the local model.
  • the gradient may include at least one weight value that can be applied to nodes constituting the updated local model, and difference values between weight values of the local model before the update.
  • the gradient of the global model may also be obtained, like the gradient of the local model described above.
  • prediction information that can be obtained by the updated global model and slope information of a loss function corresponding to the updated global model may be obtained as a gradient of the updated global model.
  • the gradient of the global model may include at least one weight value applicable to nodes constituting the updated global model and difference values between weight values of the global model before the update.
  • FIG. 7 is a diagram for describing an operation of the electronic device 1000 according to an exemplary embodiment.
  • the electronic device 1000 may perform an operation of recommending an item using the local model 701 and the context information 702.
  • a predetermined operation (ex. item recommendation, object recognition, voice recognition) may be performed in the electronic device 1000 by the local model 701, and based on the result of the performed operation, the situation Information 702 can be changed.
  • the context information 702 may be changed based on the user's feedback information on the result of the above-described operation.
  • the user's feedback information on the operation according to the local model 701 is changed context information 702 and may be used to update the local model 701.
  • the above-described predetermined operation (ex. item recommendation, object recognition, voice recognition) may be performed, but the context information ( When 702 is changed, the local model 701 may be updated based on the changed context information 702.
  • the operation is performed according to the local model 701
  • the context information 702 is changed based on the feedback information generated according to the result of the operation execution, and the changed context information 702 is converted to the local model 701 It is assumed that one step has been performed in the operation transmitted to the local model 701 as information for updating ).
  • At least one of updating the local model 701 and an operation performed by the local model 701 may be performed whenever one step is performed.
  • a gradient may be generated based on the finally updated local model 701 after repeatedly performing a predetermined number of steps.
  • the electronic device 1000 may receive information on the global model by transmitting the generated gradient to the server 2000 and update the local model 701 again based on the received information. have.
  • the gradient of a local model or a global model may further include information obtained based on information about a policy and a value.
  • the information on the policy of the local model may include information on which operation can be selected as an output value according to a value input to the local model.
  • the information on the policy ( ⁇ ) may include information for indicating at least one of a weight value and a bias value for each node described above.
  • information about the policy May be expressed as, a is an operation as an output value of the local model, s is a state information as an input value of the local model, t is time, and ⁇ is a parameter.
  • the information on the value of the local model may include information on the value indicating how appropriate the information on the policy is to perform an operation according to the local model.
  • the information on the value V may be expressed as V(S t , ⁇ v ).
  • information (L p ) on data about a policy of a local model and information (L v ) about data about a value may be obtained according to Equations 2 and 4 below, respectively.
  • Equation 2 H( ⁇ ( ⁇ )) denotes an entropy function for a policy that improves model exploration, and ⁇ denotes a parameter for adjusting the strength of an entropy regularization term.
  • Equation 2 A(s,a; ⁇ , ⁇ v ) of Equation 2 may be obtained according to Equation 2 below.
  • Equation 3 k represents the total number of times the local model has been updated, and ⁇ represents a discount factor.
  • Equation 4 R represents user feedback information.
  • the gradient may include various types of information that can be used to determine a value related to at least one node constituting the local model.
  • FIG. 8 is a diagram illustrating an example in which a local model is updated according to the passage of time according to an embodiment.
  • the electronic device 1000 performs an operation by inputting context information 801 into the local model 802, and obtains user feedback information 803 for the performed operation.
  • the context information 801 of the electronic device 1000 is input information, and is input to the local model 802 to perform a predetermined operation.
  • the above-described example is not limited, and various information required to perform an operation as well as the context information 801 may be input to the local model 802.
  • the context information collected in relation to the first artificial intelligence model for item recommendation is at least one that can be recommended by the first artificial intelligence model. It may include various information related to item recommendation, such as information on an item of, and information on an interaction between a user and at least one item.
  • Information on at least one item that can be recommended by the local model may include, for example, an application, a video, a still image, a text file, a web page, and the like.
  • the item may include link information or index information connected to an image of the item.
  • the item may be configured in the form of an object for service use.
  • An object for using a service refers to a user interface for using a service of a service provider.
  • the object for use of the service includes a user interface including icon, text, image, and link information, and description information on the function of the object, for using a service provided from a content provider. I can.
  • Information on the interaction between the user and at least one item is, for example, information on the history of using at least one item by the user, feedback information from the user on the item recommended by the local model, and at least one by the user. It may include information about a method, a time, a place, etc. in which the item of is used, information about a signal input to at least one item by a user, and the like.
  • the feedback information 803 may be generated based on information output according to an operation performed according to the local model 802.
  • the feedback information 803 may include information input by a user about a result of an operation performed according to the local model 802.
  • the feedback information 803 is output information of the local model 802, which is information about which item of the recommended items is selected, and whether the user has selected a recommended item. It may include information on, ranking information on a recommended item input by a user, and the like.
  • the feedback information 803 is output information of the local model 802 and may include information on the user's opinion on whether the object recognition or speech recognition result is appropriate. have.
  • the context information 804 may be changed based on the changed state 803-1 according to the user's feedback information 803. For example, based on the user's feedback information 803, the electronic device 1000 may change information on an interaction between the user and the item among the context information 804. Also, based on the result of the item recommendation operation, the electronic device 1000 provides information on whether each item is recommended to the user and is finally selected as feature information about each item in the context information 804. You can change it. In addition, the electronic device 1000 may change information on the object recognition or speech recognition result among the context information 804, based on the user's feedback information 803 on the result of performing the object recognition or speech recognition operation. have.
  • the situation information 804 is not limited to the above-described example, and may be changed by various methods based on the state 803-1 changed according to the user's feedback information 803.
  • the local model 805 may be updated based on the feedback information.
  • the electronic device 1000 may update the local model 805 by reflecting the user's feedback information.
  • the electronic device 1000 uses the changed situation information 804 as the local model, similarly to the time point T1.
  • An operation may be performed by inputting it into 805, and user feedback information 806 about the performed operation may be obtained.
  • the change of the context information 804 and the update of the local model 805 may be performed in the same manner as at the time points T1 and T2.
  • the context information 807 may be changed based on the changed state 805-1 according to the changed state 806-1 according to the user's feedback information 806.
  • the electronic device 1000 displays the changed situation information 807 as at the time points T1 and T2. ) Is input to the local model 808 to perform an operation, and user feedback information 809 for the performed operation may be obtained.
  • FIG. 9 is a block diagram illustrating an internal configuration of an electronic device 1000 according to an exemplary embodiment.
  • FIG. 10 is a block diagram illustrating an internal configuration of an electronic device 1000 according to an exemplary embodiment.
  • the electronic device 1000 may include a memory 1700, a processor 1300, and a communication unit 1500. However, not all of the components shown in FIG. 9 are essential components of the electronic device 1000.
  • the electronic device 1000 may be implemented by more components than the components illustrated in FIG. 9, or the electronic device 1000 may be implemented by fewer components than the components illustrated in FIG. 9.
  • the electronic device 1000 includes a user input unit 1100 in addition to the memory 1700, the processor 1300, and the communication unit 1500.
  • An output unit 1200, a sensing unit 1400, and an A/V input unit 1600 may be further included.
  • the user input unit 1100 refers to a means for a user to input data for controlling the electronic device 1000.
  • the user input unit 1100 includes a key pad, a dome switch, a touch pad (contact type capacitance method, pressure type resistive film method, infrared detection method, surface ultrasonic conduction method, integral type). Tension measurement method, piezo effect method, etc.), a jog wheel, a jog switch, and the like, but are not limited thereto.
  • the user input unit 1100 may receive a user input requesting a predetermined operation (eg, item recommendation, object recognition, voice recognition) that may be performed according to a local model. Also, the user input unit 110 may receive a user input including feedback information on an operation performed by the local model of the electronic device 1000.
  • a predetermined operation eg, item recommendation, object recognition, voice recognition
  • the output unit 1200 may output an audio signal, a video signal, or a vibration signal, and the output unit 1200 may include a display unit 1210, an audio output unit 1220, and a vibration motor 1230. have.
  • the output unit 1200 according to an embodiment may output a result of an operation performed by a local model.
  • the display 1210 displays and outputs information processed by the electronic device 1000. According to an embodiment, the display 1210 may display a result of an operation performed by the local model according to a user's input.
  • the display unit 1210 may be used as an input device in addition to an output device.
  • the display unit 1210 includes a liquid crystal display, a thin film transistor-liquid crystal display, an organic light-emitting diode, a flexible display, and a 3D display. It may include at least one of a 3D display and an electrophoretic display.
  • the electronic device 1000 may include two or more display units 1210.
  • the sound output unit 1220 outputs audio data received from the communication unit 1500 or stored in the memory 1700.
  • the sound output unit 1220 may output audio data indicating a result of an operation performed by the local model.
  • the vibration motor 1230 may output a vibration signal.
  • the vibration motor 1230 may output a vibration signal when a touch is input to the touch screen.
  • the vibration motor 1230 according to an embodiment may output a vibration signal representing a result of an operation performed by the local model.
  • the processor 1300 generally controls the overall operation of the electronic device 1000.
  • the processor 1300 executes programs stored in the memory 1700, so that the user input unit 1100, the output unit 1200, the sensing unit 1400, the communication unit 1500, the A/V input unit 1600 ) And so on.
  • the electronic device 1000 may include at least one processor 1300.
  • the processor 1300 may be configured to process instructions of a computer program by performing basic arithmetic, logic, and input/output operations.
  • the command may be provided to the processor 1300 from the memory 1700 or may be received through the communication unit 1500 and provided to the processor 1300.
  • the processor 1300 may be configured to execute an instruction according to a program code stored in a recording device such as a memory.
  • the at least one processor 1300 may detect that information about the situation of the electronic device 1000 used to update the local model is changed.
  • the information about the situation is information about the situation of the electronic device 1000.
  • the situation information includes information about the state of the electronic device 1000, information about the user of the electronic device 1000, and Information about the surrounding environment of the device 1000 may be included.
  • At least one processor 1300 may generate a gradient for updating the local model based on the changed context information. At least one processor 1300 according to an embodiment may update a local model using a gradient.
  • the at least one processor 1300 may update the local model of the electronic device 1000 again based on information on the global model received as the gradient is transmitted to the server.
  • the information on the global model may be a gradient of the global model generated based on the gradient transmitted from the electronic device 1000.
  • a gradient of the global model may be generated based on the gradient transmitted from the electronic device 1000, and the global model may be updated based on the gradient of the global model.
  • the sensing unit 1400 may detect a state of the electronic device 1000 or a state of an environment surrounding the electronic device 1000 and transmit the sensed information to the processor 1300.
  • context information used to update a local model may be obtained. For example, based on various types of sensing information about the user or surrounding environment sensed by the sensing unit 1400, information about the user or surrounding environment may be obtained from among the context information.
  • the sensing unit 1400 includes a magnetic sensor 1410, an acceleration sensor 1420, a temperature/humidity sensor 1430, an infrared sensor 1440, a gyroscope sensor 1450, and a position sensor.
  • a position sensor e.g, GPS
  • barometric pressure sensor 1470, proximity sensor 1480, and RGB sensor (illuminance sensor) 1490 may include at least one of, but is not limited thereto.
  • the communication unit 1500 may include one or more components that allow the electronic device 1000 to communicate with a server (not shown) or an external device (not shown).
  • the communication unit 1500 may include a short range communication unit 1510, a mobile communication unit 1520, and a broadcast reception unit 1530.
  • the short-range wireless communication unit 1510 includes a Bluetooth communication unit, a Bluetooth Low Energy (BLE) communication unit, a near field communication unit, a WLAN (Wi-Fi) communication unit, a Zigbee communication unit, and an infrared ( IrDA, infrared data association) communication unit, WFD (Wi-Fi Direct) communication unit, UWB (ultra wideband) communication unit, Ant+ communication unit, etc. may be included, but is not limited thereto.
  • the mobile communication unit 1520 transmits and receives a radio signal with at least one of a base station, an external terminal, and a server on a mobile communication network.
  • the wireless signal may include a voice call signal, a video call signal, or various types of data according to transmission/reception of text/multimedia messages.
  • the broadcast receiver 1530 receives a broadcast signal and/or broadcast-related information from outside through a broadcast channel.
  • Broadcast channels may include satellite channels and terrestrial channels.
  • the electronic device 1000 may not include the broadcast receiver 1530.
  • the communication unit 1500 may transmit the gradient generated by the electronic device 1000 to the server 2000, and receive information about the updated global model from the server 2000 based on the transmitted gradient. have.
  • the A/V (Audio/Video) input unit 1600 is for inputting an audio signal or a video signal, and may include a camera 1610 and a microphone 1620.
  • the camera 1610 may obtain an image frame such as a still image or a moving image through an image sensor in a video call mode or a photographing mode.
  • the image captured through the image sensor may be processed through the processor 1300 or a separate image processing unit (not shown).
  • the microphone 1620 receives an external sound signal and processes it as electrical voice data.
  • the electronic device 1000 may obtain context information based on an audio signal or a video signal input received through the audio/video (A/V) input unit 1600. For example, based on the audio signal or video signal input for the user or the surrounding environment, acquired by the A/V (Audio/Video) input unit 1600, information about the user or the surrounding environment is to be acquired. I can.
  • A/V Audio/Video
  • the memory 1700 may store a program for processing and controlling the processor 1300 and may store data input to the electronic device 1000 or output from the electronic device 1000.
  • the memory 1700 may store one or more instructions, and at least one processor 1300 of the electronic device 1000 described above executes the one or more instructions stored in the memory 1700, The operation according to the embodiment may be performed.
  • the memory 1700 may store at least one local model that can be used to perform at least one operation, and context information of the electronic device 1000 that can be used to update the local model. have.
  • the memory 1700 is a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (for example, SD or XD memory), and RAM.
  • RAM Random Access Memory
  • SRAM Static Random Access Memory
  • ROM Read-Only Memory
  • EEPROM Electrically Erasable Programmable Read-Only Memory
  • PROM Programmable Read-Only Memory
  • magnetic memory magnetic disk And at least one type of storage medium among optical disks.
  • Programs stored in the memory 1700 may be classified into a plurality of modules according to their functions, for example, a UI module 1710, a touch screen module 1720, a notification module 1730, and the like. .
  • the UI module 1710 may provide a specialized UI, GUI, etc. that are interlocked with the electronic device 1000 for each application.
  • the touch screen module 1720 may detect a user's touch gesture on the touch screen and transmit information on the touch gesture to the processor 1300.
  • the touch screen module 1720 according to some embodiments may recognize and analyze a touch code.
  • the touch screen module 1720 may be configured as separate hardware including a controller.
  • Various sensors may be provided inside or near the touch screen to sense a touch or a proximity touch of the touch screen.
  • a sensor for sensing a touch on a touch screen there is a tactile sensor.
  • the tactile sensor refers to a sensor that detects contact with a specific object to the extent that a person feels it or more.
  • the tactile sensor can detect various information such as roughness of a contact surface, hardness of a contact object, and temperature of a contact point.
  • the user's touch gesture may include tap, touch & hold, double tap, drag, pan, flick, drag and drop, swipe, and the like.
  • the notification module 1730 may generate a signal to notify the occurrence of an event of the electronic device 1000.
  • FIG. 11 is a block diagram showing an internal configuration of a server according to an embodiment.
  • the server 2000 may include a memory 2100, a processor 2200, and a communication unit 2300, like components of the electronic device 1000 illustrated in FIG. 9. have.
  • the server 2000 may be implemented by more components than the components shown in FIG. 11, or the server 2000 may be implemented by fewer components than the components shown in FIG. 11.
  • the processor 2200 typically controls the overall operation of the server 2000.
  • the processor 2200 may overall control the server 2000 by executing programs stored in the memory 2100.
  • the server 2000 may include at least one processor 2200.
  • the processor 2200 may be configured to process instructions of a computer program by performing basic arithmetic, logic, and input/output operations.
  • the command may be provided to the processor 2200 from the memory 2100 or may be received through the communication unit 2300 and provided to the processor 2200.
  • the processor 2200 may be configured to execute an instruction according to a program code stored in a recording device such as a memory.
  • At least one processor 2200 may update the global model based on the gradient received from the electronic device 1000. Also, the at least one processor 2200 may generate information on the updated global model and transmit it to the electronic device 1000.
  • the at least one processor 2200 may update the global model based on gradients received from not only the electronic device 1000 but also a plurality of external devices. Accordingly, the global model according to an embodiment may be updated based on not only the gradient of the electronic device 1000 but also the gradients of a plurality of external devices transmitted to the server 2000.
  • the communication unit 2300 may include one or more components that allow the server 2000 to communicate with the electronic device 1000 or an external device.
  • the communication unit 2300 may receive a gradient from the electronic device 1000 and transmit information about an updated global model to the electronic device 1000 based on the received gradient.
  • the memory 2100 may store a program for processing and controlling the processor 2200 and may store data input to the server 2000 or output from the server 2000.
  • the memory 2100 may store one or more instructions, and at least one processor 2200 of the server 2000 described above executes the one or more instructions stored in the memory 2100. You can perform the operation by example.
  • the memory 2100 of the server 2000 may store a global model for performing a predetermined operation (eg, item recommendation, object recognition, voice recognition).
  • a predetermined operation eg, item recommendation, object recognition, voice recognition.
  • FIG. 12 is a flowchart illustrating a method of updating a local model according to an embodiment.
  • the electronic device 1000 may detect that context information of the electronic device 1000 used to update the local model is changed.
  • the context information of the electronic device 1000 may be used to update a local model provided in the electronic device 1000, and is inputted to the local model, whereby actions according to the local model (ex. item recommendation, object recognition, voice) Recognition) may be used to perform.
  • context information related to an usable item in the electronic device 1000 may be changed.
  • context information related to the place of the electronic device 1000 may be changed.
  • context information on the surrounding environment of the electronic device 1000 may be changed.
  • context information on the talker for which voice recognition can be performed by the electronic device 1000 may be changed.
  • the context information may be changed based on the user's feedback information on the result of performing the operation according to the local model.
  • the state information is not limited to the above-described example, and the state information may include information on the situation of various types of electronic devices 1000 that may be used to update the local model or perform an operation according to the local model.
  • the electronic device 1000 may determine a gradient for updating the local model according to the changed context information.
  • the gradient may be obtained based on the changed situation information, based on a loss function indicating a difference between prediction information that can be output to a local model and observation information indicating a correct answer to the prediction information.
  • the gradient may include information related to a point at which the loss function is minimized, for example, slope information of the loss function at the point.
  • the electronic device 1000 may update the local model based on the gradient.
  • the electronic device 1000 may transmit the gradient to the server 2000.
  • the electronic device 1000 may transmit a gradient to the server 2000 instead of state information of the electronic device 1000 that may include sensitive personal information of a user. Accordingly, according to an exemplary embodiment, a risk of leakage in the process of transmitting the user's sensitive personal information may be eliminated.
  • At least one of an operation of adding noise to the gradient and an operation of performing encryption may be additionally performed so that the gradient may be protected during a transmission process.
  • a randomly generated noise value may be added to at least one value representing data modified in the updated artificial intelligence model included in the gradient.
  • the noise value may be a value that is negligibly small compared to the at least one value. Accordingly, the electronic device 1000 or the server 2000 may update a local model or a global model according to an embodiment without identifying or removing the added noise, even if the above-described noise is added to the gradient. The operation can be performed.
  • the gradient may be protected from an external attack or a risk of external leakage.
  • the electronic device 1000 may receive information on a global model as the gradient is transmitted to the server 2000.
  • the global model of the server 2000 may be updated based on the gradient transmitted by the electronic device 1000, and information on the updated global model may be transmitted to the electronic device 1000.
  • the information on the updated global model may be a gradient of the updated global model.
  • the information on the global model received in operation 1205 may be a gradient of the global model including information constituting the global model.
  • the electronic device 1000 obtains the gradient of the updated global model of the server 2000 based on the gradient of the last updated global model, and in step 1206, updates the local model based on the obtained gradient of the global model. can do.
  • the gradient of the global model may be obtained based on difference values of weight values of the global model of the server 2000 in a fixed time interval.
  • the fixed time period may be set based on a time period in which communication between the electronic device 1000 and the server 2000 is performed.
  • the fixed time period may be set as a period between the most recent time point in which communication between the electronic device 1000 and the server 2000 is performed and a time point before it.
  • the above-described example is not limited, and the gradient of the global model may include various types of information used to update the global model.
  • the electronic device 1000 may perform an operation according to the updated local model based on the gradient of the global model.
  • FIG. 13 is a flowchart illustrating a method of updating a global model in a server according to an embodiment.
  • the server 2000 may receive a gradient from the electronic device 1000.
  • the electronic device 1000 may update the local model based on a change in information on the situation of the electronic device 1000, which is used to update the local model of the electronic device 1000.
  • the electronic device 1000 may transmit a gradient including information configuring the updated local model to the server 2000.
  • the server 2000 may receive the gradient from the electronic device 1000.
  • the server 2000 may update the global model based on the gradient.
  • the global model according to an embodiment is a model for performing a predetermined operation (ex. item recommendation, object recognition, voice recognition) corresponding to a local model, and based on a gradient received from the electronic device 1000, Can be updated.
  • the server 2000 may update the global model by updating various pieces of information constituting the global model based on the gradient.
  • the server 2000 may update the global model by updating a weight value and a bias value for each node constituting the global model.
  • the server 2000 may generate information on the updated global model, for example, a gradient, and in operation 1304, the server 2000 may transmit information on the generated global model to the electronic device 1000.
  • the electronic device 1000 may update the local model of the electronic device 1000 based on information on the global model received from the server 2000.
  • the electronic device 1000 may perform a predetermined operation (eg, item recommendation, object recognition, voice recognition) using the updated local model.
  • a predetermined operation eg, item recommendation, object recognition, voice recognition
  • the gradient of the current global model stored in the server 2000 is a first gradient
  • the gradient of the local model transmitted from the electronic device 1000 to the server 2000 is a second gradient.
  • a gradient for updating a current global model that can be obtained will be referred to as a third gradient.
  • the server 2000 obtains a first gradient for a current global model, and based on the first gradient and a second gradient received from the electronic device 1000, the current global model It is possible to obtain a third gradient for updating. Based on the third gradient, the current global model may be updated, and as the third gradient is transmitted to the electronic device 1000 as information on the updated global model, the local model of the electronic device 1000 is updated again. Can be.
  • At least one representative value for information corresponding to each other between the first gradient for the updated global model and the second gradient for the updated local model may be obtained.
  • the third gradient according to an embodiment may be composed of the obtained at least one representative value.
  • the representative value may be, for example, an arithmetic average value or a weighted average value between gradient values included in the first gradient and the second gradient, respectively.
  • the representative value may be obtained based on a value obtained by simply adding gradient values included in the first gradient and the second gradient, respectively.
  • the first gradient may include gradient information of the loss function of the global model
  • the second gradient may include gradient information of the loss function of the global model.
  • the slope information may include values obtained by partial differentiation of a loss function by using each prediction value included in the prediction information as a variable.
  • the gradient information may include values corresponding to points at which the value of the loss function is minimized.
  • a representative value of the gradient information of the global model and the gradient information of the local model may be obtained as a representative value of the third gradient.
  • an arithmetic mean value (ex. (a+b)/2) or a weighted average value (ex.(a*a1) of the first gradient (a) of the global model and the second gradient (b) of the local model) +b*b1)/2) may be obtained as a representative value of the third gradient.
  • the global model may be updated based on the second gradient received from the electronic device 1000.
  • a weight value applied to each value updates the global model, and may be determined as the most suitable value.
  • the weight value may be determined as a value most suitable for updating the global model according to a result of machine learning performed to determine the weight value.
  • the weight of the weighted average value may be determined according to whether or not it is a gradient with respect to a training result to which a greater weight is desired among the training result of the global model and the training result of the local model.
  • the server 2000 may obtain a third gradient used to update the global model as information about the updated global model.
  • the information on the updated global model generated by the server 2000 may include information necessary for updating the local model of the electronic device 1000.
  • the local model of the electronic device 1000 may be updated based on the updated global model information (eg, a third gradient) received from the server 2000.
  • the local model may be updated according to the changed context information, and then updated based on the third gradient received from the server 2000 and the second gradient of the local model. For example, based on the third gradient received from the server 2000 and the second gradient of the local model, a fourth gradient for updating the local model may be obtained. Based on the fourth gradient, an update operation of the local model by receiving the third gradient may be performed.
  • At least one representative value for information corresponding to each other between information included in the third gradient and the second gradient may be obtained.
  • the fourth gradient according to an embodiment may be composed of the obtained at least one representative value.
  • the representative value may be, for example, an arithmetic average value or a weighted average value between values included in the third gradient and the second gradient.
  • the third gradient may include information related to the loss function of the updated global model
  • the second gradient may include information related to the loss function of the local model.
  • a representative value eg, an arithmetic average value, a weighted average value
  • the first gradient of the updated global model and the second gradient of the local model may be obtained as a representative value of the fourth gradient.
  • the local model may be updated based on the third gradient received from the electronic device 1000 based on the fourth gradient including at least one representative value described above. For example, information related to the loss function of the local model is replaced based on representative values included in the fourth gradient, so that the local model may be updated.
  • Weight values (ex. c1, d1) applied to each value may be determined as the most suitable value for updating the local model.
  • the weight value may be determined as a value most suitable for updating the local model according to a result of machine learning performed to determine the weight value.
  • the operation by the local model updated according to the weight value does not contradict the state of the electronic device 1000 or the surrounding situation of the electronic device 1000. It can be performed so that a weight value that can be properly performed can be determined.
  • the weight value may be learned based on information about the situation of the electronic device 1000. The above-described example is not limited, and each weight value applied to each value may be determined as a suitable value according to various methods.
  • the information on the updated global model is not limited to the above-described third gradient, and may include various types of information necessary to update the local model based on the recently updated global model. .
  • FIG. 14 is a diagram illustrating a system for updating a local model and a global model according to an embodiment.
  • a system includes an electronic device 1000, a first server 2000-1, a second server 2000-2, an external device 1 3000-1, and It may include an external device 2 (3000-2).
  • the electronic device 1000, the first server 2000-1, and the second server 2000-2 of FIG. 14 correspond to the electronic device 1000 and the server 2000 of FIG. 1.
  • external device 1 (3000-1) and external device 2 (3000-2) of FIG. 13 correspond to external device 1 (3000-1) and external device 2 (3000-2) of FIG.
  • the external device 1 (3000-1) and the external device 2 (3000-2) according to an embodiment are the same as the electronic device 1000, each of the external device 1 (3000-1) and the external device 2 ( By using the local model provided in 3000-2), various services can be provided.
  • the electronic device 1000 identifies the first server 2000-1 and the second server 2000-2 that can receive information on the global model by transmitting the gradient of the local model. can do.
  • the electronic device 1000 receives information on the first server 2000-1 and the second server 2000-2 from the outside in advance, so that the first server 2000-1 and the second server 2000-2 ) Can be identified.
  • the electronic device 1000 may determine a suitable server based on information on each of the identified first and second servers 2000-1 and 2000-2.
  • the electronic device 1000 may transmit the gradient to the determined server and receive information on the updated global model based on the gradient.
  • the electronic device 1000 may determine a server whose physical location is close to the electronic device 1000 among the identified first server 2000-1 and second server 2000-2 as a server to transmit the gradient. I can.
  • the electronic device 1000 transmits a gradient to a server including a global model having the same or similar characteristics as the characteristics of the local model among the identified first server 2000-1 and the second server 2000-2. It can be decided by the server.
  • the global model of the server is suitable for performing various operations such as an operation of recommending an item under a specific condition, an operation of performing object recognition, and an operation of performing voice processing, depending on the location of the server or various characteristics.
  • the global model of a server located in the UK may be a model created in consideration of cultural characteristics of the UK, and may be suitably used for the electronic device 1000 located in the UK to perform an operation of recommending an item. have.
  • the electronic device 1000 updates the local model with the global model of the first server 2000-1 among the first server 2000-1 and the second server 2000-2. If it is determined that it is appropriate, the gradient may be transmitted to the first server 2000-1.
  • the external device 1 (3000-1), similarly to the electronic device 1000, determines that it is appropriate to update the local model to the global model of the first server (2000-1). ) To send the gradient.
  • the external device 2 (3000-2) may transmit the gradient to the second server (2000-2) as it determines that it is appropriate to update the local model to the global model of the second server (2000-2). .
  • 15 is a diagram illustrating a system for updating a local model and a global model through an intermediary device according to an embodiment.
  • a system may include an electronic device 1 (1000-2), an electronic device 2 (1000-3), an electronic device 3 (1000-4), and a server 2000. I can.
  • the server 2000 of FIG. 15 corresponds to the electronic device 1000 of FIG. 1.
  • the electronic device 1 (1000-2), the electronic device 2 (1000-3), and the electronic device 3 (1000-4) are respectively )
  • a local model included in the electronic device 3 (1000-4) various services (eg, item recommendation, object recognition, and voice recognition) may be provided to a user.
  • the electronic device 1 (1000-2 ), the electronic device 2 (1000-3) and the electronic device 3 (1000-4) may be classified into one group 1000-1.
  • the electronic device 1 (1000-2), the electronic device 2 (1000-3), and the electronic device 3 (1000-4) are determined as devices used in the user's premises by the same user, and thus one group It can be classified as (1000-1).
  • the electronic device 2 (1000-3) and the electronic device 3 (1000-4) are, among devices in the group 1000-1 to which each electronic device belongs, each electronic device 2 (1000-3) and an electronic device.
  • An intermediary device to transmit the gradient of the local model of the device 3 1000-4 to the server 2000 may be determined.
  • the intermediate device may transmit the updated local model gradient to the server 2000 according to the change of context information. Further, the intermediary device receives the gradient of global information from the server 2000 as it transmits the updated local model gradient, and transmits the received gradient of the global information to the electronic device 2 (1000-3) and the electronic device 3 (1000). -4).
  • the electronic device 2 (1000-3) and the electronic device 3 (1000-4) refer to an intermediary device among the devices in the group 1000-1 to which each electronic device belongs to the electronic device 1 (1000-2). You can decide.
  • the electronic device 1 1000-2 determined as an intermediary device may be a device having better performance or a network environment than the electronic device 2 1000-3 and the electronic device 3 1000-4.
  • the electronic device 1 1000-2 may be a smartphone having better performance than the other electronic devices 1000-3 and 1000-4.
  • Other electronic devices 1000-3 and 1000-4 may be home appliances (eg, robot cleaners, refrigerators, and smart TVs) having lower processing performance than smart phones.
  • the gradient generated by each device through the electronic device 1 (1000-2) is transmitted to the server rather than when the electronic device 2 (1000-3) and the electronic device 3 (1000-4) directly transmit the gradient to the server 2000.
  • the electronic device 2 (1000-3) and the electronic device 3 (1000-4) directly transmit the gradient to the server 2000.
  • information about the updated global model can be obtained more quickly.
  • the electronic device 1 (1000-2) may directly process the gradient received from the electronic device 2 (1000-3) and the electronic device 3 (1000-4) according to the edge computing method.
  • the electronic device 1 (1000-2) is based on the gradient received from the electronic device 2 (1000-3) or the electronic device 3 (1000-4), the local model of the electronic device 1 (1000-2) Is updated, and the gradient for the updated local model may be transmitted to the electronic device 2 (1000-3) or the electronic device 3 (1000-4).
  • the electronic device 1 1000-2 may transmit the updated gradient for the local model of itself to the server 2000 and receive the gradient for the global model from the server 2000.
  • the electronic device 1 1000-2 may update its own local model again by using the received gradient of the global model.
  • the electronic device 1 (1000-2) is based on a gradient received from the electronic device 2 (1000-3) and the electronic device 3 (1000-4) and a local model of the electronic device 1 (1000-2). At least one of the generated gradients may be transmitted to the server 2000.
  • the electronic device 1 (1000-2) includes the electronic device 2 (1000-3) and the electronic device within a preset time.
  • a gradient is received from at least one of the three devices 1000-4, it may be transmitted to the server 2000 together with the gradient of the electronic device 1 1000-2.
  • the server 2000 may update the global model based on at least one gradient received from the electronic device 1 1000-2.
  • the server 2000 may transmit information on the updated global model to the electronic device 1 1000-2.
  • the electronic device 1 (1000-2) may also transmit information on the global model received from the server 2000 to the electronic device 2 (1000-3) and the electronic device 3 (1000-4).
  • Each of the electronic device 1 (1000-2), the electronic device 2 (1000-3) and the electronic device 3 (1000-4) is based on the information on the global model received from the server 2000, a local model in each device. Can be updated.
  • 16 is a flowchart illustrating a method of updating a global model and a local model according to a gradient according to an embodiment.
  • the global model and the local model may be updated based on gradients for the global model and the local model.
  • the gradient of the current global model stored in the server 2000 is a first gradient
  • the local model transmitted from the electronic device 1000 to the server 2000 is the same as described in the description of FIG. 1.
  • the gradient for updating the current global model that can be obtained will be referred to as a third gradient.
  • the electronic device 1000 may update the local model as it detects that the context information of the electronic device 1000 is changed.
  • the electronic device 1000 may generate a second gradient for the updated local model.
  • the second gradient according to an embodiment is information used to update the local model, for example, information obtained in relation to a loss function in order to derive a weight value applicable to each node and a bias value (ex.
  • the slope value associated with the point at which the value of the loss function becomes minimum) can be included.
  • the electronic device 1000 may transmit a second gradient for the updated local model to the server 2000.
  • the server 2000 may update the global model based on the second gradient received from the electronic device 1000.
  • the server 2000 may obtain a first gradient for the global model currently stored in the server 2000. Then, in step 1604, the server 2000 may generate a third gradient based on the first gradient and the second gradient.
  • the third gradient may include at least one representative value of information corresponding to each other between information related to the loss function included in the first gradient and information related to the loss function included in the second gradient.
  • the representative value according to an embodiment may be obtained based on an arithmetic average value, a weighted average value, or a simple addition value between values included in each gradient.
  • the server 2000 may transmit the third gradient to the electronic device 1000. Also, in step 1607, the server 2000 may update the global model according to the third gradient. For example, a weight value and a bias value that can be applied to each node constituting the global model are modified based on information related to a loss function included in the third gradient, so that the global model may be updated.
  • the electronic device 1000 may update the local model based on the third gradient received from the server 2000.
  • the updating of the local model in step 1606 may be performed in the same manner as the operation of generating the third gradient in step 1604 and updating the global model according to the third gradient generated in step 1607.
  • a weight value and a bias value that can be applied to each node constituting the local model may be at least one representative value of information corresponding to each other between information included in the second gradient and the information included in the third gradient.
  • the local model may be updated by modifying the corresponding prediction information to be output by the local model.
  • the at least one representative value may be an arithmetic average value and a weighted average value between values included in the second gradient and the third gradient.
  • the weight applied to each value in the weighted average value may be determined as a learned value so that the operation according to the local model is suitable for the current situation of the electronic device 1000. For example, among weights applied to each information in the weighted average value, a weight applied to a value included in a gradient for a local model may be determined based on the context information of the electronic device 1000.
  • 17 is a flowchart illustrating a method of updating a global model, a first local model of the electronic device 1000, and a second local model of the external device 3000 according to a gradient, according to an exemplary embodiment.
  • the electronic device 1000 may update the first local model as the context information of the electronic device 1000 changes, and the external device 3000 changes the context information of the external device 3000.
  • the second local model may be updated.
  • the server 2000 of FIG. 17 may update the global model based on the gradient of the updated local model received from a plurality of devices. In addition, the server 2000 may transmit the updated global model gradient to each device that has transmitted the local model gradient to the server 2000.
  • the gradient for the current global model stored in the server 2000 is transmitted to the server 2000 from the external device 3000 in the same manner as described in the description of FIGS. 1 and 16.
  • the gradient of the local model will be referred to as a third gradient to update the current global model that can be obtained.
  • the gradient of the local model transmitted from the electronic device 1000 to the server 2000 will be referred to as a fourth gradient.
  • the gradient of the global model when the local model is updated is the third gradient
  • the gradient for updating the current global model that can be obtained based on the third gradient and the fourth gradient will be referred to as a fifth gradient. do.
  • the second local model of the external device 3000 can be updated. If the second local model is updated based on the changed context information, the external device 3000 may generate a second gradient for the updated local model in operation 1701.
  • the external device 3000 may transmit a second gradient for the updated local model to the server 2000.
  • the server 2000 may update the global model based on the second gradient received from the external device 3000.
  • the server 2000 may obtain a first gradient for the global model currently stored in the server 2000.
  • the first gradient may be information previously generated when the global model is updated and already stored in the server 2000.
  • the server 2000 may generate a third gradient based on the first gradient and the second gradient.
  • the server 2000 may transmit the third gradient to the external device 3000. Also, in step 1707, the server 2000 may update the global model according to the third gradient.
  • the external device 3000 may update the second local model based on the second gradient and the third gradient received from the server 2000.
  • the update of the second local model in step 1706 may be performed in the same manner as the operation of generating the third gradient in step 1704 and updating the global model according to the third gradient generated in step 1707 and the first gradient. .
  • step 1708 the electronic device 1000, in the same manner as the update of the second local model in step 1701, detects that the context information of the electronic device 1000 is changed, the first local model of the electronic device 1000 Can be updated.
  • the electronic device 1000 may generate a fourth gradient for the updated first local model in operation 1708.
  • the electronic device 1000 may transmit a fourth gradient for the updated first local model to the server 2000.
  • the fourth gradient may be received in the server 2000 after the global model is updated according to the third gradient in operation 1707.
  • the server 2000 may re-update the global model updated in operation 1707 based on the fourth gradient received from the electronic device 1000.
  • the global model may be updated based on a plurality of gradients for a local model updated in a plurality of devices.
  • the server 2000 may obtain a third gradient for the current global model.
  • the server 2000 may generate a fifth gradient based on the third gradient and the fourth gradient.
  • the fifth gradient according to an embodiment may be configured with at least one representative value for information corresponding to each other between the third gradient and the fourth gradient.
  • the representative value according to an embodiment may be obtained based on an arithmetic average value, a weighted average value, or a simple addition value between values included in each gradient.
  • the server 2000 may transmit the fifth gradient to the electronic device 1000.
  • the server 2000 may update the global model according to the fifth gradient.
  • the electronic device 1000 includes a fifth gradient and a fifth gradient received from the server 2000. 4
  • the first local model may be updated based on the gradient.
  • the update of the first local model in step 1713 may be performed in the same manner as the update of the second local model in step 1706.
  • FIG. 18 is a flowchart illustrating a method of updating a global model, a first local model of an electronic device, and a second local model of an external device, according to an exemplary embodiment.
  • Each of the steps of FIG. 18 partially corresponds to each of the steps of FIG. 17, but in FIG. 18, unlike the method illustrated in FIG. 17, the first local model, the second local model, and the global model may be updated.
  • the external device 3000 may generate a second gradient for the updated local model in operation 1801.
  • the external device 3000 may transmit a second gradient for the updated local model to the server 2000.
  • the server 2000 may update the global model based on the second gradient received from the external device 3000.
  • the server 2000 may obtain a first gradient for the global model currently stored in the server 2000.
  • the first gradient for the global model may be obtained based on difference values of weight values of the global model in a fixed time interval.
  • the fixed time period may be set as a period between the most recent time point in which communication between the electronic device 1000 or other external devices and the server 2000 is performed for updating a local model or a global model and a time point before it. I can.
  • the server 2000 may transmit the first gradient obtained in step 1803 to the external device 3000.
  • the server 2000 may update the global model based on the second gradient.
  • the server 2000 may generate a third gradient based on the first gradient and the second gradient, and may update the global model according to the third gradient.
  • the third gradient may be composed of at least one representative value for information corresponding to each other between the first gradient and the second gradient.
  • the above-described example is not limited, and the server 2000 may update the global model according to various methods based on the second gradient.
  • the external device 3000 may update the second local model based on the second gradient and the first gradient received from the server 2000.
  • the updating of the second local model in step 1806 may be performed in the same manner as the operation of generating the third gradient in step 1805 and updating the global model according to the generated third gradient and the first gradient.
  • the third gradient used to update the global model or the second local model based on the first gradient and the second gradient is based on a representative value suitable for updating the global model or the second local model, respectively.
  • the representative value suitable for updating the global model is a simple added value
  • the global model is based on a third gradient generated based on a value obtained by simply adding information corresponding to each other between the first gradient and the second gradient. Can be updated.
  • the global model and the second local model may be updated according to various methods, based on information obtained according to various methods, based on the first gradient and the second gradient.
  • step 1807 the electronic device 1000, in the same manner as the update of the second local model in step 1801, detects that the context information of the electronic device 1000 is changed, the first local model of the electronic device 1000 Can be updated.
  • the electronic device 1000 may generate a fourth gradient for the updated first local model in operation 1807.
  • the electronic device 1000 may transmit a fourth gradient for the updated first local model to the server 2000.
  • the fourth gradient may be received in the server 2000 after the global model is updated in step 1805.
  • the server 2000 according to an embodiment may re-update the global model updated in operation 1805 based on the fourth gradient received from the electronic device 1000.
  • the global model may be updated based on a plurality of gradients for a local model updated in a plurality of devices.
  • the server 2000 may obtain a fifth gradient for the current global model.
  • a fifth gradient for the global model at the current time is obtained.
  • I can.
  • the fifth gradient may be obtained based on difference values of weight values of the global model.
  • the fixed time period may be set as a period between the most recent time point in which communication between the electronic device 1000 or other external devices and the server 2000 is performed for updating a local model or a global model and a time point before it. I can.
  • the server 2000 may transmit the fifth gradient obtained in operation 1809 to the electronic device 1000 so that the first local model of the electronic device 1000 may be updated according to the current global model. .
  • the electronic device 1000 may update the first local model based on the fourth gradient and the fifth gradient received from the server 2000.
  • the updating of the first local model in step 1811 may be performed in the same manner as the operation of generating the sixth gradient in step 1812 and updating the global model according to the generated sixth gradient.
  • the server 2000 may update the global model based on the fourth gradient.
  • the server 2000 may generate a sixth gradient based on the fourth gradient and the fifth gradient, and update the global model according to the generated sixth gradient.
  • the sixth gradient according to an embodiment may be configured with at least one representative value for information corresponding to each other between the fourth gradient and the fifth gradient.
  • the representative value according to an embodiment may be obtained based on an arithmetic average value, a weighted average value, or a simple addition value between values included in each gradient.
  • the above-described example is not limited, and the server 2000 may update the global model according to various methods based on the fourth gradient.
  • the sixth gradient used to update the global model or the first local model based on the fourth gradient and the fifth gradient is based on a representative value suitable for updating the global model or the first local model, respectively.
  • a representative value suitable for updating the global model is a simple added value
  • the global model is based on a sixth gradient generated based on a simple added value of information corresponding to each other between the fourth and fifth gradients. Can be updated.
  • the global model and the first local model are not limited to the above-described examples, and may be updated according to various methods, based on information obtained according to various methods, based on the fourth gradient and the fifth gradient.
  • a gradient related to a local model updated based on the situation information is transmitted to the server, thereby increasing the risk of personal information leakage during the transmission process. Can be removed.
  • a gradient having a smaller data size than the situation information instead of the situation information may be transmitted to the server, so that the update of the local model is performed faster. Can be.
  • An embodiment may also be implemented in the form of a recording medium including instructions executable by a computer, such as a program module executed by a computer.
  • Computer-readable media can be any available media that can be accessed by a computer, and includes both volatile and nonvolatile media, removable and non-removable media. Further, the computer-readable medium may include both computer storage media and communication media.
  • Computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data.
  • Communication media typically include computer readable instructions, data structures, or program modules, and include any information delivery media.
  • the “unit” may be a hardware component such as a processor or a circuit, and/or a software component executed by a hardware configuration such as a processor.

Abstract

La présente invention concerne un système d'intelligence artificielle (IA) qui simule des fonctions, telles que la connaissance et le jugement, d'un cerveau humain par l'utilisation d'un algorithme d'apprentissage automatique, tel qu'un apprentissage profond, et par son application. La présente invention concerne un procédé de mise à jour d'un modèle d'apprentissage dans un dispositif électronique, le procédé comprenant les étapes consistant à : détecter que des informations concernant un contexte du dispositif électronique à utiliser pour mettre à jour un modèle local ont été modifiées ; sur la base des informations de contexte modifiées, acquérir un gradient pour la mise à jour du modèle local ; mettre à jour le modèle local sur la base du gradient ; transmettre le gradient à un serveur ; recevoir, du serveur, des informations concernant un modèle global mis à jour sur la base du gradient ; et mettre à jour le modèle local sur la base des informations reçues.
PCT/KR2020/004965 2019-04-12 2020-04-13 Dispositif électronique pour mise à jour d'un modèle d'intelligence artificielle, serveur, et procédé de fonctionnement associé WO2020209693A1 (fr)

Priority Applications (2)

Application Number Priority Date Filing Date Title
EP20788289.5A EP3816917A4 (fr) 2019-04-12 2020-04-13 Dispositif électronique pour mise à jour d'un modèle d'intelligence artificielle, serveur, et procédé de fonctionnement associé
CN202080010910.1A CN113366520A (zh) 2019-04-12 2020-04-13 用于更新人工智能模型的电子设备、服务器及其操作方法

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
KR20190043309 2019-04-12
KR10-2019-0043309 2019-04-12
KR1020190120851A KR20200120469A (ko) 2019-04-12 2019-09-30 인공지능 모델을 갱신하는 전자 장치, 서버 및 그 동작 방법
KR10-2019-0120851 2019-09-30

Publications (1)

Publication Number Publication Date
WO2020209693A1 true WO2020209693A1 (fr) 2020-10-15

Family

ID=72751395

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/KR2020/004965 WO2020209693A1 (fr) 2019-04-12 2020-04-13 Dispositif électronique pour mise à jour d'un modèle d'intelligence artificielle, serveur, et procédé de fonctionnement associé

Country Status (2)

Country Link
KR (1) KR20200120557A (fr)
WO (1) WO2020209693A1 (fr)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112347500A (zh) * 2021-01-11 2021-02-09 腾讯科技（深圳）有限公司 分布式系统的机器学习方法、装置、系统、设备及存储介质

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20220057353A (ko) * 2020-10-29 2022-05-09 삼성전자주식회사 인공지능 모델을 갱신하는 전자 장치 및 그 동작 방법
KR102346142B1 (ko) * 2021-04-16 2022-01-03 주식회사 테스트웍스 선택적 데이터 수집 시스템 및 그 동작 방법

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20150002542A (ko) * 2013-06-28 2015-01-07 삼성전자주식회사 어플리케이션 업데이트 방법 및 장치
KR20170009991A (ko) * 2014-08-26 2017-01-25 구글 인코포레이티드 글로벌 모델로부터의 로컬화된 학습
KR20180037760A (ko) * 2016-10-05 2018-04-13 삼성전자주식회사 배터리 상태 추정 장치 및 방법
KR20180112653A (ko) * 2017-03-30 2018-10-12 삼성전자주식회사 데이터 학습 서버 및 이의 학습 모델 생성 및 이용 방법
JP2018169706A (ja) * 2017-03-29 2018-11-01 マツダ株式会社 車両運転支援システム

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20150002542A (ko) * 2013-06-28 2015-01-07 삼성전자주식회사 어플리케이션 업데이트 방법 및 장치
KR20170009991A (ko) * 2014-08-26 2017-01-25 구글 인코포레이티드 글로벌 모델로부터의 로컬화된 학습
KR20180037760A (ko) * 2016-10-05 2018-04-13 삼성전자주식회사 배터리 상태 추정 장치 및 방법
JP2018169706A (ja) * 2017-03-29 2018-11-01 マツダ株式会社 車両運転支援システム
KR20180112653A (ko) * 2017-03-30 2018-10-12 삼성전자주식회사 데이터 학습 서버 및 이의 학습 모델 생성 및 이용 방법

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP3816917A4 *

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112347500A (zh) * 2021-01-11 2021-02-09 腾讯科技（深圳）有限公司 分布式系统的机器学习方法、装置、系统、设备及存储介质

Also Published As

Publication number Publication date
KR20200120557A (ko) 2020-10-21

Similar Documents

Publication Publication Date Title
WO2020091210A1 (fr) Système et procédé d'intégration de bases de données d'après un graphe de connaissances
WO2016018004A1 (fr) Procédé, appareil et système de fourniture de contenu traduit
WO2020209693A1 (fr) Dispositif électronique pour mise à jour d'un modèle d'intelligence artificielle, serveur, et procédé de fonctionnement associé
WO2018088794A2 (fr) Procédé de correction d'image au moyen d'un dispositif et dispositif associé
WO2019225961A1 (fr) Dispositif électronique permettant de générer une réponse à une entrée vocale à l'aide d'une application, et procédé de fonctionnement associé
WO2019107945A1 (fr) Procédé de fourniture de service sur la base de l'emplacement d'une source sonore et dispositif de reconnaissance vocale associé
WO2019168383A1 (fr) Dispositif électronique
WO2016017970A1 (fr) Procédé et dispositif pour chiffrer ou déchiffrer un contenu
WO2015142023A1 (fr) Procédé et dispositif corporel pour fournir une interface d'entrée virtuelle
WO2019168380A1 (fr) Dispositif électronique
WO2016064250A2 (fr) Dispositif et procédé permettant le remplacement adaptatif de sujets exécutant une tâche
WO2014119884A1 (fr) Procédé et système d'affichage d'objet et procédé et système de fourniture de cet objet
WO2013069936A1 (fr) Appareil électronique et procédé pour le commander
WO2018117685A1 (fr) Système et procédé de fourniture d'une liste à faire d'un utilisateur
WO2016108660A1 (fr) Procédé et dispositif pour commander un dispositif domestique
WO2016047837A1 (fr) Terminal de type montre et procédé de fonctionnement de celui-ci
WO2017052043A1 (fr) Terminal mobile et son procédé de commande
WO2019135621A1 (fr) Dispositif de lecture vidéo et son procédé de commande
WO2013129799A1 (fr) Dispositif électronique et procédé de commande de dispositif électronique
WO2015093667A1 (fr) Dispositif électronique et procédé de commande de dispositif électronique
WO2016089075A1 (fr) Dispositif pouvant être porté et procédé de transmission de message à partir de celui-ci
WO2021045447A1 (fr) Appareil et procédé de fourniture d'un service d'assistant vocal
EP3552163A1 (fr) Système et procédé de fourniture d'une liste à faire d'un utilisateur
WO2015093665A1 (fr) Dispositif électronique et procédé de commande de dispositif électronique
WO2021029457A1 (fr) Serveur d'intelligence artificielle et procédé permettant de fournir des informations à un utilisateur

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20788289

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE