KR20210072687A - Electronic device and controlling method of electronic device - Google Patents

Abstract

Disclosed are an electronic device and a control method thereof. The electronic device of the present invention comprises: a communication unit; a memory for storing first device information on hardware of the electronic device and a hardware suitability identifying module for identifying a neural network model suitable for the hardware of the electronic device; and a processor for controlling the communication unit such that a first signal is transmitted to request information on at least one neural network model included in at least one external device when a user input is received. In accordance with the present invention, the electronic device can identify whether each of at least one neural network model included in the first external device is suitable for the hardware of the electronic device and whether each of the at least one neural network model identified as suitable for the hardware of the electronic device is suitable to replace the neural network model included in the electronic device, based on the first device information on the hardware of the electronic device, second device information on the hardware of the first external device, first model information on at least one neural network model included in the first external device, and second model information on at least one neural network model included in the electronic device. In addition, the electronic device can transmit a request for installation data of the at least one neural network model identified to the first external device and then install a neural network model based on the installation data received from the first external device.

Description

ELECTRONIC DEVICE AND CONTROLLING METHOD OF ELECTRONIC DEVICE

The present disclosure relates to an electronic device and a control method of the electronic device, and more particularly, to an electronic device and a control method of the electronic device for identifying transition suitability of a neural network model included in an external device.

In recent years, user and industry demand for a technology that provides a customized service for each user through a personalized neural network model is increasing. However, on the other hand, the demand for protection of personal data related to the privacy of individual users is also increasing, so there are many cases in which it is limited to collect personal data necessary for personalization of the neural network model and train the neural network model.

Therefore, a technology that enables a neural network model to be used on-device without transmitting personal data to an external server or cloud, etc., and a device to device-to-device direct communication of information about a personalized neural network model in a specific device Device), research on technology for transferring to another device is attracting attention.

On the other hand, when transferring information about a personalized neural network model from a specific device to another device, it may not be appropriate to transfer the neural network model if the types of devices are different or if the hardware specifications are different even if the device types are the same. have. Therefore, when transferring information about a personalized neural network model from a device to another device, it is necessary to perform an identification process for whether it is appropriate to transfer the neural network model between different devices.

In addition, when a user of an electronic device purchases a new electronic device, there is a need for a technology capable of transferring a personalized neural network model from an existing electronic device to a new electronic device in an efficient and reliable manner.

The present disclosure is in accordance with the above-described necessity, and an object of the present disclosure is to provide a neural network model included in an external device to an electronic device based on information on hardware specifications and neural network models of an electronic device and an external device. An object of the present invention is to provide an electronic device capable of identifying whether transition is appropriate and a method for controlling the electronic device.

According to an embodiment of the present disclosure for achieving the above object, an electronic device for identifying transition suitability of a neural network model included in an external device includes a communication unit, first device information for a hardware specification of the electronic device, and When a memory for storing a hardware suitability identification module for identifying a neural network model suitable for hardware of the electronic device and a user input are received, a first signal for requesting information related to at least one neural network model included in at least one external device control the communication unit to transmit, in response to the first signal, from a first external device among the at least one external device through the communication unit, second device information about the hardware specification of the first external device and the Receive a second signal including first model information for at least one neural network model included in a first external device, and identify the hardware suitability of the first device information, the second device information, and the first model information Input to the module, it is identified whether each of the at least one neural network model included in the first external device is suitable for the hardware of the electronic device, and at least one neural network model identified as being suitable for the hardware of the electronic device is installed Controls the communication unit to transmit a third signal including a request for data to the first external device, and in response to the third signal, from the first external device to the hardware of the electronic device through the communication unit a processor for receiving a fourth signal including installation data of at least one neural network model identified as suitable, wherein the processor includes a specification of each of a plurality of hardware components included in the electronic device included in the first external device If the specification of the plurality of hardware components is greater than or equal to the specification, the at least one neural network model included in the first external device is identified as being suitable for the hardware of the electronic device, and at least one of the plurality of hardware configurations included in the electronic device is identified. of the specification included in the first external device If it is less than the specification of the plurality of hardware configurations, one or more neural network models having a hardware requirement lower than the specifications of the plurality of hardware configurations included in the electronic device among the at least one neural network model included in the first external device is the Identifies that it is suitable for the hardware of the electronic device.

Here, the first device information includes information about the specification of the processor, the specification of the memory, and the specification of the data acquisition unit included in the electronic device, and the data acquisition unit included in the electronic device is included in the electronic device. Acquires data input to at least one neural network model, and includes at least one of a camera, a microphone, and a sensor included in the electronic device, and the second device information includes: a specification of a processor included in the first external device; and information on a specification of a memory included in the first external device and a specification of a data acquisition unit included in the first external device, wherein the data acquisition unit included in the first external device includes at least one data acquisition unit included in the first external device Acquires data input to one neural network model, and includes at least one of a camera, a microphone, and a sensor included in the first external device, wherein the first model information includes at least one neural network included in the first external device It may include information about the hardware requirements of each model.

Here, the memory stores second model information on at least one neural network model included in the electronic device and a model suitability identification module for identifying a neural network model suitable for replacing the neural network model included in the electronic device, and When at least one neural network model suitable for the hardware of the electronic device is identified, the processor inputs the first model information and the second model information to the model suitability identification module, and the at least one neural network model that is suitable for the hardware of the electronic device is identified. at least one neural network identified as suitable for replacing each neural network model included in the electronic device, and the third signal being suitable for replacing a neural network model included in the electronic device It may include a request for installation data of a model, and the fourth signal may include installation data of at least one neural network model identified as suitable for replacing a neural network model included in the electronic device.

Here, the first model information further includes information about a service type and a personalization level of each of the at least one neural network model included in the first external device, and the second model information is included in the electronic device It may include information on each service type and personalization level of each of the at least one neural network model.

Here, the processor is configured to be configured to be suitable for a service type of at least one neural network model included in the electronic device and hardware of the electronic device based on information on a service type included in each of the first model information and the second model information. The service type of the at least one neural network model identified as one is compared, and the service type of the first neural network model among the at least one neural network model identified as suitable for the hardware of the electronic device is selected from among the plurality of neural network models included in the electronic device. If the service type of the second neural network model is the same, the personalization level of the first neural network model and the personalization of the second neural network model based on the information on the personalization level included in each of the first model information and the second model information Comparing the levels, if the personalization level of the first neural network model is higher than the personalization level of the second neural network model, it may be identified that the first neural network model is suitable to replace the second neural network model.

Here, the personalization level of the first neural network model is information about the user's history of use of the first external device for the first neural network model and the feedback of the user of the first external device for the first neural network model. is identified based on at least one of information, and the personalization level of the second neural network model is information about a user's history of use of the electronic device for the second neural network model and a user of the electronic device for the second neural network model may be identified based on at least one of information on the feedback of .

Meanwhile, the second signal is received from the first external device through the communication unit as user authentication is completed based on the first user information on the electronic device and the second user information on the first external device, , the first user information includes at least one of account information for a user of the electronic device and identification information for the electronic device, and the second user information includes account information for a user of the first external device and the It may include at least one of identification information on the first external device.

Meanwhile, the electronic device further includes a display, and the processor controls the display to display a user interface including information on at least one neural network model identified as suitable for hardware of the electronic device, and the user interface Receives a user input for selecting one or more neural network models from among the one or more neural network models identified as suitable for the hardware of the electronic device, and the third signal includes a request for installation data of the selected one or more neural network models and the fourth signal may include installation data of the selected one or more neural network models.

According to an embodiment of the present disclosure for achieving the object as described above, a hardware conformance identification module for identifying first device information on a hardware specification of an electronic device and a neural network model suitable for the hardware of the electronic device is stored, When a user input is received, the control method of an electronic device for identifying the transitive suitability of a neural network model included in an external device receives a first signal for requesting information related to at least one neural network model included in at least one external device. transmitting, in response to the first signal, from a first external device among the at least one external device, second device information about a hardware specification of the first external device, and at least one included in the first external device receiving a second signal including first model information for a neural network model of Identifying whether each of the at least one neural network model included in the external device is suitable for the hardware of the electronic device, comprising a request for installation data of the at least one neural network model identified as being suitable for the hardware of the electronic device transmitting a third signal to the first external device, and in response to the third signal, including installation data of at least one neural network model identified as suitable for hardware of the electronic device from the first external device and receiving a fourth signal, wherein the identifying may include, if a specification of each of a plurality of hardware components included in the electronic device is greater than or equal to a specification of a plurality of hardware components included in the first external device, the first identifying the at least one neural network model included in the external device as being suitable for the hardware of the electronic device; and specifying at least one specification among a plurality of hardware configurations included in the electronic device. If it is less than the specification of the hardware configuration, it is included in the first external device and identifying one or more neural network models having a hardware requirement specification lower than a specification of a plurality of hardware components included in the electronic device among the at least one neural network model as being suitable for hardware of the electronic device.

Here, the first device information includes information on a specification of a processor included in the electronic device, a specification of a memory included in the electronic device, and a specification of a data acquisition unit included in the electronic device, and is included in the electronic device The obtained data acquisition unit acquires data input to at least one neural network model included in the electronic device, and includes at least one of a camera, a microphone, and a sensor included in the electronic device, wherein the second device information includes the first Acquire data included in the first external device, including information on a specification of a processor included in the external device, a specification of a memory included in the first external device, and a specification of a data acquisition unit included in the first external device The unit obtains data input to at least one neural network model included in the first external device, and includes at least one of a camera, a microphone, and a sensor included in the first external device, wherein the first model information is 1 may include information on hardware requirements for each of the at least one neural network model included in the external device.

Here, the electronic device further stores second model information on at least one neural network model included in the electronic device and a model suitability identification module for identifying a neural network model suitable for replacing the neural network model included in the electronic device, , when at least one neural network model suitable for hardware of the electronic device is identified, the control method of the electronic device inputs the first model information and the second model information into the model suitability identification module, Further comprising the step of identifying whether each of the at least one neural network model identified as suitable to be suitable for replacing the neural network model included in the electronic device, the third signal is the neural network model included in the electronic device and a request for installation data of at least one neural network model identified as suitable for replacement, wherein the fourth signal includes installation of at least one neural network model identified as suitable for replacing the neural network model included in the electronic device. It may contain data.

Here, the first model information further includes information about a service type and a personalization level of each of the at least one neural network model included in the first external device, and the second model information is included in the electronic device It may include information on each service type and personalization level of each of the at least one neural network model.

Here, the step of identifying at least one neural network model suitable for replacing the neural network model included in the electronic device includes the electronic device based on information on service types included in each of the first model information and the second model information. Comparing a service type of at least one neural network model included in a device with a service type of at least one neural network model identified as suitable for hardware of the electronic device, at least one neural network identified as suitable for hardware of the electronic device If the service type of the first neural network model among the models is the same as the service type of the second neural network model among the plurality of neural network models included in the electronic device, the personalization level included in each of the first model information and the second model information comparing the personalization level of the first neural network model with the personalization level of the second neural network model based on the information about the first neural network model, and if the personalization level of the first neural network model is higher than the personalization level of the second neural network model, the first identifying that a neural network model is suitable to replace the second neural network model.

Here, the personalization level of the first neural network model is information about the user's history of use of the first external device for the first neural network model and the feedback of the user of the first external device for the first neural network model. is identified based on at least one of information, and the personalization level of the second neural network model is information about a user's history of use of the electronic device for the second neural network model and a user of the electronic device for the second neural network model may be identified based on at least one of information on the feedback of .

Meanwhile, the second signal is received from the first external device as user authentication is completed based on the first user information on the electronic device and the second user information on the first external device, and the first signal is received from the first external device. The user information includes at least one of account information for a user of the electronic device and identification information for the electronic device, and the second user information includes account information for a user of the first external device and the first external device. It may include at least one of identification information for.

Meanwhile, the method for controlling the electronic device includes displaying a user interface including information on at least one neural network model identified as being suitable for hardware of the electronic device, and determining that it is suitable for the hardware of the electronic device through the user interface. Receiving a user input for selecting one or more neural network models from among the identified at least one neural network model, wherein the third signal includes a request for installation data of the selected one or more neural network models; The signal may include installation data of the selected one or more neural network models.

According to an embodiment of the present disclosure for achieving the object as described above, a hardware conformance identification module for identifying first device information on a hardware specification of an electronic device and a neural network model suitable for the hardware of the electronic device is stored, A non-transitory computer-readable recording medium including a program for executing a control method of an electronic device for identifying transition suitability of a neural network model included in an external device, wherein the control method of the electronic device is configured by a user When an input is received, transmitting a first signal for requesting information related to at least one neural network model included in at least one external device, in response to the first signal, a second one of the at least one external device 1 Receiving, from an external device, a second signal including second device information on a hardware specification of the first external device and first model information on at least one neural network model included in the first external device; Whether each of the at least one neural network model included in the first external device is suitable for the hardware of the electronic device by inputting the first device information, the second device information, and the first model information into the hardware suitability identification module Transmitting a third signal including a request for installation data of at least one neural network model identified as suitable for hardware of the electronic device to the first external device, and a response to the third signal receiving, from the first external device, a fourth signal including installation data of at least one neural network model identified as suitable for hardware of the electronic device, wherein the identifying includes in the electronic device If the specification of each of the plurality of hardware components is greater than or equal to the specification of the plurality of hardware components included in the first external device, it is identified that the at least one neural network model included in the first external device is suitable for the hardware of the electronic device. and a plurality of hardware included in the electronic device. If the specification of at least one of the configuration is less than the specification of a plurality of hardware configurations included in the first external device, a plurality of hardware configurations included in the electronic device among the at least one neural network model included in the first external device and identifying that one or more neural network models having a hardware requirement specification lower than the specification of the electronic device are suitable for the hardware of the electronic device.

According to an embodiment of the present disclosure for achieving the object as described above, an electronic device for identifying the transitive suitability of a neural network model included in an external device includes a communication unit, and at least one neural network model included in the electronic device. When a memory for storing internal model information and a model suitability identification module for identifying a neural network model suitable for replacing a neural network model included in the electronic device and a user input are received, at least one neural network model included in at least one external device control the communication unit to transmit a first signal for requesting information related to, and in response to the first signal, include in the first external device from a first external device among the at least one external device through the communication unit Receive a second signal including external model information for at least one neural network model that has been used, input the internal model information and the external model information to the model suitability identification module, and at least one included in the first external device Identifies whether each of the neural network models is suitable for replacing the neural network model included in the electronic device, and for installation data of at least one neural network model identified as being suitable for replacing the neural network model included in the electronic device. control the communication unit to transmit a third signal including a request to the first external device, and in response to the third signal, the identified at least one neural network model from the first external device through the communication unit A processor for receiving a fourth signal including installation data, wherein the processor includes at least one neural network included in the electronic device based on information on a service type included in each of the internal model information and the external model information. The service type of the model is compared with the service type of at least one neural network model included in the first external device, and the service type of the first neural network model among the at least one neural network model included in the first external device is determined by the electronic device. A second neural network among a plurality of neural network models included in If it is the same as the service type of the model, comparing the personalization level of the first neural network model and the personalization level of the second neural network model based on the information on the personalization level included in each of the internal model information and the external model information, If the personalization level of the first neural network model is higher than the personalization level of the second neural network model, it may be identified that the first neural network model is suitable to replace the second neural network model.

Here, the personalization level of the first neural network model is information about the user's history of use of the first external device for the first neural network model and the feedback of the user of the first external device for the first neural network model. is identified based on at least one of information, and the personalization level of the second neural network model is information about a user's history of use of the electronic device for the second neural network model and a user of the electronic device for the second neural network model may be identified based on at least one of information on the feedback of .

1 is a flowchart illustrating a control method of an electronic device according to an embodiment of the present disclosure;
2 to 4 are diagrams for specifically explaining the operation of the hardware conformity identification module for performing the control method of the electronic device according to FIG. 1;
5A is a flowchart illustrating a control method of an electronic device according to another embodiment of the present disclosure;
5A is a flowchart illustrating a control method of an electronic device according to another embodiment of the present disclosure;
6 to 8 are diagrams for explaining in detail the operation of the model suitability identification module for performing the control method of the electronic device according to FIGS. 5A and 5B;
9 is a view for explaining a method of controlling an electronic device according to another embodiment of the present disclosure;
10 is a sequence diagram for explaining an embodiment when there are a plurality of external devices according to the present disclosure;
11 is a view for explaining a user interface provided by an electronic device according to an embodiment of the present disclosure;
12 is a view for explaining a user interface provided by a first external device according to an embodiment of the present disclosure;
13 is a diagram illustrating an example of an electronic device, a first external device, and a second external device according to the present disclosure;
14 is a sequence diagram for explaining a process of identifying transition suitability by an electronic device when a neural network model included in a first external device is transferred to a second external device;
15 is a block diagram illustrating in detail an architecture of a software module included in an electronic device according to the present disclosure;
16 is a block diagram illustrating in detail the architecture of a software module included in a first external device according to the present disclosure;
17 is a block diagram schematically illustrating an architecture of a hardware configuration included in an electronic device according to the present disclosure;
18 is a block diagram illustrating in more detail an architecture of a hardware configuration included in an electronic device according to the present disclosure.

Since the present embodiments can apply various transformations and can have various embodiments, specific embodiments are illustrated in the drawings and described in detail in the detailed description. However, this is not intended to limit the scope of the specific embodiments, and should be understood to include various modifications, equivalents, and/or alternatives of the embodiments of the present disclosure. In connection with the description of the drawings, like reference numerals may be used for like components.

In describing the present disclosure, if it is determined that a detailed description of a related known function or configuration may unnecessarily obscure the subject matter of the present disclosure, a detailed description thereof will be omitted.

In addition, the following examples may be modified in various other forms, and the scope of the technical spirit of the present disclosure is not limited to the following examples. Rather, these embodiments are provided to more fully and complete the present disclosure, and to fully convey the technical spirit of the present disclosure to those skilled in the art.

The terms used in the present disclosure are used only to describe specific embodiments, and are not intended to limit the scope of rights. The singular expression includes the plural expression unless the context clearly dictates otherwise.

In the present disclosure, expressions such as “have,” “may have,” “include,” or “may include” indicate the presence of a corresponding characteristic (eg, a numerical value, function, operation, or component such as a part). and does not exclude the presence of additional features.

In this disclosure, expressions such as “A or B,” “at least one of A and/and B,” or “one or more of A or/and B” may include all possible combinations of the items listed together. . For example, "A or B," "at least one of A and B," or "at least one of A or B" means (1) includes at least one A, (2) includes at least one B; Or (3) it may refer to all cases including both at least one A and at least one B.

As used in the present disclosure, expressions such as "first," "second," "first," or "second," may modify various elements, regardless of order and/or importance, and refer to one element. It is used only to distinguish it from other components, and does not limit the components.

A component (eg, a first component) is "coupled with/to (operatively or communicatively)" to another component (eg, a second component) When referring to "connected to", it will be understood that the certain element may be directly connected to the other element or may be connected through another element (eg, a third element).

On the other hand, when it is said that a component (eg, a first component) is "directly connected" or "directly connected" to another component (eg, a second component), the component and the It may be understood that other components (eg, a third component) do not exist between other components.

The expression "configured to (or configured to)" as used in this disclosure depends on the context, for example, "suitable for," "having the capacity to" ," "designed to," "adapted to," "made to," or "capable of." The term “configured (or configured to)” may not necessarily mean only “specifically designed to” in hardware.

Instead, in some circumstances, the expression “a device configured to” may mean that the device is “capable of” with other devices or parts. For example, the phrase “a processor configured (or configured to perform) A, B, and C” refers to a dedicated processor (eg, an embedded processor) for performing the operations, or by executing one or more software programs stored in a memory device. , may mean a generic-purpose processor (eg, a CPU or an application processor) capable of performing corresponding operations.

In an embodiment, a 'module' or 'unit' performs at least one function or operation, and may be implemented as hardware or software, or a combination of hardware and software. In addition, a plurality of 'modules' or a plurality of 'units' may be integrated into at least one module and implemented by at least one processor, except for 'modules' or 'units' that need to be implemented with specific hardware.

Meanwhile, various elements and regions in the drawings are schematically drawn. Accordingly, the technical spirit of the present invention is not limited by the relative size or spacing drawn in the accompanying drawings.

Meanwhile, the electronic device according to various embodiments of the present disclosure may include, for example, at least one of a smart phone, a tablet PC, a desktop PC, a laptop PC, and a wearable device. A wearable device may be an accessory (e.g., watch, ring, bracelet, anklet, necklace, eyewear, contact lens, or head-mounted-device (HMD)), a textile or clothing integral (e.g. electronic garment); It may include at least one of body-attached (eg, skin pad or tattoo), or bioimplantable circuitry.

In some embodiments, the electronic device may include, for example, a television, digital video disk (DVD) player, audio, refrigerator, air conditioner, vacuum cleaner, oven, microwave oven, washing machine, air purifier, set-top box, home automation control panel, Secure at least one of a control panel, media box (eg Samsung HomeSync ^TM , Apple TV ^TM , or Google TV ^TM ), game console (eg Xbox ^TM , PlayStation ^TM ), electronic dictionary, electronic key, camcorder, or electronic picture frame. may include

In another embodiment, the electronic device may include various medical devices (eg, various portable medical measuring devices (eg, a blood glucose monitor, a heart rate monitor, a blood pressure monitor, or a body temperature monitor), magnetic resonance angiography (MRA), magnetic resonance imaging (MRI), or CT. (computed tomography, camera, or ultrasound), navigation devices, global navigation satellite system (GNSS), event data recorder (EDR), flight data recorder (FDR), automotive infotainment devices, marine electronic equipment ( Examples: navigation devices for ships, gyro compasses, etc.), avionics, security devices, head units for vehicles, industrial or household robots, drones, ATMs in financial institutions, point of sale (POS) in stores sales), or Internet of Things devices (eg, light bulbs, various sensors, sprinkler devices, fire alarms, thermostats, street lights, toasters, exercise equipment, hot water tanks, heaters, boilers, etc.).

Hereinafter, embodiments according to the present disclosure will be described in detail with reference to the accompanying drawings so that those of ordinary skill in the art to which the present disclosure pertains can easily implement them.

1 is a flowchart illustrating a control method of an electronic device 100 according to an embodiment of the present disclosure. 2 to 4 are diagrams for specifically explaining the operation of the hardware suitability identification module 1100 for performing the control method of the electronic device 100 according to FIG. 1 .

First, the 'electronic device' according to the present disclosure may be implemented in various types such as a smart phone, a tablet PC, a notebook computer, a TV, and a robot, and the electronic device according to the present disclosure is not limited to a specific type of device. Hereinafter, an electronic device according to the present disclosure will be referred to as an electronic device 100 .

Meanwhile, the 'neural network model' refers to an artificial intelligence model including an artificial neural network, and may be learned by deep learning. For example, a neural network model is one of Deep Neural Network (DNN), Convolution Neural Network (CNN), Recurrent Neural Network (RNN), and Generative Adversarial Networks (GAN). It may include at least one artificial neural network. However, the neural network model according to the present disclosure is not limited to the above-described example.

Meanwhile, in the electronic device 100 according to the present disclosure, the first device information regarding the hardware specification of the electronic device 100 and the hardware suitability identification module 1100 may be stored.

Here, 'first device information' refers to information on the hardware specification of the electronic device 100 . Specifically, the first device information is a term to collectively refer to information on specifications indicating the performance of each of a plurality of hardware included in the electronic device 100 , and includes a plurality of hardware components included in the electronic device 100 . (hardware component) may include detailed information about the presence, number, type, and performance of each.

In particular, the first device information may include information on a specification of a processor included in the electronic device 100 , a specification of a memory included in the electronic device 100 , and a specification of a data acquisition unit included in the electronic device 100 . have. Here, the data acquisition unit is a configuration for acquiring data input to at least one neural network model included in the electronic device 100 , and may include at least one of a camera, a microphone, and a sensor included in the electronic device 100 .

Meanwhile, the first device information may include performance evaluation information for each of a plurality of hardware included in the electronic device 100 . Here, 'performance evaluation information' is scored by an expert comprehensively evaluating the performance based on experiments and analysis for each hardware configuration, and may be pre-stored in the electronic device 100 as well as received from an external server. may be updated.

The 'hardware suitability identification module 1100' refers to a module for identifying a neural network model suitable for hardware of the electronic device 100 . Specifically, the hardware suitability identification module 1100 may output information on whether it is appropriate to execute the neural network model included in the external device using the hardware of the electronic device 100 . In describing the present disclosure, the term 'suitable' may be replaced with terms such as 'compatible' or 'substitutable'.

1 , when a user input is received, the electronic device 100 may transmit a first signal for requesting information related to at least one neural network model included in at least one external device (S110). .

Here, 'user input' refers to a user touch input through the display of the electronic device 100 , a user voice received through a microphone of the electronic device 100 , or an input of a physical button provided in the electronic device 100 , the electronic device It may be received based on a control signal transmitted by a remote control device for controlling 100 . The term 'transmission' refers to unicasting, which targets a specific external device when transmitting a signal or data, and broadcasting, which transmits to all external devices connected to the network simultaneously without targeting a specific external device. may be used in the meaning of including The 'information related to at least one neural network model' may include second device information and first model information, which will be described later.

Meanwhile, like the electronic device 100 , the 'external device' may be implemented in various types such as a smart phone, a tablet PC, a laptop computer, a TV, and a robot, and the type of the external device may be different from the type of the electronic device 100 . may be The electronic device 100 and the external device may be 'connected' to each other, which means that a communication connection is established as a result of exchanging a request and a response for a communication connection between the electronic device 100 and the external device. . There is no particular limitation on the method of communication connection according to the present disclosure.

In response to the first signal, the electronic device 100 receives, from the first external device among the at least one external device, second device information on the hardware specification of the first external device, and at least one neural network included in the first external device. A second signal including first model information on the model may be received (S120). That is, when a first signal corresponding to a request to search for at least one neural network model is received, the first external device may transmit a second signal to the electronic device 100 as a response to the request, and the second signal may include second device information and first model information. In describing the present disclosure, the term 'first external device' is used as a term for specifying an external device capable of transmitting installation data of a neural network model to the electronic device 100 . Hereinafter, the first external device according to the present disclosure will be referred to as a first external device 200 - 1 .

Meanwhile, the first external device 200-1 performs a user authentication process based on the first user information included in the first signal and the second user information stored in the first external device 200-1, and performs user authentication. When this is completed, the second signal may be transmitted to the electronic device 100 . The user authentication process or user suitability identification process will be described in detail with reference to FIGS. 9 and 10 .

'Second device information' refers to information on hardware specifications of the first external device 200 - 1 . Specifically, the second device information is a general term for information on specifications indicating what kind of performance each hardware included in the first external device 200 - 1 has, and is dependent on the hardware performance of the electronic device 100 . It corresponds to the first device information about the

That is, the second device information may include detailed information on the presence, number, type, and performance of each of the plurality of hardware components included in the first external device 200 - 1 , as in the case of the first device information. , the second device information may also include performance evaluation information for each hardware included in the first external device 200 - 1 . In addition, the second device information includes a specification of a processor included in the first external device 200 - 1 , a specification of a memory included in the first external device 200 - 1 and included in the first external device 200 - 1 . It may include information about the specifications of the data acquisition unit. Here, the data acquisition unit of the first external device 200 - 1 acquires data input to at least one neural network model included in the first external device 200 - 1 , similarly to the data acquisition unit of the electronic device 100 . As a configuration to do so, at least one of a camera, a microphone, and a sensor included in the first external device 200 - 1 may be included.

The 'first model information' refers to information on at least one neural network model included in the first external device 200 - 1 . Specifically, the first model information may include information on a service type of each of at least one neural network model included in the first external device 200-1, information on a personalization level, and information on a hardware requirement specification. .

When a second signal including the second device information and the first model information is received from the first external device 200 - 1 , the electronic device 100 transmits the first device information, the second device information, and the first model information. It may be input to the hardware suitability identification module 1100 ( S130 ) to identify whether each of the at least one neural network model included in the first external device 200 - 1 is suitable for the hardware of the electronic device 100 . That is, when the first device information, the second device information, and the first model information are input, the hardware suitability identification module 1100 performs various implementations of the present disclosure based on the first device information, the second device information, and the first model information. A hardware suitability identification process according to an example may be performed. Hereinafter, the 'hardware suitability identification process' will be described in detail with reference to FIGS. 2 to 4 together with FIG. 1 . Specifically, the hardware suitability identification process according to the present disclosure may include the processes of steps 1 and 2 as shown in FIGS. 2 and 3 .

First, when 'step 1' is described with reference to FIG. 2 together with FIG. 1 , the electronic device 100 receives the first device information stored in the electronic device 100 and the first external device 200 - 1 . By identifying whether the hardware specification of the electronic device 100 is equal to or superior to the hardware specification of the first external device 200-1 in all parts based on the obtained second device information, the first external device 200- Hardware conformance identification process can be performed for all neural network models included in 1).

Specifically, if the specification of each of the plurality of hardware components included in the electronic device 100 is equal to or greater than the specification of the plurality of hardware components included in the first external device 200-1 (S140-Y), the electronic device 100 may identify that at least one neural network model included in the first external device 200 - 1 is suitable for hardware of the electronic device 100 ( S150 - 1 ).

That is, the electronic device 100 identifies the hardware configuration of the first external device 200 - 1 corresponding to each of the plurality of hardware configurations included in the electronic device 100 , and compares the respective specifications for each corresponding hardware configuration. can And, as a result of comparing the specifications of each of the plurality of hardware components included in the electronic device 100 with the specifications of each of the plurality of hardware components included in the first external device 200 - 1 , the If the specification of each of the plurality of hardware components is greater than or equal to the specification of the plurality of hardware components included in the first external device 200-1, it means that each of the at least one neural network model included in the first external device 200-1 is Regardless of whether a hardware specification is required, it may be assumed that all of the at least one neural network model included in the first external device 200 - 1 can be executed using the hardware of the electronic device 100 . Accordingly, in this case, the electronic device 100 may identify all of the at least one neural network model included in the first external device 200 - 1 as suitable for the hardware of the electronic device 100 .

Specifically, as shown in FIG. 4 , the hardware suitability identification module 1100 includes a processor suitability identification module 1110 , a memory suitability identification module 1120 , a camera suitability identification module 1130 , and a microphone suitability identification module 1140 . and a sensor suitability identification module 1150 and the like. And, the electronic device 100 through each module included in the hardware suitability identification module 1100, a processor specification, a memory specification, a camera specification, a microphone specification, and a sensor included in each of the first device information and the second device information Based on the information on the specifications, the hardware conformity identification process for each hardware configuration can be performed.

For example, the specifications of the processor, memory, camera, microphone, and sensor included in the electronic device 100 are all specifications of the processor, memory, camera, microphone, and sensor included in the first external device 200-1. If this is the case, the electronic device 100 may identify that all of the at least one neural network model included in the first external device 200 - 1 are suitable for the hardware of the electronic device 100 . In this case, the electronic device 100 may perform the hardware suitability identification process for each hardware configuration in the order of the processor, memory, camera, microphone, and sensor, but there is a special order restriction in the hardware suitability identification process for each hardware configuration according to the present disclosure. there is not this

Meanwhile, in the above description, the specifications of the processor, memory, camera, microphone, and sensor included in each of the electronic device 100 and the first external device 200 - 1 have been compared, but this is only for convenience of description. , the electronic device 100 may compare the specifications of each detailed configuration included in each of the processor, the memory, the camera, the microphone, and the sensor according to the present disclosure.

For example, the electronic device 100 may include a CPU (Central Processing Unit), a GPU (Graphic Processing Unit), and a Neural Processing Unit (NPU) among processors of the electronic device 100 and the first external device 200 - 1 . The specifications are compared, and the specifications of the electronic device 100 and the first external device 200-1 are compared for each RAM (Random Access Memory) and ROM (Read Only Memory) of the memory, and the Global Positioning System (GPS) of the sensors. A hardware suitability identification process is performed by comparing the specifications of the electronic device 100 and the first external device 200 - 1 for each sensor such as a sensor, a gyro sensor, an acceleration sensor, and a lidar sensor. You may. Of course, the detailed configuration included in each of the processor, memory, camera, microphone, and sensor is not limited to the example described above.

Meanwhile, according to an embodiment of the present disclosure, it means that the specification of the hardware configuration included in the electronic device 100 is greater than or equal to the specification of the hardware configuration included in the first external device 200 - 1 , in the electronic device 100 . It may mean that all specifications indicating the performance of the included hardware configuration are superior to or at least equal to all specifications indicating the performance of the hardware configuration included in the first external device 200 - 1 .

For example, a CPU included in the electronic device 100 has 8 cores, 16 threads, a clock speed of 3.6 GHz, and a capacity of a cache memory. If this is 8 MB, the number of cores and the number of threads of the CPU included in the first external device 200-1 is 4, the number of threads, the clock speed is 3.3 GHz, and the capacity of the cache memory is 8 MB, the electronic device 100 is It may be identified that the CPU performance of the electronic device 100 is equal to or greater than the CPU performance of the first external device 200 - 1 . In this example, the number of cores, the number of threads, clock speed, and cache memory are exemplified as performance indicators of the CPU, but it goes without saying that performance indicators such as bus speed and TDP (thermal design power) may be additionally considered.

According to another embodiment, that the specification of the hardware configuration included in the electronic device 100 is greater than or equal to the specification of the hardware configuration included in the first external device 200 - 1 means that the hardware configuration included in the electronic device 100 is It may mean that the performance evaluation information indicating the performance is higher than the performance evaluation information indicating the performance of the hardware configuration included in the first external device 200 - 1 . As described above, the performance evaluation information refers to a score obtained by comprehensively evaluating the performance of an expert based on experiments and analysis on each hardware configuration, and may be included in the first device information and the second device information. For example, if the score according to the performance evaluation information is 97 in the case of the CPU included in the electronic device 100 and the score according to the performance evaluation information is 86 in the case of the CPU included in the first external device 200-1, The electronic device 100 may identify that the CPU performance of the electronic device 100 is higher than the CPU performance of the first external device 200 - 1 .

Second, when 'step 2' is described with reference to FIG. 3 together with FIG. 1 , the electronic device 100 receives the first device information stored in the electronic device 100 and the first external device 200 - 1 . By identifying whether the hardware specifications of the electronic device 100 are equal to or superior to the hardware requirements of each of the at least one neural network model included in the first external device 200-1 based on the first model information, A hardware conformance process may be performed for each neural network model included in the first external device 200 - 1 .

As illustrated in FIG. 3 , step 2 of the hardware conformance identification process may be performed only when step 1 of the hardware conformance identification process as described above has not been passed. That is, as a result of performing step 1 of the hardware suitability identification process as described above, one or more neural network models among at least one neural network model included in the first external device 200 - 1 are suitable for the hardware of the electronic device 100 . If it is determined that not, the electronic device 100 may perform step 2 of the hardware suitability identification process.

Specifically, when the specification of at least one of the plurality of hardware components included in the electronic device 100 is less than the specification of the plurality of hardware components included in the first external device 200-1 (S140-N), the electronic device ( 100 indicates that one or more neural network models having a hardware requirement specification lower than a specification of a plurality of hardware components included in the electronic device 100 among at least one neural network model included in the first external device 200-1 is an electronic device ( It can be identified as being suitable for the hardware of 100) (S150-2).

That is, as a result of comparing the specifications of each of the plurality of hardware components included in the electronic device 100 with the specifications of each of the plurality of hardware components included in the first external device 200 - 1 , the If the specification of at least one of the plurality of hardware components is less than the specification of the plurality of hardware components included in the first external device 200 - 1 , each of the at least one neural network model included in the first external device 200 - 1 It can be said that all of the neural network models suitable for execution using the hardware of the electronic device 100 can be identified only when the hardware requirements of the device 100 and the specifications of a plurality of hardware components included in the electronic device 100 are individually compared. have.

Accordingly, in this case, the electronic device 100 determines the hardware requirements of the neural network model and the specifications of a plurality of hardware components included in the electronic device 100 for each at least one neural network model included in the first external device 200 - 1 . By comparison, it may be identified whether all of the hardware requirements of the at least one neural network model included in the first external device 200 - 1 are less than the specifications of a plurality of hardware configurations included in the electronic device 100 .

For example, if all of the hardware requirements of the first neural network model included in the first external device 200 - 1 are less than the specifications of a plurality of hardware configurations included in the electronic device 100 , the electronic device 100 may 1 It may be identified that the neural network model is suitable for the hardware of the electronic device 100 . On the other hand, if at least one of the hardware requirements of the second neural network model included in the first external device 200 - 1 is greater than or equal to the specification of a plurality of hardware configurations included in the electronic device 100 , the electronic device 100 may 2 It may be identified that the neural network model is not suitable for the hardware of the electronic device 100 . As described above, information on hardware requirements for each of the at least one neural network model included in the first external device 200-1 may be included in the first model information received from the first external device 200-1. have.

Various methods for comparing the specifications of the plurality of hardware components included in the electronic device 100 in step 1 with the specifications of the plurality of hardware components included in each of the first external device 200-1 include, in step 2, the electronic device ( 100 ) may be directly applied to a method of comparing specifications of a plurality of hardware configurations with hardware requirements specifications of at least one neural network model included in the first external device 200 - 1 . Therefore, in step 2, the redundant description of the method of comparing the specifications of a plurality of hardware components included in the electronic device 100 with the hardware requirements specifications of at least one neural network model included in the first external device 200-1 is omit

Meanwhile, in comparing the hardware requirements of each of the at least one neural network model included in the first external device 200 - 1 with the specifications of a plurality of hardware configurations included in the electronic device 100 , the The hardware specifications at the time of product release may be compared with the hardware requirements of each of the at least one neural network model included in the first external device 200-1, but at least one neural network included in the first external device 200-1 may be compared. At least one hardware specification included in the first external device 200 - 1 is available at the time of comparing the hardware requirements of each model with the specifications of a plurality of hardware components included in the electronic device 100 . It can also be compared with the hardware requirements of each neural network model.

Meanwhile, as described above, in step 1 , comparing the specifications of the plurality of hardware components included in the electronic device 100 with the specifications of the plurality of hardware components included in the first external device 200 - 1 includes each It is assumed that the components correspond to each other and there is a comparison target. That is, if there is a hardware configuration included in the first external device 200 - 1 but not included in the electronic device 100 , the hardware configuration can be replaced with another hardware configuration included in the electronic device 100 . Otherwise, the electronic device 100 may perform the hardware suitability identification process according to step 2 .

Furthermore, as described above, in step 2, comparing the specifications of a plurality of hardware components included in the electronic device 100 and the hardware requirements specifications of at least one neural network model included in the first external device 200-1 In addition, it is premised that each configuration corresponds to each other and a comparison target exists. That is, if there is a hardware configuration required by the hardware requirement specification of the neural network model of the first external device 200 - 1 but not included in the electronic device 100 , the hardware configuration is transferred to another hardware configuration included in the electronic device 100 . Unless a hardware configuration can replace it, the electronic device 100 may identify that the neural network model is not suitable for the hardware of the electronic device 100 .

As described above, when at least one neural network model suitable for hardware of the electronic device 100 is identified from among at least one neural network model included in the first external device 200 - 1 , the electronic device 100 displays the identified at least one neural network model. A third signal including a request for installation data of a neural network model of may be transmitted to the first external device 200 - 1 ( S160 ). And, in response to the third signal, the electronic device 100 is configured to include installation data of at least one neural network model identified from the first external device 200 - 1 as being suitable for the hardware of the electronic device 100 . 4 signals may be received (S170).

That is, when a third signal including a request for installation data of at least one identified neural network model is transmitted to the first external device 200-1, the first external device 200-1 transmits the identified at least one Installation data of the neural network model may be transmitted to the electronic device 100 . And, when installation data of at least one identified neural network model is received, the electronic device 100 may install at least one identified neural network model based on the received installation data. Accordingly, the electronic device 100 replaces at least one of the plurality of neural network models included in the electronic device 100 with the identified at least one neural network model, thereby providing at least one personalized by the first external device 200 - 1 . One neural network model can be executed in the electronic device 100 . The personalization of the neural network model will be described later with reference to FIGS. 5A to 8 .

Specifically, the installation data received from the first external device 200-1 may include identification information for each of the at least one identified neural network model and configuration information necessary for installing the identified at least one neural network model. . Here, 'configuration information' refers to the structure and type of the neural network included in the neural network model, the number of layers included in the neural network, the number of nodes for each layer, the weight value of each node, connection relationships between a plurality of nodes, etc. may include information about

Meanwhile, 'installing' the neural network model may include 'conversion' of the neural network model included in the electronic device 100 to the neural network model of the first external device 200 - 1 . And, 'converting the neural network model included in the electronic device 100 to the neural network model included in the external device is not only a case of replacing the entire neural network model of the electronic device 100 with the entire neural network model of the external device, but also electronic A part of the neural network model of the electronic device 100 is replaced with a part of the neural network model of the external device, such as when only the weight values of the nodes included in the neural network model of the device 100 are changed to the weights of the nodes included in the neural network model of the external device cases may include.

According to various embodiments as described above with reference to FIGS. 1 to 4 , when a neural network model is transferred from an external device, the electronic device 100 executes the neural network model using the hardware of the electronic device 100 . It is possible to determine whether a neural network model is transferred by identifying whether it is suitable, and accordingly, the efficiency and reliability of transfer learning can be remarkably improved.

5A is a flowchart illustrating a control method of the electronic device 100 according to another embodiment of the present disclosure, and FIG. 5A is a flowchart illustrating a control method of an electronic device according to another embodiment of the present disclosure. And, FIGS. 6 to 8 are diagrams for specifically explaining the operation of the model suitability identification module 1200 for performing the control method of the electronic device 100 according to FIG. 5A .

In the above, the hardware suitability identification process according to the present disclosure has been described with reference to FIGS. 1 to 4 . However, according to another embodiment of the present disclosure, the electronic device 100 may perform a 'model fitting process' for each neural network model included in the first external device of the electronic device 100 . Hereinafter, a model suitability identification process according to the present disclosure will be described in detail with reference to FIGS. 5A to 8 .

First, the electronic device 100 may further store the first device information, the second device information, and the hardware suitability identification module 1100 as described above, as well as the second model information and the model suitability identification module 1200 . .

Here, 'second model information' refers to information on at least one neural network model included in the electronic device 100 . Specifically, the second model information may include information on a service type of each of at least one neural network model included in the electronic device 100 and information on a personalization level.

The 'model suitability identification module 1200' refers to a module for identifying a neural network model suitable to replace the neural network model included in the electronic device 100 . Specifically, as shown in FIG. 6 , the model suitability identification module 1200 based on the second model information stored in the electronic device 100 and the first model information received from the first external device 200 - 1 , information on whether it is appropriate to replace the neural network model included in the electronic device 100 with the neural network model included in the external device may be output.

That is, when at least one neural network model suitable for hardware of the electronic device 100 is identified ( S210 -Y ), the electronic device 100 inputs the first model information and the second model information to the model suitability identification module 1200 . Accordingly, it may be identified whether each of the at least one neural network model identified as suitable for the hardware of the electronic device 100 is suitable to replace the neural network model included in the electronic device 100 ( S220 ).

Specifically, the model suitability identification process according to the present disclosure may include two processes performed through each of the model type suitability identification module 1210 and the personalization level suitability identification module 1220 as shown in FIG. , Hereinafter, each process included in the model suitability identification process will be described in detail, and then a control method according to an embodiment of the present disclosure will be described again.

First, as shown in FIG. 7 , the 'model type conformity identification module 1210' is configured to provide information on the service type included in each of the first model information and the second model information to the electronic device 100 . By comparing the service type of each of the included at least one neural network model with the service type of each of the at least one neural network model identified as being suitable for the hardware of the electronic device 100 , neural network models having the same service type may be identified.

Here, the 'type of the neural network model' may be distinguished according to input/output information for the neural network model and a function of the neural network model accordingly. For example, when both the first neural network model and the second neural network model are neural network models that provide a voice recognition service by inputting a voice signal according to the user's utterance and outputting text corresponding to the user's voice, model type suitability identification The module 1210 may identify that the service type of the first neural network model and the service type of the second neural network model are the same.

Second, as shown in FIG. 7 , the 'personalization level suitability identification module 1220' is a neural network having the same service type based on the personalization level information included in each of the first model information and the second model information. By comparing the personalization levels between the models, it is possible to identify a neural network model having a high personalization level among neural network models having the same service type.

Here, the 'personalization level of the neural network model' refers to the degree to which the neural network model is updated to suit the user as the neural network model is trained based on the personalization data related to the user of the electronic device 100 after the neural network model is installed in the electronic device 100. it means. Here, 'personalized data' refers to learning data used for personalization of a neural network model included in an external device. An operation of the personalization level suitability identification module 1220 will be described in more detail with reference to FIG. 8 together with FIG. 7 .

Specifically, the personalization level may be determined based on 'information on the user's history'. Here, the information on the user's history may include information on the number, frequency, and period of use of the neural network model by the user. For example, when the second neural network model included in the electronic device 100 is used 100 times in one month, when the first neural network model included in the external device is used 30 times in one month or 100 times in two months It may be determined that the level of personalization is higher than that in the case of

In particular, as shown in FIG. 8 , information on the history of the user using the neural network model 2000 included in the electronic device 100 is the electronic device based on data obtained by the user through the electronic device 100 . In addition to the case of using the neural network model 2000 included in 100 (type 1 in FIG. 8 ), the neural network is based on data obtained by the user through another electronic device 100 connected to the electronic device 100 . A case in which the model 2000 is used (type 2 in FIG. 8 ) may be included.

For example, not only when the user uses the neural network model 2000 included in the smart phone based on the image acquired through the smart phone, but also the image obtained by the user through a robot cleaner connected to the smart phone is transmitted to the smart phone Accordingly, the case in which the neural network model 2000 included in the smart phone is used may also be included in the history of using the neural network model 2000 included in the smart phone.

Meanwhile, the personalization level may be determined based on 'information on user feedback'. Here, the information about the user's feedback may include direct evaluation information input by the user for the result of using the neural network model 2000 and indirect evaluation information related to the result of using the neural network model 2000 (type 3 of FIG. 8 ). .

For example, when the user directly inputs positive evaluation information for an application including the neural network model 2000 after the neural network model 2000 is used in the electronic device 100, the The level of personalization can be increased. On the other hand, when the user directly inputs negative evaluation information for an application including the neural network model 2000 after the neural network model 2000 is used in the electronic device 100, the personalization level for the neural network model 2000 is can increase On the other hand, when a user input of the same content is repeatedly input to the neural network model 2000, it can be indirectly confirmed that the user is not satisfied with the output of the neural network model 2000, The personalization level may be reduced.

Meanwhile, the personalization level may be quantitatively calculated according to a weighted sum of various types of information for evaluating the personalization level as described above. Specifically, referring to FIG. 8 , the personalization level may be calculated by summing values obtained by multiplying each 'number of times for each type' and a preset 'weight for each type' for evaluating the personalization level.

For example, when the user directly inputs evaluation information for an application including the neural network model 2000 , the highest weight is given, and an image captured using the camera of the electronic device 100 or the electronic device 100 . When data acquired from the electronic device 100, such as a voice acquired using a microphone of , is input to the neural network model 2000, the second highest weight is given to the electronic device 100 and not the electronic device 100. When data received from another connected electronic device 100 is input to the neural network model 2000, a third highest weight may be assigned. In addition, the value obtained by multiplying the number of times per type as above by the weight per type becomes a weighted sum for each type, and the value obtained by adding the weighted sum per type for all types can be a quantitative number representing the personalization level of the neural network model (2000). have.

Referring back to FIG. 5A , a service type of a first neural network model among at least one neural network model identified as suitable for hardware of the electronic device 100 is a second neural network model among a plurality of neural network models included in the electronic device 100 . is the same as the service type of (S230-Y), the electronic device 100 sets the personalization level of the first neural network model and the second neural network based on the personalization level information included in each of the first model information and the second model information. You can compare the personalization levels of the models. And, if the personalization level of the first neural network model is higher than the personalization level of the second neural network model (S240-Y), it may be identified that the first neural network model is suitable to replace the second neural network model (S250).

When the model suitability identification process as described above is performed, the electronic device 100 includes a request for installation data of at least one neural network model identified as suitable to replace the neural network model included in the electronic device 100. A third signal may be transmitted to the first external device 200-1 (S260). That is, when a third signal is transmitted to the first external device 200 - 1 after performing not only the hardware suitability identification process but also the model suitability identification process, the third signal is at least one identified as the model suitability identification process is performed. may include a request for installation data of the neural network model of

In response to the third signal, the electronic device 100 may receive a fourth signal including installation data for at least one identified neural network model from the first external device 200 - 1 ( S270 ). That is, when the third signal is transmitted to the first external device 200 - 1 after performing not only the hardware suitability identification process but also the model suitability identification process, the fourth signal received in response to the third signal is model suitability identification It may include installation data of at least one neural network model identified as the process is performed.

Meanwhile, although the embodiment in which the model suitability identification process is performed after performing the hardware suitability identification process has been described above, there is no time-series limitation between the hardware suitability identification process and the model suitability identification process according to the present disclosure. Specifically, the electronic device 100 may first input the first model information and the second model information into the model suitability identification module 1200 to identify whether it is appropriate to replace the first neural network model with the second neural network model. have. And, if it is identified that replacing the first neural network model with the second neural network model is suitable, the electronic device 100 compares the hardware performance of the electronic device 100 with the hardware performance required to execute the second neural network model. , whether it is appropriate to execute the second neural network model using the hardware of the electronic device 100 may be identified.

Meanwhile, in the above description, the personalization level of the first neural network model and the personalization level of the first neural network model only when the service types of the first neural network model included in the first external device 200 - 1 and the second neural network model included in the electronic device 100 are the same. It was described as comparing the personalization level of the second neural network model. However, according to another embodiment of the present disclosure, when a neural network model having the same service type as that of the first neural network model included in the first external device 200 - 1 does not exist in the electronic device 100 , The electronic device 100 may identify that it is suitable to transfer the first neural network model to the electronic device 100 without an identification process for the personalization level of the first neural network model.

Meanwhile, in the above, an embodiment in which a model conformance process is additionally performed for each of at least one neural network model identified as suitable for the hardware of the electronic device 100 after performing the hardware suitability identification process has been described. However, the present invention is not limited thereto. That is, according to another embodiment of the present disclosure, the electronic device 100 performs only the model suitability identification process without performing the hardware suitability identification process, so that the installation data for the neural network model satisfying the model suitability is transmitted to the first external device. It may be received from (200-1). Hereinafter, an embodiment of receiving installation data from the first external device 200 - 1 by performing only a model suitability identification process will be described with reference to FIG. 5B . However, in the description of FIG. 5B below, redundant descriptions of the same contents as those described above will be omitted.

As illustrated in FIG. 5B , when a user input is received, the electronic device 100 may transmit a first signal for requesting information related to at least one neural network model included in at least one external device (S205). And, in response to the first signal, the electronic device 100 transmits at least one neural network model included in the first external device 200-1 from the first external device 200-1 among at least one external device. It is possible to receive a second signal including the external model information for (S215). Hereinafter, within the limits for describing the embodiment as shown in FIG. 5B, the term 'external model information' is used in the same meaning as the term 'first model information' in the description for FIGS. 1 to 5A. do.

When the second signal including the external model information is received from the first external device 200 - 1 , the electronic device 100 transmits the internal model information stored in the electronic device and the external device received from the first external device 200 - 1 . By inputting model information into the model suitability identification module 1200 ( S220 ), each of the at least one neural network model identified as suitable for the hardware of the electronic device 100 replaces the neural network model included in the electronic device 100 . You can determine whether it is suitable or not. Hereinafter, within the limits for describing the embodiment as shown in FIG. 5B , the term 'internal model information' is used in the same meaning as the term 'second model information' in the description of FIG. 5A.

As described above, the model suitability identification process according to the present disclosure is a model type suitability identification process and personalization performed through each of the model type suitability identification module 1210 and the personalization level suitability identification module 1220 as shown in FIG. 7 . It may include a level conformity identification process.

Specifically, the service type of the first neural network model among the at least one neural network model included in the first external device 200 - 1 is the service type of the second neural network model among the plurality of neural network models included in the electronic device 100 , and If the same (S230-Y), the electronic device 100 compares the personalization level of the first neural network model with the personalization level of the second neural network model based on the information on the personalization level included in each of the internal model information and the external model information. can And, if the personalization level of the first neural network model is higher than the personalization level of the second neural network model (S240-Y), it may be identified that the first neural network model is suitable to replace the second neural network model (S250).

When the model suitability identification process as described above is performed, the electronic device 100 includes a request for installation data of at least one neural network model identified as suitable to replace the neural network model included in the electronic device 100. A third signal may be transmitted to the first external device 200-1 (S260). And, in response to the third signal, the electronic device 100 may receive a fourth signal including installation data for at least one neural network model identified from the first external device 200 - 1 ( S270 ). ).

According to various embodiments as described above with reference to FIGS. 5A to 8 , when the neural network model is transferred from the external device, the electronic device 100 is configured based on the degree of personalization of the neural network model included in the external device. It is possible to determine whether the neural network model is transferred by identifying whether it is appropriate to replace the neural network model included in ( 100 ), and thus the efficiency and reliability of transfer learning can be further improved.

9 is a diagram for explaining a method of controlling the electronic device 100 according to another embodiment of the present disclosure. And, FIG. 10 is a sequence diagram for explaining an embodiment when there are a plurality of external devices according to the present disclosure.

In the above, the hardware suitability identification process and the model suitability identification process according to the present disclosure have been described above, but according to another embodiment of the present disclosure, the user suitability identification process may be additionally performed prior to the hardware suitability identification process and the model suitability identification process. have. The 'user suitability identification process' is a process of identifying whether it is appropriate for the user of the electronic device 100 to use a neural network model of an external device, and may also be referred to as a 'user authentication process'. In the present disclosure, the term 'transitional suitability' may be used to encompass hardware suitability, model suitability, and user suitability. Hereinafter, a user suitability identification process will be described in detail with reference to FIGS. 9 and 10 .

1 to 8 , the electronic device 100 transmits a first signal for requesting information related to at least one neural network model included in the first external device 200-1 to the first external device 200- 1), the external device responds to the first signal with second device information on the hardware specification of the first external device 200-1 and at least one piece of information included in the first external device 200-1 It has been described on the premise that the second signal including the first model information on the neural network model is transmitted to the electronic device 100 .

However, prior to providing the second device information and the first model information to the electronic device 100 , the external device performs a user suitability identification process by the external device or performs a user suitability identification process by the electronic device 100 . can be made to be performed. In the user suitability identification process, first user information and second user information may be used, and the first user information includes at least one of account information for a user of the electronic device 100 and identification information for an electronic external device. may be included, and the second user information may include at least one of account information for a user of the first external device 200 - 1 and identification information for the first external device 200 - 1 .

First, referring to FIG. 9 , a user suitability identification process may be performed by the electronic device 100 according to the present disclosure. Specifically, the user suitability identification process may be performed through the user suitability identification module 1300 included in the electronic device 100 as shown in FIG. 9 .

Here, the 'user suitability identification module 1300' is the user of the electronic device 100 based on the first user information stored in the electronic device 100 and the second user information received from the first external device 200-1. refers to a module for identifying whether it is appropriate to use the neural network model included in the first external device 200 - 1 .

Specifically, the first user information may be stored in the electronic device 100 . In addition, the electronic device 100 may transmit a first signal for requesting information related to at least one neural network model included in the first external device 200 - 1 to the first external device 200 - 1 . When the first signal is received, the first external device 200 - 1 may transmit second user information to the electronic device 100 . When the second user information is received, the electronic device 100 may perform a user suitability identification process based on the first user information and the second user information.

In particular, when the account information for the user of the electronic device 100 included in the first user information matches the account information for the user of the first external device 200-1 included in the second user information, the electronic device ( 100 may identify that it is suitable for the user of the electronic device 100 to use the neural network model included in the first external device 200 - 1 . On the other hand, if the account information on the user of the electronic device 100 included in the first user information and the account information on the user of the first external device 200 - 1 included in the second user information do not match, the electronic device Reference numeral 100 may identify that it is not suitable for the user of the electronic device 100 to use the neural network model included in the first external device 200 - 1 .

In the above description, user suitability is satisfied when the account information for the user of the electronic device 100 and the account information for the user of the first external device 200 - 1 match, but the present disclosure is limited thereto no. That is, according to an embodiment, when the account for the user of the electronic device 100 has higher authority than the account for the user of the first external device 200 - 1 , and the user of the electronic device 100 User suitability according to the present disclosure may be satisfied even when the account for the user of the first external device 200 - 1 is included in the same group registered as the account for the user of the first external device 200 - 1 .

Meanwhile, according to another embodiment, the external device that has received the first signal does not transmit the second user information to the electronic device 100 , but rather the first device including the encrypted second device information and the encrypted first model information. By transmitting the 2 signals to the electronic device 100 , user authentication may be induced. Specifically, when a password or biometric information (eg, fingerprint information, iris information, etc.) for decrypting the encrypted second device information and the encrypted first model information is input, the electronic device 100 may It can be identified that it is suitable for the user of the user to use the neural network model included in the first external device 200 - 1 . Here, there are no special restrictions on encryption and decryption methods and authentication procedures required for decryption.

Next, referring to FIG. 10 , the user suitability identification process may be performed by the first external device 200 - 1 according to the present disclosure. Specifically, although not illustrated, the user suitability identification process may be performed through the user suitability identification module 1300 included in the first external device 200 - 1 .

As shown in FIG. 10 , the electronic device 100 may receive a first user input ( S1010 ), and according to the first user input, information related to at least one neural network model included in at least one external device is displayed. The requesting first signal may be transmitted to the first external device 200 - 1 ( S1020 ).

When the first signal is received, the first external device 200-1 performs a user suitability identification process based on the first user information included in the first signal and the second user information stored in the first external device 200-1. can be performed (S1030).

Specifically, when the account information for the user of the electronic device 100 included in the first user information matches the account information for the user of the first external device 200-1 included in the second user information, the first The external device 200 - 1 may identify that it is suitable for the user of the electronic device 100 to use the neural network model included in the first external device 200 - 1 . On the other hand, if the account information on the user of the electronic device 100 included in the first user information and the account information on the user of the first external device 200-1 included in the second user information do not match, the first The external device 200 - 1 may identify that it is not suitable for the user of the electronic device 100 to use the neural network model included in the first external device 200 - 1 . On the other hand, if the first signal received from the electronic device 100 does not include the first user information, the first external device 200 - 1 transmits a request for the first user information to the electronic device 100 . The first user information may be received.

When it is identified that the user of the electronic device 100 is suitable to use the neural network model included in the first external device 200 - 1 , the first external device 200 - 1 is the first external device 200 - 1 A second signal including second device information on hardware specifications and first model information on at least one neural network model included in the first external device 200 - 1 may be transmitted to the electronic device 100 ( S1040 ). ). On the other hand, if it is identified that it is not suitable for the user of the electronic device 100 to use the neural network model included in the first external device 200-1, the first external device 200-1 transmits the second signal to the electronic device. (100) may not be transmitted. And, if the second signal is not received from the first external device 200-1 for a preset time, the electronic device 100 provides information related to at least one neural network model included in the first external device 200-1. A user notification may be provided indicating that the request for .

As a result of performing the user suitability identification process as described above, if it is identified that it is suitable for the user of the electronic device 100 to use the neural network model included in the first external device 200-1, the electronic device 100 is A hardware suitability identification process and a model suitability identification process according to the disclosure may be performed.

That is, when the second signal is received, the electronic device 100 performs the first external device 200 - 1 based on the first device information about the hardware specification of the electronic device 100 , the second device information, and the first model information. ), at least one neural network model suitable for hardware of the electronic device 100 may be identified among at least one neural network model included in ( S1050 ). In addition, the electronic device 100 includes at least one piece of information identified as suitable for the hardware of the electronic device 100 based on the first model information and the second model information on at least one neural network model included in the electronic device 100 . At least one neural network model suitable for replacing the neural network model included in the electronic device 100 from among the neural network models may be identified ( S1060 ). Since the hardware suitability identification process and the model suitability identification process have been described in detail with reference to FIGS. 1 to 8 , a redundant description of the specific content will be omitted.

When the hardware suitability identification process and the model suitability identification process are performed, the electronic device 100 may receive a second user input ( S1070 ), and may first transmit installation data of at least one neural network model selected according to the second user input ( S1070 ). The request may be made to the external device 200-1 (S1080). When installation data of the neural network model is requested, the first external device 200 - 1 may transmit installation data of at least one selected neural network model to the electronic device 100 ( S1090 ).

Meanwhile, in the above, an embodiment in which the first external device 200-1 performs a user suitability identification process and the electronic device 100 performs a hardware suitability identification process and a model suitability identification process has been described. It is not limited thereto. That is, according to another embodiment of the present disclosure, the first external device 200 - 1 may perform all of the user suitability identification process, the hardware suitability identification process, and the model suitability identification process according to the present disclosure.

According to various embodiments as described above with reference to FIGS. 9 to 10 , only when it is identified that it is suitable for the user of the electronic device 100 to use the neural network model included in the external device through the user authentication process, the Since it is possible to determine whether the neural network model is transferred, security and reliability can be improved in the process of transferring the neural network model.

11 is a diagram for explaining a user interface provided by the electronic device 100 according to an embodiment of the present disclosure.

11 , the electronic device 100 according to the present disclosure may display a user interface (UI) for receiving a user input on the display of the electronic device 100 . In addition, the electronic device 100 may receive a user input for searching for at least one neural network model included in the at least one first external device 200 - 1 through the user interface. For example, a user input for searching for at least one neural network model may be received according to a user interaction of selecting a 'find' UI element 1110 from among a plurality of UI elements included in the user interface.

According to a user input, the electronic device 100 may transmit a first signal for requesting information related to at least one neural network model included in the at least one first external device 200 - 1 . In response to the first signal, the electronic device 100 provides identification information for the first external device 200-1 and the first external device 200-1 from each of the at least one first external device 200-1. ) can receive identification information for the neural network model included in each. In addition, the electronic device 100 may display the received identification information on the first external device 200 - 1 and identification information on the neural network model in the user interface. For example, the electronic device 100 includes neural network models of models A1, model A2, and model A3 in a first external device 200-1 called device A, and a first external device 200-1 called device B ) that includes neural network models named model B1 and model B2 may be displayed in the user interface.

In particular, the electronic device 100 determines whether each neural network model is a neural network model that satisfies the transitive suitability according to the present disclosure, together with the identification information for the first external device 200-1 and the identification information for the neural network model. The indicated UI element can be displayed in the user interface. For example, the electronic device 100 displays UI elements 1120 and 1130 in the form of a 'checkable check box' to indicate that the models A1 and A2 included in the device A are neural network models that satisfy transitive suitability. can be displayed In addition, the electronic device 100 may display the UI element 1140 in the form of a 'non-checkable check box' to indicate that the model A3 included in the device A is a neural network model that does not satisfy transition suitability. Meanwhile, in the case of the models B1 and B2 included in the device B, the electronic device 100 may display a UI element 1150 to indicate that the transition suitability identification process according to the present disclosure is 'in progress'. have.

After the UI element indicating whether each neural network model is a neural network model that satisfies the transitive suitability according to the present disclosure is displayed, the electronic device 100 is installed in the electronic device 100 among at least one neural network model that satisfies the transitive suitability It may receive user input for selecting one or more neural network models to become. For example, a user input for selecting one or more neural network models to be installed in the electronic device 100 is received according to a user interaction of selecting one UI element 1130 from among the UI elements 1120 and 1130 displayed on the display. can be In this case, the electronic device 100 may display the UI element 1130 in the form of a 'checked check box' as shown in FIG. 11 to indicate that the model A2 is selected.

After one or more neural network models to be installed in the electronic device 100 are selected, when a user input for installing the selected one or more neural network models in the electronic device 100 is received, the electronic device 100 includes the one or more selected neural network models may transmit a request for installation data of one or more selected neural network models to each of the first external devices 200 - 1 . For example, a user input for installing one or more selected neural network models in the electronic device 100 may be received according to a user interaction of selecting an 'install' UI element 1160 from among a plurality of UI elements included in the user interface. have.

In response to a request for installation data of one or more selected neural network models, the electronic device 100 receives installation data of one or more selected neural network models, and installs a corresponding neural network model in the electronic device 100 based on the installation data. can

As described above, an embodiment in which the electronic device 100 displays a UI element indicating whether or not the neural network model satisfies the transitive conformance has been described. In the present disclosure, satisfying the transitive conformance means hardware conformance, model conformance, and user conformance. means that all of the conditions required according to each embodiment are satisfied, and failing to meet the transitive suitability means that at least some of the conditions required according to each embodiment among hardware suitability, model suitability, and user suitability are not satisfied. .

According to the embodiment described above with reference to FIG. 11 , the electronic device 100 installs a neural network model selected by a user from among the neural network models of the first external device 200 - 1 satisfying the transition suitability according to the present disclosure. By requesting and receiving only data from the first external device 200 - 1 , it is possible to improve the efficiency of the neural network model transition process and user convenience.

12 is a diagram for explaining a user interface provided by the first external device 200 - 1 according to an embodiment of the present disclosure.

Although the user interface provided by the electronic device 100 has been described in FIG. 11 , according to another embodiment, the user interface for receiving a user input according to the present disclosure is displayed on the display of the first external device 200 - 1 . may be displayed in The user interface as shown in FIG. 12 represents a user interface for installing at least one application included in the first external device 200 - 1 in the electronic device 100 .

Specifically, the user interface may include a UI element representing applications included in the first external device 200 - 1 . For example, as shown in FIG. 12 , in the user interface, a UI element 1210 indicating a 'call and contact' application, a UI element 1220 indicating a 'message' application, and a UI element indicating an 'AI assistant' application 1230 and a UI element representing the 'gallery' application 1250 may be included.

In addition, the user interface may include a UI element representing a neural network model included in each application included in the first external device 200 - 1 . For example, as shown in FIG. 12 , in the user interface, a UI element 1240 representing a neural network model called 'AI model X' included in the 'AI assistant' application and 'AI model' included in the 'Gallery' application A UI element 1260 representing the neural network model Y' may be included.

The first external device 200 - 1 may receive a user input for selecting at least one application from among a plurality of applications included in the first external device 200 - 1 through a user interface. For example, the first external device 200-1 selects a 'call and contact' application, an 'AI assistant' application, and a 'gallery' application from among a plurality of applications included in the first external device 200-1. It can receive user input. In FIG. 12 , a UI element in the form of a 'checked check box' indicates that it is a selected application.

When a user input for selecting at least one application is received, the first external device 200 - 1 may transmit installation data of the at least one selected application to the electronic device 100 . For example, a user input for selecting at least one application may be received according to a user interaction of selecting a 'send' element 1270 from among a plurality of UI elements included in the user interface.

Meanwhile, the first external device 200 - 1 is included in each of the selected at least one application and the second device information about the hardware specification of the first external device 200 - 1 together with installation data of the selected at least one application. The first model information on the obtained neural network model may be transmitted to the electronic device 100 . In addition, the electronic device 100 based on the second device information and the first model information received from the first external device 200 - 1 together with the first device information and the second model information stored in the electronic device 100 . As described above, the hardware suitability identification process and the model suitability identification process may be performed. Thereafter, when a request for installation data of a neural network model identified as satisfying hardware suitability and model suitability is received from the electronic device 100 , the first external device 200 - 1 transmits the identified neural network model installation data may be transmitted to the electronic device 100 .

Meanwhile, at least some of the installation data of the application and the installation data of the neural network model may be directly transmitted from the first external device 200-1 to the electronic device 100, and the installation data of the application or the installation data of the neural network model Of course, it may be transmitted to the electronic device 100 through a server that provides

According to the embodiment as described above with reference to FIG. 12 , the first external device 200 - 1 transmits the installation data of the application selected by the user to the electronic device 100 , and at the same time, the neural network included in the selected application After performing the process of identifying the transitional suitability of the model, the installation data of the neural network model may be transmitted from the first external device 200 - 1 to the electronic device 100 , and thus user convenience may be further improved. In particular, in the embodiment of FIG. 12 , in the initial setting stage of electronic autonomy, which is a new device, at least part of the information on the application and neural network model included in the first external device 200-1, which is the old device, is transferred to the electronic device as the new device. It can be applied to the case for collectively transmitting to (100).

13 is a diagram illustrating an example of the electronic device 100, the first external device 200-1, and the second external device 200-2 according to the present disclosure. And, FIG. 14 is a diagram for explaining a process of identifying transition suitability by the electronic device 100 when the neural network model included in the first external device 200-1 is transferred to the second external device 200-2. It is a sequence diagram.

13 , the electronic device 100 according to the present disclosure may be implemented as a smart phone, the first external device 200-1 may be implemented as a robot cleaner, and the second external device 200- 2) can be implemented as an intelligent companion robot. That is, the types of the electronic device 100 , the first external device 200 - 1 , and the second external device 200 - 2 according to the present disclosure may be different from each other. In the present disclosure, the term 'the first external device 200 - 1' is used as a term for specifying an external device capable of transmitting the installation data of the neural network model to the electronic device 100, while the term 'second external device' The term '(200-2)' is used as a term for specifying an external device capable of receiving installation data of the neural network model from the first external device 200-1.

Meanwhile, when the neural network model included in the first external device 200-1 is transferred to the second external device 200-2, as shown in FIG. 13, the first external device 200-1 and the second When the external device 200 - 2 does not include a display, since the user interface as described above with reference to FIGS. 11 and 12 cannot be displayed, a user input for performing an embodiment according to the present disclosure is received. The problem is that it is difficult to do. In addition, when the first external device 200-1 and the second external device 200-2 do not include the software module according to the present disclosure, the first external device 200-1 or the second external device 200-1 ( 200-2), there is a problem that the transition suitability identification process according to the present disclosure cannot be performed.

Accordingly, according to an embodiment of the present disclosure, the electronic device 100 receives a user input according to the present disclosure of the first external device 200 - 1 , and the first external device 200 - 1 and the second external device 200 - 1 Based on the information received from (200-2), a hardware suitability identification process and a model suitability identification process may be performed.

Specifically, as shown in FIG. 14 , the electronic device 100 may receive a first user input (S1410). And, according to the first user input, the electronic device 100 receives first device information on the hardware specification of the first external device 200-1 and at least one neural network included in the first external device 200-1. The request for the first model information for the model is transmitted to the first external device 200-1 (S1420-1), and the second device information and the second device information for the hardware specification of the second external device 200-2 are transmitted (S1420-1). A request for second model information on at least one neural network model included in the external device 200-2 may be transmitted to the second external device 200-2 (S1420-2).

Unlike the description of FIGS. 1 to 12 , in the description of FIGS. 13 and 14 , the first device information is information on the hardware specification of the first external device 200-1, and the second device information is the second device information. 2 Information on the hardware specification of the external device 200 - 2 , the first model information includes information on at least one neural network model included in the first external device 200 - 1 , and the second model information includes the second model information It is used as a term for specifying information on at least one neural network model included in the external device 200 - 2 .

In response to the request received from the electronic device 100 , the first external device 200 - 1 may transmit the first device information and the first model information to the electronic device 100 ( S1430-1 ), and the second The external device 200 - 2 may transmit the second device information and the second model information to the electronic device 100 ( S1430 - 2 ).

When the first device information and the first model information are received from the first external device 200-1 and the second device information and the second model information are received from the second external device 200-2, the electronic device 100 is at least one suitable for hardware of the second external device 200-2 among at least one neural network model included in the first external device 200-1 based on the first device information, the second device information, and the first model information. One neural network model may be identified (S1440). Furthermore, the electronic device 100 includes a second external device 200-2 among at least one neural network model identified as being suitable for the hardware of the first external device 200-1 based on the first model information and the second model information. At least one neural network model suitable for replacing the neural network model included in may be identified (S1450). In the embodiment described with reference to FIGS. 13 and 14 , the hardware suitability identification process and the model suitability identification process as described above may be similarly applied.

When the hardware suitability identification process and the model suitability identification process are performed, the electronic device 100 may receive a second user input (S1460). Then, the electronic device 100 may transmit a request for installation data of at least one neural network model selected according to the second user input to the first external device 200 - 1 ( S1470 ). When a request for installation data of at least one selected neural network model is received, the first external device 200 - 1 may transmit installation data of the selected at least one neural network model to the second external device 200 - 2 ( S1480). Meanwhile, the first external device 200 - 1 transmits installation data of at least one selected neural network model to the electronic device 100 , and the electronic device 100 transmits the received installation data to the second external device 200 - 2 . ) can also be sent.

According to the embodiment described above with reference to FIGS. 13 and 14 , the electronic device 100 functions as an intermediary device between the first external device 200-1 and the second external device, and does not include a display. It is possible to improve the efficiency and reliability of the process of transferring the neural network model between external devices that do not include the software module or the software module according to the present disclosure.

15 is a block diagram illustrating in detail an architecture of a software module included in the electronic device 100 according to the present disclosure.

As shown in FIG. 15 , the electronic device 100 according to the present disclosure may include a software module such as a neural network model installation module 1500 and a neural network model performance monitor. In addition, the neural network model installation module may include a conformance identification module, a neural network model conversion module, and a neural network model fine-tuning module, and the neural network model performance monitor may include a personalization level monitor and a hardware requirement monitor.

The 'conformity identification module' refers to a module of a concept that collectively refers to modules capable of identifying transition conformance according to the present disclosure. Specifically, the suitability identification module may include the hardware suitability identification module 1100 , the model suitability identification module 1200 and the user suitability identification module 1300 as described above. In addition, the hardware suitability identification module 1100 , the model suitability identification module 1200 , and the user suitability identification module 1300 include the first device information and the second model information stored in the electronic device 100 and the second received from the external device. Based on the device information and the first model information, the hardware suitability identification process, the model suitability identification process, and the user suitability identification process according to various embodiments of the present disclosure may be performed, respectively. Meanwhile, in FIG. 15 , user information is illustrated in a form included in the first device information, but this is only related to an exemplary embodiment. Specific operations of the hardware suitability identification module 1100 , the model suitability identification module 1200 , and the user suitability identification module 1300 have been described above with reference to FIGS. 1 to 14 , so a redundant description will be omitted.

The 'neural network model conversion module' refers to a module for converting a neural network model included in the electronic device 100 into a neural network model included in an external device based on installation data of the neural network model received from the external device. Specifically, the neural network model conversion module includes the structure and type of the neural network included in the neural network model included in the installation data, the number of layers included in the neural network, the number of nodes for each layer, the weight value of each node, and a plurality of A neural network model included in the electronic device 100 may be converted into a neural network model included in an external device based on information on a connection relationship between nodes.

Here, 'conversion' of the neural network model included in the electronic device 100 to the neural network model included in the external device means not only replacing the entire neural network model of the electronic device 100 with the entire neural network model of the external device, but also As in the case of changing only the weight values of nodes included in the neural network model of the electronic device 100 to the weights of nodes included in the neural network model of the external device, a part of the neural network model of the electronic device 100 is converted to a part of the neural network model of the external device. Substitution may be included.

The 'neural network model fine-tuning module' refers to a module that performs fine-tuning on details after converting a neural network model included in the electronic device 100 into a neural network model included in an external device. Specifically, the neural network model fine-tuning module reflects the detailed difference between the hardware specification of the electronic device 100 and the hardware specification of the external device to determine the number of nodes for each layer included in the neural network model, the weight value of each node, etc. By adjusting, the neural network model may be more suitable for the electronic device 100 . Also, the neural network model fine-tuning module may adjust parameters related to personalization of the neural network model based on personalization data received from an external device. As described above, 'personalized data' refers to learning data used for personalization of a neural network model included in an external device.

On the other hand, when the neural network model is switched/adjusted according to the neural network model conversion module and the neural network model fine adjustment module, information about it may be stored/updated as second model information.

The 'personalization level monitor' refers to a module for monitoring the personalization level of neural network models included in the electronic device 100 . In particular, when the second neural network model included in the external device is installed in a state where the first neural network model included in the electronic device 100 is personalized to some extent, the personalization level monitor provides information on the personalization level of the first neural network model. can be obtained and transmitted to the conformance identification module. Then, the conformance identification module compares the information on the personalization level of the first neural network model received from the personalization level monitor with the information on the personalization level of the second neural network model received from the external device so that the second neural network model is the first It can identify whether it is suitable as a replacement for neural network models.

The 'hardware requirement specification monitor' refers to a module for monitoring hardware specifications required to execute the neural network model included in the electronic device 100 . Specifically, the hardware requirement specification monitor may acquire information on hardware specifications required to execute each of the at least one neural network model included in the electronic device 100 , which may be stored as information included in the second model information. have. On the other hand, 'hardware specification information' included in the first device information is information about the hardware specification at the time of product release of the electronic device 100 or the hardware specification at the time of model suitability determination, and is information about the neural network model acquired through the hardware requirement specification monitor. It is distinct from information about hardware requirements.

Although the software module included in the electronic device 100 has been described above, this is only an embodiment according to the present disclosure, and a new configuration may be added or some modules may be omitted in addition to the illustrated module. to be. In addition, at least two or more of the software modules according to the present disclosure may be implemented as one module in an integrated form.

16 is a block diagram illustrating in detail an architecture of a software module included in the first external device 200 - 1 according to the present disclosure.

As shown in FIG. 16 , the first external device 200 - 1 may include a software module such as a neural network model installation module 1600 and a neural network model performance monitor. In addition, the neural network model installation module 1600 may include a neural network model description information management module 1610 and a neural network model installation information management module 1620 , and the neural network model performance monitor includes a personalization level monitor 1710 and hardware requirements. A monitor 1720 may be included.

The 'neural network model description information management module 1610' refers to a module that collectively manages information necessary for determining transition suitability for a neural network model included in the first external device 200 - 1 . Specifically, the neural network model description information management module 1610 may transmit the second device information and the first model information to the electronic device 100 .

As described above, the second device information is information on the hardware specification of the first external device 200 - 1 , specifically, what kind of performance each hardware included in the first external device 200 - 1 has. It is a term used to collectively refer to information on specifications that indicate And, the first model information is information on at least one neural network model included in the first external device 200-1, and specifically, at least one neural network model included in the first external device 200-1. It may include information on each service type, information on personalization level, and information on hardware requirements.

In addition, the neural network model description information management module 1610 may collectively manage various information such as identification information for each neural network model included in the first external device 200 - 1 , information on performance and version, and the like.

The 'neural network model installation information management module 1620' refers to a module for managing installation data of a neural network model. Specifically, the neural network model installation information management module 1620 may transmit installation data of at least one neural network model to the electronic device 100 in response to a request for installation data received from the electronic device 100 . In particular, the installation data may include identification information for each of the at least one neural network model and configuration information necessary for installing the identified at least one neural network model. Here, 'configuration information' refers to the structure and type of the neural network included in the neural network model, the number of layers included in the neural network, the number of nodes for each layer, the weight value of each node, connection relationships between a plurality of nodes, etc. may include information about

Also, the neural network model installation information management module 1620 may transmit personalization data for at least one neural network model together with installation data of the at least one neural network model to the electronic device 100 .

The 'personalization level monitor 1710' refers to a module for monitoring the personalization level of neural network models included in the first external device 200 - 1 . In particular, the personalization level monitor 1710 may output information on the personalization level of the neural network models included in the first external device 200-1 based on the information on the user's history and the user's feedback. can Here, the information on the user's history may include information on the number, frequency, and period of use of the neural network model by the user, and the information on the user's feedback is about the results of the use of the neural network model after the user uses the neural network model. It has been described above that the input direct evaluation information may include indirect evaluation information related to a result of its use.

The 'hardware requirement specification monitor 1720' refers to a module for monitoring hardware specifications required to execute the neural network model included in the first external device 200 - 1 . Specifically, the hardware requirement specification monitor 1720 may acquire information on hardware specifications required to execute each of the at least one neural network model included in the first external device 200 - 1 , which is based on the first model information. may be stored as embedded information.

Although the software module included in the first external device 200-1 has been described above, this is only an embodiment according to the present disclosure, and a new configuration may be added or some modules may be omitted in addition to the illustrated module. Of course you can. In addition, at least two neural network models among the software modules according to the present disclosure may be implemented as one neural network model in an integrated form.

17 is a block diagram schematically illustrating an architecture of a hardware configuration included in the electronic device 100 according to the present disclosure. And, FIG. 18 is a block diagram illustrating in more detail an architecture of a hardware configuration included in the electronic device 100 according to the present disclosure.

17 , the electronic device 100 according to an embodiment of the present disclosure includes a communication unit 110 , a memory 120 , and a processor 130 . Also, as shown in FIG. 18 , the electronic device 100 according to an embodiment of the present disclosure may further include a data acquisition unit 140 and a data output unit 150 . However, the configurations shown in FIGS. 17 and 18 are merely exemplary, and in implementing the present disclosure, a new configuration may be added or some configuration may be omitted in addition to the configuration shown in FIGS. 17 and 18 . of course there is

The communication unit 110 includes a circuit and may communicate with an external device. Specifically, the processor 130 may receive various data or information from an external device connected through the communication unit 110 , and may transmit various data or information to the external device.

The communication unit 110 may include at least one of a WiFi module, a Bluetooth module, a wireless communication module, and an NFC module. Specifically, each of the WiFi module and the Bluetooth module may perform communication using a WiFi method and a Bluetooth method. In the case of using a WiFi module or a Bluetooth module, various types of connection information, such as an SSID, are first transmitted and received, and various types of information can be transmitted/received after communication connection using this.

In addition, the wireless communication module may perform communication according to various communication standards such as IEEE, Zigbee, 3rd Generation (3G), 3rd Generation Partnership Project (3GPP), Long Term Evolution (LTE), 5th Generation (5G), and the like. In addition, the NFC module may perform communication using a Near Field Communication (NFC) method using a 13.56 MHz band among various RF-ID frequency bands such as 135 kHz, 13.56 MHz, 433 MHz, 860 to 960 MHz, and 2.45 GHz.

In particular, in various embodiments of the present disclosure, the communication unit 110 includes a first signal for requesting information related to at least one neural network model included in the first external device 200 - 1 and at least one neural network model. A third signal including a request for installation data may be transmitted to the first external device 200 - 1 . In addition, in response to the first signal, the communication unit 110 receives from the first external device 200-1 second device information about the hardware specification of the first external device 200-1 and the first external device 200 It is possible to receive a second signal including first model information for at least one neural network model included in -1), and in response to the third signal, including installation data of at least one identified neural network model A fourth signal may be received.

At least one instruction related to the electronic device 100 may be stored in the memory 120 . In addition, an operating system (O/S) for driving the electronic device 100 may be stored in the memory 120 . In addition, various software programs or applications for operating the electronic device 100 according to various embodiments of the present disclosure may be stored in the memory 120 . In addition, the memory 120 may include a semiconductor memory such as a flash memory or a magnetic storage medium such as a hard disk.

Specifically, various software modules for operating the electronic device 100 may be stored in the memory 120 according to various embodiments of the present disclosure, and the processor 130 executes various software modules stored in the memory 120 . Thus, the operation of the electronic device 100 may be controlled. That is, the memory 120 is accessed by the processor 130 , and reading/writing/modification/deletion/update of data by the processor 130 may be performed.

Meanwhile, in the present disclosure, the term memory 120 refers to the memory 120 , a ROM (not shown) in the processor 130 , a RAM (not shown), or a memory card (not shown) mounted in the electronic device 100 (eg, For example, micro SD card, memory stick) may be used in the sense of including.

In particular, according to various embodiments of the present disclosure, in the memory 120 , first device information on a hardware specification of the electronic device 100 and a second model on at least one neural network model included in the electronic device 100 . Information may be stored. In addition, various modules as described above with reference to FIG. 15 may be stored in the memory 120 , including the hardware suitability identification module 1100 , the model suitability identification module 1200 , and the user suitability identification module 1300 . In addition, various information necessary within the scope for achieving the object of the present disclosure may be stored in the memory 120, and the information stored in the memory 120 may be updated as received from an external device or input by a user. .

The processor 130 controls the overall operation of the electronic device 100 . Specifically, the processor 130 is connected to the configuration of the electronic device 100 including the communication unit 110 and the memory 120 as described above, and receives at least one instruction stored in the memory 120 as described above. By executing it, the overall operation of the electronic device 100 may be controlled.

The processor 130 may be implemented in various ways. For example, the processor 130 may include an application specific integrated circuit (ASIC), an embedded processor, a microphone 142, a processor, hardware control logic, a hardware finite state machine (FSM), and a digital signal. It may be implemented by at least one of a processor (Digital Signal Processor, DSP). Meanwhile, in the present disclosure, the term processor 130 may be used to include a central processing unit (CPU), a graphic processing unit (GPU), a main processing unit (MPU), and the like.

In particular, in various embodiments of the present disclosure, when a user input is received, the processor 130 transmits a first signal for requesting information related to at least one neural network model included in at least one external device to the communication unit. control 110 and, in response to the first signal, from the first external device 200-1 among at least one external device through the communication unit 110 to the hardware specification of the first external device 200-1 Receives a second signal including second device information about the device and first model information on at least one neural network model included in the first external device 200-1, and receives the first device information, the second device information, and the second signal 1 By inputting model information into the hardware suitability identification module 1100 , it is identified whether each of the at least one neural network model included in the first external device 200 - 1 is suitable for the hardware of the electronic device 100 , Controls the communication unit 110 to transmit a third signal including a request for installation data of at least one neural network model identified as suitable for the hardware of the device 100 to the first external device 200-1, 3 In response to the signal, a fourth signal including installation data of at least one neural network model identified as suitable for the hardware of the electronic device 100 from the first external device 200-1 through the communication unit 110 can receive

In addition, when at least one neural network model suitable for the hardware of the electronic device 100 is identified, the processor 130 inputs the first model information and the second model information to the model suitability identification module 1200, and the electronic device ( It may also be identified whether each of the at least one neural network model identified as suitable for the hardware of 100 ) is suitable to replace the neural network model included in the electronic device 100 . Various embodiments according to the present disclosure based on the control of the processor 130 have been described above with reference to FIGS. 1 to 16 , and thus a redundant description thereof will be omitted.

The data acquisition unit 140 includes a circuit, and the processor 130 may acquire various types of data used in the electronic device 100 through the data acquisition unit 140 . Specifically, the data acquisition unit 140 may include a camera 141 , a microphone 142 , and a sensor 143 .

The camera 141 may acquire an image of at least one object. Specifically, the camera 141 may include an image sensor, and the image sensor may convert light entering through a lens into an electrical image signal. In addition, the microphone 142 may receive a voice signal and convert the received voice signal into an electrical signal.

The sensor 143 may sense various information inside and outside the electronic device 100 . Specifically, the sensor 143 is a GPS (Global Positioning System) sensor, a gyro sensor (gyroscope), an acceleration sensor (acceleration sensor, accelerometer), a lidar sensor (lidar sensor), an inertial sensor (Inertial Measurement Unit, IMU) ) and may include at least one of a motion sensor. In addition, the sensor 143 may include various types of sensors such as a temperature sensor, a humidity sensor, an infrared sensor, and a biosensor.

In particular, according to various embodiments of the present disclosure, the data acquisition unit 140 may acquire data input to at least one neural network model included in the electronic device 100 . For example, the processor 130 may acquire image data input to a neural network model related to object recognition through the camera 141 , and receives a voice signal input to a neural network model related to voice recognition through the microphone 142 . may be obtained. In addition, the processor 130 may acquire location information input to a neural network model related to autonomous driving through at least one sensor 143 among a GPS sensor and a lidar sensor.

The data output unit 150 includes a circuit, and the processor 130 may output various functions that the electronic device 100 can perform through the data output unit 150 . Specifically, the data output unit 150 may include at least one of a display 151 , a speaker 152 , and an indicator 153 .

The display 151 may output image data. Specifically, the display 151 may display an image or a user interface stored in the memory 120 under the control of the processor 130 . The display 151 may be implemented as a liquid crystal display panel (LCD), organic light emitting diodes (OLED), or the like, and the display 151 may be implemented as a flexible display, a transparent display, etc. in some cases. However, the display 151 according to the present disclosure is not limited to a specific type. Meanwhile, the display 151 may be implemented in the form of a touch display and configured to receive a user's touch interaction. Meanwhile, the speaker 152 may output audio data under the control of the processor 130 , and the indicator 153 may be lit under the control of the processor 130 .

In particular, according to various embodiments of the present disclosure, the display 151 may display a user interface including information on a neural network model that satisfies hardware conformance and model conformance according to the present disclosure. In addition, the processor 130 may receive a user input for selecting one or more neural network models from among the one or more neural network models through the user interface.

On the other hand, if it is identified that there is no neural network model that satisfies hardware suitability and model suitability among neural network models included in the external device, the processor 130 determines that all neural network models included in the external device meet the transitive suitability according to the present disclosure. You can also output a user notification to indicate that you can't. Here, the user notification may be output in the form of visual information through the display 151 as well as in the form of voice information through the speaker 152 or may be output in such a way that the indicator 153 is turned on. .

Meanwhile, the control method of the electronic device 100 according to the above-described embodiment may be implemented as a program and provided to the electronic device 100 . In particular, a program including a control method of the electronic device 100 may be stored and provided in a non-transitory computer readable medium.

Specifically, in a non-transitory computer-readable recording medium including a program for executing a control method of the electronic device 100 , the control method of the electronic device 100 is included in at least one external device when a user input is received transmitting a first signal for requesting information related to the at least one neural network model, in response to the first signal, from the first external device 200-1 among at least one external device ( Receiving a second signal including second device information on the hardware specification of 200-1) and first model information on at least one neural network model included in the first external device 200-1, a first By inputting the device information, the second device information, and the first model information to the hardware suitability identification module 1100 , each of the at least one neural network model included in the first external device 200 - 1 is the hardware of the electronic device 100 . Transmitting a third signal including a request for installation data of at least one neural network model identified as suitable for hardware of the electronic device 100 to the first external device 200 - 1 and in response to the third signal, receiving a fourth signal including installation data of at least one neural network model identified as suitable for the hardware of the electronic device 100 from the first external device 200-1 may include steps.

And, in the identifying step, if the specification of each of the plurality of hardware components included in the electronic device 100 is equal to or greater than the specification of the plurality of hardware components included in the first external device 200-1, the first external device 200- The step of identifying that at least one neural network model included in 1) is suitable for the hardware of the electronic device 100 and the specification of at least one of a plurality of hardware configurations included in the electronic device 100 is determined by the first external device 200- If it is less than the specification of a plurality of hardware components included in 1), a hardware requirement lower than the specification of a plurality of hardware components included in the electronic device 100 among at least one neural network model included in the first external device 200-1 It may include identifying one or more neural network models having the specification as suitable for the hardware of the electronic device 100 .

Here, the non-transitory readable medium refers to a medium that stores data semi-permanently, rather than a medium that stores data for a short moment, such as a register, cache, memory 120, and the like, and can be read by a device. Specifically, the various applications or programs described above may be provided by being stored in a non-transitory readable medium such as a CD, DVD, hard disk, Blu-ray disk, USB, memory 120 card, ROM, and the like.

In the above, the control method of the electronic device 100 and the computer-readable recording medium including the program for executing the control method of the electronic device 100 have been briefly described, but this is only for omitting redundant description, and Of course, various embodiments of the device 100 may be applied to a computer-readable recording medium including a control method of the electronic device 100 and a program for executing the control method of the electronic device 100 .

According to various embodiments of the present disclosure as described above, when a neural network model is transferred from an external device, the electronic device 100 identifies whether it is appropriate to transfer the neural network model to the electronic device 100 and identifies the neural network model. It is possible to determine whether to transfer , and thus the efficiency and reliability of transfer learning can be remarkably improved.

Meanwhile, the functions related to the neural network model as described above may be performed through the memory 120 and the processor 130 . The processor 130 may include one or a plurality of processors 130 . At this time, one or a plurality of processors 130 is a general-purpose processor 130, such as a CPU, AP, GPU. It may be a graphics-only processor 130 such as a VPU or the like, or an artificial intelligence-only processor 130 such as an NPU. One or more processors 130 control to process input data according to a non-volatile memory 120 and a predefined operation rule or artificial intelligence model stored in the volatile memory 120 . The predefined action rule or artificial intelligence model is characterized in that it is created through learning.

Here, being made through learning means that a predefined operation rule or artificial intelligence model of a desired characteristic is created by applying a learning algorithm to a plurality of learning data. Such learning may be performed in the device itself on which the artificial intelligence according to the present disclosure is performed, or may be performed through a separate server/system.

The artificial intelligence model may be composed of a plurality of neural network layers. Each layer has a plurality of weight values, and the calculation of the layer is performed through the operation of the previous layer and the operation of the plurality of weights. Examples of neural networks include Convolutional Neural Network (CNN), Deep Neural Network (DNN), Recurrent Neural Network (RNN), Restricted Boltzmann Machine (RBM), Deep Belief Network (DBN), Bidirectional Recurrent Deep Neural Network (BRDNN), GAN. There are (Generative Adversarial Networks) and Deep Q-Networks, and the neural network in the present disclosure is not limited to the above-described examples, except as otherwise specified.

The learning algorithm is a method of training a predetermined target device (eg, a robot) using a plurality of learning data so that the predetermined target device can make a decision or make a prediction by itself. Examples of the learning algorithm include supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, and the learning algorithm in the present disclosure is specified It is not limited to the above-described example except for.

The device-readable storage medium may be provided in the form of a non-transitory storage medium. Here, 'non-transitory storage medium' is a tangible device and only means that it does not contain a signal (eg, electromagnetic wave), and this term refers to cases in which data is semi-permanently stored in a storage medium and temporary It does not distinguish the case where it is stored as For example, the 'non-transitory storage medium' may include a buffer in which data is temporarily stored.

According to an embodiment, the method according to various embodiments disclosed in this document may be provided by being included in a computer program product. Computer program products may be traded between sellers and buyers as commodities. The computer program product is distributed in the form of a machine-readable storage medium (eg compact disc read only memory (CD-ROM)), or through an application store (eg Play Store ^TM ) or on two user devices ( It can be distributed (eg downloaded or uploaded) directly, online between smartphones (eg: smartphones). In the case of online distribution, at least a portion of a computer program product (eg, a downloadable app) is stored at least in a machine-readable storage medium such as a memory of a manufacturer's server, a server of an application store, or a relay server. It may be temporarily stored or temporarily created.

As described above, each of the components (eg, a module or a program) according to various embodiments of the present disclosure may be composed of a singular or a plurality of entities, and some of the above-described corresponding sub-components are omitted. Alternatively, other sub-components may be further included in various embodiments. Alternatively or additionally, some components (eg, a module or a program) may be integrated into a single entity to perform the same or similar functions performed by each corresponding component prior to integration.

According to various embodiments, operations performed by a module, program, or other component may be sequentially, parallel, repetitively or heuristically executed, or at least some operations may be executed in a different order, omitted, or other operations may be added. can

Meanwhile, the term “unit” or “module” used in the present disclosure includes a unit composed of hardware, software, or firmware, and may be used interchangeably with terms such as, for example, logic, logic block, part, or circuit. can A “unit” or “module” may be an integrally formed component or a minimum unit or a part of one or more functions. For example, the module may be configured as an application-specific integrated circuit (ASIC).

Various embodiments of the present disclosure may be implemented as software including instructions stored in a machine-readable storage medium readable by a machine (eg, a computer). The device calls the stored instructions from the storage medium. and an electronic device (eg, the electronic device 100 ) according to the disclosed embodiments as a device capable of operating according to the called command.

When the instruction is executed by the processor 130 , the processor 130 may perform a function corresponding to the instruction by using other components directly or under the control of the processor 130 . Instructions may include code generated or executed by a compiler or interpreter.

In the above, preferred embodiments of the present disclosure have been illustrated and described, but the present disclosure is not limited to the specific embodiments described above, and it is common in the technical field to which the disclosure pertains without departing from the gist of the present disclosure as claimed in the claims. Various modifications may be made by those having the knowledge of, of course, and these modifications should not be individually understood from the technical spirit or prospect of the present disclosure.

100: electronic device 110: communication unit
120: memory 130: processor
140: data acquisition unit 150: data output unit
1000: conformance identification module 1100: hardware conformance identification module
1200: model conformance identification module 1300: user conformance identification module

Claims (19)

An electronic device for identifying the transitive suitability of a neural network model included in an external device, the electronic device comprising:
communication department;
a memory for storing first device information on the hardware specification of the electronic device and a hardware conformance identification module for identifying a neural network model suitable for the hardware of the electronic device; and
When a user input is received, the communication unit is controlled to transmit a first signal for requesting information related to at least one neural network model included in at least one external device,
In response to the first signal, from a first external device among the at least one external device through the communication unit, second device information about the hardware specification of the first external device and at least one of the first external devices included in the first external device Receive a second signal including first model information for a neural network model of
The first device information, the second device information, and the first model information are input to the hardware suitability identification module to determine whether each of the at least one neural network model included in the first external device is suitable for the hardware of the electronic device. to identify whether
controlling the communication unit to transmit a third signal including a request for installation data of at least one neural network model identified as suitable for hardware of the electronic device to the first external device,
a processor for receiving a fourth signal including installation data of at least one neural network model identified as suitable for hardware of the electronic device from the first external device through the communication unit in response to the third signal; includes,
The processor is
When the specification of each of the plurality of hardware components included in the electronic device is greater than or equal to the specification of the plurality of hardware components included in the first external device, the at least one neural network model included in the first external device is the identified as suitable for the hardware;
When the specification of at least one of the plurality of hardware configurations included in the electronic device is less than the specification of the plurality of hardware configurations included in the first external device, the at least one neural network model included in the first external device An electronic device for identifying that one or more neural network models having a hardware requirement specification lower than a specification of a plurality of hardware components included in the electronic device are suitable for hardware of the electronic device.
According to claim 1,
The first device information includes information on a specification of the processor, a specification of the memory, and a specification of a data acquisition unit included in the electronic device,
The data acquisition unit included in the electronic device acquires data input to at least one neural network model included in the electronic device, and includes at least one of a camera, a microphone, and a sensor included in the electronic device,
The second device information includes information on a specification of a processor included in the first external device, a specification of a memory included in the first external device, and a specification of a data acquisition unit included in the first external device,
The data acquisition unit included in the first external device acquires data input to at least one neural network model included in the first external device, and includes at least one of a camera, a microphone, and a sensor included in the first external device and,
The first model information includes information on hardware requirements for each of the at least one neural network model included in the first external device.
3. The method of claim 2,
The memory stores second model information on at least one neural network model included in the electronic device and a model suitability identification module for identifying a neural network model suitable for replacing the neural network model included in the electronic device,
The processor is
When at least one neural network model suitable for the hardware of the electronic device is identified, the first model information and the second model information are input to the model suitability identification module, and at least one neural network model identified as suitable for the hardware of the electronic device is identified. Identifies whether each neural network model is suitable to replace a neural network model included in the electronic device,
The third signal is
including a request for installation data of at least one neural network model identified as suitable for replacing the neural network model included in the electronic device;
The fourth signal is
An electronic device including installation data of at least one neural network model identified as suitable for replacing a neural network model included in the electronic device.
4. The method of claim 3,
The first model information further includes information on a service type and personalization level of each of the at least one neural network model included in the first external device,
The second model information includes information on a service type of each of at least one neural network model included in the electronic device and information on a personalization level.
5. The method of claim 4,
The processor is
At least one service type of at least one neural network model included in the electronic device and at least one identified as being suitable for hardware of the electronic device based on the information on the service type included in each of the first model information and the second model information Compare the service types of neural network models of
When a service type of a first neural network model among at least one neural network model identified as suitable for hardware of the electronic device is the same as a service type of a second neural network model among a plurality of neural network models included in the electronic device, the first model comparing the personalization level of the first neural network model and the personalization level of the second neural network model based on the information on the personalization level included in each of the information and the second model information,
If the personalization level of the first neural network model is higher than the personalization level of the second neural network model, the electronic device identifies that the first neural network model is suitable to replace the second neural network model.
6. The method of claim 5,
The personalization level of the first neural network model may be selected from among information on a user's history of use of the first external device for the first neural network model and information on feedback of a user of the first external device for the first neural network model. identified based on at least one;
The personalization level of the second neural network model is based on at least one of information about a user's history of use of the electronic device for the second neural network model and information about a feedback of a user of the electronic device with respect to the second neural network model An electronic device identified as
According to claim 1,
The second signal is received from the first external device through the communication unit as user authentication is completed based on the first user information on the electronic device and the second user information on the first external device,
The first user information includes at least one of account information for a user of the electronic device and identification information for the electronic device,
The second user information includes at least one of account information for a user of the first external device and identification information for the first external device.
According to claim 1,
display; further comprising,
The processor is
controlling the display to display a user interface including information on at least one neural network model identified as suitable for hardware of the electronic device;
receiving a user input for selecting one or more neural network models from among at least one neural network model identified as suitable for hardware of the electronic device through the user interface;
The third signal is
Including a request for installation data of the selected one or more neural network models,
The fourth signal is
An electronic device including installation data of the one or more selected neural network models.
The first device information on the hardware specification of the electronic device and the hardware suitability identification module for identifying the neural network model suitable for the hardware of the electronic device are stored, and the control of the electronic device for identifying the transfer suitability of the neural network model included in the external device In the method,
when a user input is received, transmitting a first signal for requesting information related to at least one neural network model included in at least one external device;
In response to the first signal, from a first external device among the at least one external device, second device information on a hardware specification of the first external device and at least one neural network model included in the first external device Receiving a second signal including the first model information about the;
The first device information, the second device information, and the first model information are input to the hardware suitability identification module to determine whether each of the at least one neural network model included in the first external device is suitable for the hardware of the electronic device. identifying whether or not;
transmitting a third signal including a request for installation data of at least one neural network model identified as suitable for hardware of the electronic device to the first external device; and
receiving, from the first external device, a fourth signal including installation data of at least one neural network model identified as being suitable for hardware of the electronic device in response to the third signal; including,
The identifying step is
When the specification of each of the plurality of hardware components included in the electronic device is greater than or equal to the specification of the plurality of hardware components included in the first external device, the at least one neural network model included in the first external device is the identifying as suitable for the hardware; and
When the specification of at least one of the plurality of hardware components included in the electronic device is less than the specification of the plurality of hardware components included in the first external device, the electronic device among the at least one neural network model included in the first external device identifying one or more neural network models having hardware requirements lower than those of a plurality of hardware components included in the device as being suitable for hardware of the electronic device; A control method of an electronic device comprising a.
10. The method of claim 9,
The first device information includes information on a specification of a processor included in the electronic device, a specification of a memory included in the electronic device, and a specification of a data acquisition unit included in the electronic device,
The data acquisition unit included in the electronic device acquires data input to at least one neural network model included in the electronic device, and includes at least one of a camera, a microphone, and a sensor included in the electronic device,
The second device information includes information on a specification of a processor included in the first external device, a specification of a memory included in the first external device, and a specification of a data acquisition unit included in the first external device,
The data acquisition unit included in the first external device acquires data input to at least one neural network model included in the first external device, and includes at least one of a camera, a microphone, and a sensor included in the first external device and,
The first model information includes information on hardware requirements for each of the at least one neural network model included in the first external device.
11. The method of claim 10,
The electronic device further stores second model information on at least one neural network model included in the electronic device and a model suitability identification module for identifying a neural network model suitable for replacing the neural network model included in the electronic device,
The control method of the electronic device,
When at least one neural network model suitable for the hardware of the electronic device is identified, the first model information and the second model information are input to the model suitability identification module, and at least one neural network model identified as suitable for the hardware of the electronic device is identified. identifying whether each neural network model is suitable to replace a neural network model included in the electronic device; further comprising,
The third signal is
including a request for installation data of at least one neural network model identified as suitable for replacing the neural network model included in the electronic device;
The fourth signal is
A method of controlling an electronic device including installation data of at least one neural network model identified as suitable for replacing a neural network model included in the electronic device.
12. The method of claim 11,
The first model information further includes information on a service type and personalization level of each of the at least one neural network model included in the first external device,
The second model information includes information on a service type of each of at least one neural network model included in the electronic device and information on a personalization level.
13. The method of claim 12,
The step of identifying at least one neural network model suitable for replacing the neural network model included in the electronic device includes:
At least one service type of at least one neural network model included in the electronic device and at least one identified as being suitable for hardware of the electronic device based on the information on the service type included in each of the first model information and the second model information comparing the service types of the neural network models of
When a service type of a first neural network model among at least one neural network model identified as suitable for hardware of the electronic device is the same as a service type of a second neural network model among a plurality of neural network models included in the electronic device, the first model comparing the personalization level of the first neural network model with the personalization level of the second neural network model based on the information and the personalization level information included in each of the second model information; and
if the personalization level of the first neural network model is higher than the personalization level of the second neural network model, identifying that the first neural network model is suitable to replace the second neural network model; A control method of an electronic device comprising a.
14. The method of claim 13,
The personalization level of the first neural network model may be selected from among information on a user's history of use of the first external device for the first neural network model and information on feedback of a user of the first external device for the first neural network model. identified based on at least one;
The personalization level of the second neural network model is based on at least one of information about a user's history of use of the electronic device for the second neural network model and information about a feedback of a user of the electronic device with respect to the second neural network model A method of controlling an electronic device identified as
10. The method of claim 9,
The second signal is received from the first external device as user authentication is completed based on the first user information on the electronic device and the second user information on the first external device,
The first user information includes at least one of account information for a user of the electronic device and identification information for the electronic device,
The second user information includes at least one of account information for a user of the first external device and identification information for the first external device.
10. The method of claim 9,
The control method of the electronic device,
displaying a user interface including information on at least one neural network model identified as being suitable for hardware of the electronic device; and
receiving a user input for selecting one or more neural network models from among at least one neural network model identified as suitable for hardware of the electronic device through the user interface; further comprising,
The third signal is
Including a request for installation data of the selected one or more neural network models,
The fourth signal is
A method of controlling an electronic device including installation data of the selected one or more neural network models.
The first device information on the hardware specification of the electronic device and the hardware suitability identification module for identifying the neural network model suitable for the hardware of the electronic device are stored, and the control of the electronic device for identifying the transfer suitability of the neural network model included in the external device In a non-transitory computer-readable recording medium comprising a program for executing a method,
The control method of the electronic device,
when a user input is received, transmitting a first signal for requesting information related to at least one neural network model included in at least one external device;
In response to the first signal, from a first external device among the at least one external device, second device information on a hardware specification of the first external device and at least one neural network model included in the first external device Receiving a second signal including the first model information about the;
The first device information, the second device information, and the first model information are input to the hardware suitability identification module to determine whether each of the at least one neural network model included in the first external device is suitable for the hardware of the electronic device. identifying whether or not;
transmitting a third signal including a request for installation data of at least one neural network model identified as suitable for hardware of the electronic device to the first external device; and
receiving a fourth signal including installation data for at least one neural network model identified as suitable for hardware of the electronic device from the first external device in response to the third signal; including,
The identifying step is
When the specification of each of the plurality of hardware components included in the electronic device is greater than or equal to the specification of the plurality of hardware components included in the first external device, the at least one neural network model included in the first external device is the identifying as suitable for the hardware; and
When the specification of at least one of the plurality of hardware components included in the electronic device is less than the specification of the plurality of hardware components included in the first external device, the electronic device among the at least one neural network model included in the first external device identifying one or more neural network models having hardware requirements lower than those of a plurality of hardware components included in the device as being suitable for hardware of the electronic device; A computer-readable recording medium comprising a.
An electronic device for identifying the transitive suitability of a neural network model included in an external device, the electronic device comprising:
communication department;
a memory for storing internal model information on at least one neural network model included in the electronic device and a model suitability identification module for identifying a neural network model suitable for replacing the neural network model included in the electronic device; and
When a user input is received, the communication unit is controlled to transmit a first signal for requesting information related to at least one neural network model included in at least one external device,
In response to the first signal, a second signal including external model information on at least one neural network model included in the first external device is received from a first external device among the at least one external device through the communication unit. and,
By inputting the internal model information and the external model information into the model suitability identification module, it is determined whether each of the at least one neural network model included in the first external device is suitable to replace the neural network model included in the electronic device. identify,
Control the communication unit to transmit a third signal including a request for installation data of at least one neural network model identified as suitable to replace the neural network model included in the electronic device to the first external device,
a processor for receiving a fourth signal including installation data of the identified at least one neural network model from the first external device through the communication unit in response to the third signal; An electronic device comprising a.
The processor is
Based on the information on the service type included in each of the internal model information and the external model information, the service type of at least one neural network model included in the electronic device and the at least one neural network model included in the first external device Compare service types,
When a service type of a first neural network model among at least one neural network model included in the first external device is the same as a service type of a second neural network model among a plurality of neural network models included in the electronic device, the internal model information and the comparing the personalization level of the first neural network model with the personalization level of the second neural network model based on the information on the personalization level included in each of the external model information,
If the personalization level of the first neural network model is higher than the personalization level of the second neural network model, the electronic device identifies that the first neural network model is suitable to replace the second neural network model.
19. The method of claim 18,
The personalization level of the first neural network model may be selected from among information on a user's history of use of the first external device for the first neural network model and information on feedback of a user of the first external device for the first neural network model. identified based on at least one;
The personalization level of the second neural network model is based on at least one of information about a user's history of use of the electronic device for the second neural network model and information about a feedback of a user of the electronic device with respect to the second neural network model An electronic device identified as

Images