KR20210088162A - Electronic device and Method for controlling the electronic device thereof - Google Patents

Abstract

Provided are an electronic device and a controlling method thereof. The electronic device includes: a communication unit including a circuit; a memory for storing at least one instruction; and a processor connected to the communication unit and the memory to control the electronic device, wherein the processor is configured, by executing the at least one instruction, to: acquire information on user data and information on a usage pattern of a user terminal, which are obtained by the user terminal from each of a plurality of user terminals registered in one user account through the communication unit; training a neural network model for determining whether a user normally uses the user terminal based on the acquired information on the user data and the acquired information on the usage pattern; input information related to an access to the user data in the neural network model to determine normal usage of the user terminal when the access to the user data is requested from a first user terminal among the user terminals; and provide a message about abnormal usage of the user terminal when the access to the user data is determined to be the abnormal usage of the user terminal.

Description

Electronic device and method for controlling the same

The present disclosure relates to an electronic device and a control method thereof, and the present disclosure relates to an electronic device for managing a plurality of user terminals registered in one user account and a control method thereof.

In recent years, the number of user terminals possessed or used per user is increasing. As the number of user terminals increases, there is a inconvenience in that the user has to individually manage the privacy of data or resources stored in each user terminal.

In addition, as leaks and attacks on privacy data stored in a plurality of user terminals diversify and increase, the need to integrate and manage privacy for a plurality of user terminals is emerging.

The present disclosure has been devised to solve the problems as described above, and an object of the present disclosure is to learn the user data stored in each of a plurality of user terminals registered in a plurality of user accounts and the usage patterns of the user terminals to prevent abnormal terminals of the user. An object of the present invention is to provide an electronic device capable of detecting the use of the device and a method for controlling the same.

According to an embodiment of the present disclosure, an electronic device includes: a communication unit including a circuit; a memory storing at least one instruction; and a processor connected to the communication unit and the memory to control the electronic device, wherein the processor executes the at least one instruction, and each of a plurality of user terminals registered in one user account through the communication unit information on the user data acquired by the user terminal and information on the usage pattern of the user terminal is obtained from the user terminal, and whether the user normally uses the user terminal based on the acquired user data and the information on the usage pattern Learning a neural network model to determine, and when access to the user data is requested from a first user terminal among the plurality of user terminals, input information related to access to the user data into the neural network model to use a normal user terminal It is determined whether or not it is recognized, and if it is determined that the access to the user data is an abnormal use of the user terminal, a message about the abnormal use of the user terminal is provided.

Also, in a method of controlling an electronic device according to another embodiment of the present disclosure, information on user data obtained by a user terminal from each of a plurality of user terminals registered in one user account and information on a usage pattern of the user terminal obtaining a; learning a neural network model for determining whether a user normally uses a user terminal based on the acquired user data and information on the usage pattern; when access to the user data is requested from a first user terminal among the plurality of user terminals, inputting information related to access to the user data into the neural network model to determine whether a normal user terminal is used; and if it is determined that the access to the user data is an abnormal use of the user terminal, providing a message about the abnormal use of the user terminal.

According to various embodiments of the present disclosure as described above, the user can provide a more improved privacy protection function to the user by integrally managing privacy for a plurality of user terminals used by the user.

1 is a diagram illustrating a system including a plurality of user terminals, according to an embodiment of the present disclosure;
2 is a block diagram illustrating a configuration of an electronic device according to an embodiment of the present disclosure;
3A is a view for explaining an embodiment of determining whether a user terminal is normally used using a plurality of neural network models, according to an embodiment of the present disclosure;
3B is a diagram for explaining an embodiment of determining whether a user terminal is normally used using an integrated neural network model, according to an embodiment of the present disclosure;
4 is a block diagram illustrating a module for managing privacy data, according to an embodiment of the present disclosure;
5A is a diagram illustrating a screen displayed on an electronic device of a user who has generated privacy data according to an embodiment of the present disclosure;
5B is a view showing a screen displayed on an external terminal of another user who has received privacy data, according to an embodiment of the present disclosure;
6 is a sequence diagram for explaining an embodiment of determining normal use of a plurality of user terminals based on user data and usage patterns of the user terminals, according to an embodiment of the present disclosure;
7 is a sequence diagram for integrated management of privacy data according to an embodiment of the present disclosure;
8 is a flowchart for explaining a method of controlling an electronic device, according to an embodiment of the present disclosure;
9 is a detailed block diagram illustrating the configuration of an electronic device according to an embodiment of the present disclosure.

Hereinafter, the present disclosure will be described in more detail with reference to the drawings. 1 is a diagram illustrating a system including a plurality of user terminals according to an embodiment of the present disclosure. As shown in FIG. 1 , the system may include an electronic device 100 , a plurality of user terminals 10 - 1 , 10 - 2 , 10 - 3 and 10 - 4 , and an external terminal 30 .

The electronic device 100 is a device for storing a personal information protection engine module (hereinafter, “engine module”) for protecting user privacy. As shown in FIG. 1 , the electronic device 100 may be a terminal device such as a smart phone. This is only an example, tablet PC, mobile phone, video phone, e-book reader, desktop PC, laptop PC, netbook computer, workstation, server, PDA, PMP (portable multimedia player), MP3 player, medical device, camera, It may be implemented as one of a home appliance (eg, a TV, a refrigerator, etc.), a wearable device, and a server. At this time, the engine module collects information about user data and usage patterns obtained from the electronic device 100 and the plurality of user terminals 10-1, 10-2, 10-3, and 10-4, so that the user Learning a neural network model for determining normal use of the terminal, and normal use of the electronic device 100 and the plurality of user terminals 10-1, 10-2, 10-3, and 10-4 through the learned neural network model It is a module that can determine whether or not

In addition, the plurality of user terminals 10-1, 10-2, 10-3, and 10-4 are devices for storing personal information protection agent modules (hereinafter, “agent modules”) for protecting user privacy, in FIG. 1, it may be implemented as a terminal device such as an artificial intelligence speaker 10-1, a smart TV 10-2, a tablet PC 10-3, etc., but this is only an embodiment, and the cloud It may be implemented as a server 10-4. In this case, the agent module is a module that collects user data and usage patterns obtained by the user terminal and transmits them to the electronic device 100 for storing the engine module.

In particular, the electronic device 100 and the plurality of user terminals 10-1, 10-2, 10-3, and 10-4 may be devices registered in one user account. In this case, the engine module may be installed in a device set by a user among the electronic device 100 and the plurality of user terminals 10-1, 10-2, 10-3, and 10-4.

In addition, the external terminal 30 is a terminal device that uses a user account of another user different from the user of the electronic device 100 and the plurality of user terminals 10-1,10-2,10-3,10-4, Privacy data generated by at least one of the electronic device 100 and the plurality of user terminals 10-1, 10-2, 10-3, and 10-4 may be stored.

On the other hand, each of the plurality of user terminals (10-1, 10-2, 10-3, 10-4) collects information about user data and information about the usage pattern of the user terminal obtained by using the agent module to electronically may be transmitted to the device 100 . In this case, the user data may be data related to the user, such as image data, text data, schedule data, etc. generated in the user terminal. In addition, the usage pattern of the user terminal may include a usage pattern for an application or web application installed in the user terminal, a usage pattern related to access or transmission of user data, a network connection pattern, and the like. In this case, each of the plurality of user terminals 10-1, 10-2, 10-3, and 10-4 may transmit information on user data and information on usage patterns to the electronic device 100 at a preset period. This is only an exemplary embodiment, and when a specific event such as when a request is received from the electronic device 100 or when connected to a network is satisfied, information on user data and information on usage patterns are transmitted to the electronic device 100 . can When the user terminal transmits information on user data and information on usage patterns, information on the user terminal (eg, product name, product number, registered name, etc.) may be transmitted together.

The electronic device 100 allows the user to normally use the user terminal based on the information on the user data and the information on the usage pattern obtained from the plurality of user terminals 10-1, 10-2, 10-3, and 10-4. It is possible to train a neural network model to determine whether to use it.

As an embodiment, the electronic device 100 may learn a neural model for determining whether a user normally uses a user terminal for each user terminal. For example, the electronic device 100 may provide a first neural network model corresponding to the first user terminal 10 - 1 based on information on user data and information on a usage pattern received from the first user terminal 10 - 1 . , and learns a second neural network model corresponding to the second user terminal 10-2 based on information on user data and usage patterns received from the second user terminal 10-2. can do.

As another embodiment, the electronic device 100 may learn a neural network model for determining whether a user normally uses a user terminal for each usage pattern type. As an example, the electronic device 100 may learn a third neural network model based on information on a usage pattern of an application or a web application installed in the user terminal, and based on information on access and transmission of user data, the fourth neural network model A neural network model may be trained, and a fifth neural network model may be trained based on information on a network usage pattern. As another embodiment, the electronic device 100 may learn an integrated neural network model for integrally determining a usage pattern of the user terminal 100 . This will be described in detail with reference to FIGS. 3A and 3B .

When access to user data is requested from the first user terminal 10-1 among the plurality of user terminals 10-1, 10-2, 10-3, and 10-4, the electronic device 100 performs the learned neural network By inputting information related to access to user data into the model, it can be determined whether the user terminal is being used normally. In this case, the information related to access to user data includes information about an application or web application that performs access to user data, information about an external terminal that wants to access user data, and information about a network accessing user data. Information and information about user data may be included.

And, if it is determined that the access to the user data is the abnormal use of the user terminal, the electronic device 100 may provide a message about the abnormal use of the user terminal. In this case, the abnormal use of the user terminal may mean illegal use of the user terminal, such as hacking, phishing, and smishing of user data.

In this case, the electronic device 100 may output a message through an output device (eg, a display, a speaker) included in the electronic device 100 , but this is only an example, and the plurality of user terminals 10 - 1 ,10-2,10-3,10-4) can output a message. Alternatively, if it is determined that the access to user data is abnormal use of the user terminal, the electronic device 100 may provide a message for suggesting a security update for the first user terminal. For example, the electronic device 100 may provide a message for suggesting an update of a security solution or library version for protecting the privacy of the first user terminal.

Also, the electronic device 100 may include a private data protection module for remotely protecting private data among user data. In this case, the private data refers to data including the user's personal information (eg, a user image, an image related to the user's family, personal information, body information, financial information, etc.) among user data.

In particular, when the management right for the private data is set in the electronic device 10 , the private data protection module may remotely manage the private data.

Specifically, the electronic device 100 may determine the user's private data among the user data. For example, the electronic device 100 may determine image data including the user or the user's family among image data photographed by the user or received from the outside as the private data. As another example, the electronic device 100 may determine text data including the user's personal information (eg, personal information, body information, financial information, etc.) among text data generated by the user or received from the outside as private data. have.

In this case, the electronic device 100 may determine the private data using a face detection module, a text analysis module, etc., but this is only an example and may determine the private data using a learned neural network model.

In addition, when the management right for private data is set in the electronic device 100 , the electronic device 100 may track and store the transmission path and the copy path of the private data. Then, when a deletion request for the private data is received from the user, the electronic device 100 may transmit a deletion command for the private data to the outside based on the stored transmission path and the copy path. That is, the electronic device 100 can remotely manage the private data by transmitting a delete command for the private data to both the external terminals stored in the transmission path and the copy path.

Also, when a transmission request for the private data is received from the external terminal 30 , the electronic device 100 may process the private data so that it cannot be reproduced and transmit the processed private data to the external terminal 30 . In this case, the electronic device 100 may delete some files of the private data and transmit it to the external terminal 30 , or encrypt the private data and transmit it to the external terminal 30 . In addition, when a request for reproduction of private data is received from the external terminal 30 , the electronic device 100 may transmit information on some deleted files or decrypted private data to the external terminal 30 according to a user input. That is, when the user of the external terminal 30 wants to reproduce the private data received from the electronic device 100 , the external device 30 can reproduce the private data after being authenticated by the electronic device 100 .

In addition, when a transmission request for private data is received from the external terminal 30 , the electronic device 100 may transmit temporary authorization information granted for a preset period for the private data together with the private data to the external terminal 30 . have. In this case, the temporary permission information may include information on a storage period, a playback period, a storage permission and a playback permission of the private data, and the like. That is, since the external terminal 30 can store and reproduce private data only for a certain period of time, the privacy of the user of the electronic device 100 can be protected.

Meanwhile, although it has been described that only the electronic device 100 includes the engine module in the above-described embodiment, this is only an embodiment, and the engine module may be installed in at least one of the user terminals.

2 is a block diagram illustrating a configuration of an electronic device according to an embodiment of the present disclosure. As shown in FIG. 2 , the electronic device 100 may include a memory 110 , a communication unit 120 , and a processor 130 . Meanwhile, it goes without saying that some components may be added or omitted depending on the type of the electronic device 100 in the configuration of the electronic device 100 illustrated in FIG. 2 .

The memory 110 may store commands or data related to at least one other component of the electronic device 100 . In particular, the memory 110 may include a non-volatile memory and a volatile memory, and may be implemented as, for example, a flash-memory, a hard disk drive (HDD), or a solid state drive (SSD). The memory 110 is accessed by the processor 120 , and reading/writing/modification/deletion/update of data by the processor 120 may be performed. Also, the memory 110 may store the engine module 210 for protecting user privacy.

As shown in FIG. 2, the engine module 210 includes a PRIX-CORE module 211, a PRIX-APP module 212, a PRIX-WEB module 213, a PRIX-DATA module 214, and a PRIX-NETWORK module. 215 , may include a PRIX PROTECT module 216 .

The PRIX-CORE module 211 is a module for managing a user account and a user terminal registered in the user account. That is, the PRIX-CORE module 211 may register and delete a user terminal in a user account according to a user input. Also, the PRIX-CORE module 211 may store information about a user terminal registered in a user account. For example, the PRIX-CORE module 211 may store information about the product name, product number, manufacturing date, registration name, registration date, management authority, etc. of the user terminal registered in the user account.

In particular, the PRIX-CORE module 211 may identify that the user terminal is a terminal device used by the user of the electronic device 100 through user account login. Alternatively, the PRIX-CORE module 211 may identify the user account of the user terminal through service or application login.

The PRIX-CORE module 211 may manage policies for user rights. That is, the PRIX-CORE module 211 may manage a policy for a provisionable authority (eg, a storage authority, a copy authority, a deletion authority, etc.) among a plurality of registered user terminals.

The PRIX-CORE module 211 may provide a notification message to the user by detecting the abnormal use of the user terminal and the leakage of private data for each user terminal.

The PRIX-APP module 212 is a usage pattern of an application or service stored in the user terminal collected from the user terminal (eg, the execution time of the application or service, start/end time, the type and number of accessed data, execution location, etc.) ) and API access patterns (eg, API, number of calls, call time, etc.), a neural network model for determining abnormal use of an application can be trained. And, when access to an application or user data within the application is requested from the user terminal, the PRIX-APP module 212 may detect abnormal use of the application using the learned neural network model. When abnormal application usage is detected, the PRIX-APP module 212 may provide a message including information on abnormal usage.

The PRIX-WEB module 213 may learn a neural network model for determining abnormal use of a web application based on a usage pattern and API access pattern of a web application or service executed in a user terminal such as a web browser or runtime. . And, when access to the web application is requested from the user terminal, the PRIX-WEB module 213 may detect abnormal use of the web application using the learned neural network model. When abnormal use of the web application is detected, the PRIX-WEB module 213 may provide a message including information on abnormal use.

The PRIX-DATA module 214 may collect information about user data (particularly, private data) that is accessed inside the user terminal or transmitted outside the user terminal. Specifically, the PRIX-DARA module 214 obtains access history (eg, access time to data or resource, access number, etc.) and hash information for user data stored in each of a plurality of user terminals, and a plurality of users It is possible to obtain history and hash information on user data that each terminal transmits to the outside. Then, the PRIX-DATA module 214 learns a neural network model for determining abnormal use of user data based on the access history and hash information for stored user data, and the history and hash information for the user data transmitted to the outside. can do. In addition, when information on an access request to user data or information on a transmission request is received from the user terminal, the PRIX-DATA module 214 can detect non-ideal use of user data using the learned neural network model. . When an abnormal access request or transmission request of user data is detected, the PRIX-DATA module 214 may provide a message including information about the abnormal access request or transmission request.

The PRIX-NETWORK module 215 provides network IP and URL information connected to receive or transmit user data from each of a plurality of user terminals (eg, a Destination Weblink URL that a user wants to access, a Source Weblink that a user wants to access to a user terminal) URL, etc.) or a network access pattern, and a neural network model for determining abnormal use of a network based on the obtained network IP and URL information or network access pattern may be learned. When information on a network for transmitting or receiving user data is requested, the PRIX-NETWORK module 215 may determine abnormal use of the network using the learned neural network model. When abnormal network use is detected, the PRIX-NETWORK module 215 may provide a message including information on abnormal network use.

When the private data is transmitted to an external terminal, the PRIX-PROTECT module 216 may manage processing and reproduction of the private data. Specifically, the PRIX-PROTECT module 216 tracks and stores the transmission path and copy path of private data transmitted or copied to an external terminal, and manages authority information for the external terminal to process or reproduce private data, and When the data usage right or storage period has expired, a command for remote deletion can be transmitted to an external terminal. The PRIX-PROTECT module 216 will be described in more detail with reference to FIG. 4 .

In particular, when the engine module 210 is executed, the plurality of modules 211 to 216 stored in the nonvolatile memory described with reference to FIG. 2 may be loaded into the volatile memory and executed.

Meanwhile, at least some of the modules included in the memory 110 of FIG. 2 may also be included in the user terminal 10 in which the agent module is stored. That is, of course, the agent module may include at least some of the modules included in the engine module in addition to the collection module (not shown) for collecting user data and usage patterns.

The communication unit 120 is configured to communicate with various types of external devices according to various types of communication methods. In particular, the processor 130 may communicate with various user terminals or external terminals using the communication unit 120 . Specifically, the communication unit 120 may receive information about user data and usage information of the user terminal from the user terminal. In addition, the communication unit 120 may receive information on the access to the user data of the user terminal from the user terminal, whether it is a normal access obtained by inputting the information on the access to the user data into the learned neural name model information may be transmitted to the user terminal. Also, the communication unit 120 may transmit information on the private data and a deletion command to the external terminal.

Meanwhile, the communication unit 120 may include a Wi-Fi module, a Bluetooth module, an infrared communication module, a wireless communication module, and the like, and may communicate with the user terminal using different communication modules according to the type of the user terminal. For example, when the user terminal is an AI speaker, the communication unit 120 may perform communication using a Bluetooth module or a Wi-Fi direct module that is a short-range communication module, and when the user terminal is a tablet PC, the communication unit 120 is a remote module may perform communication with a wireless communication module (eg, 5G, LTE, 3GPPS, etc.) or a Wi-Fi module.

The processor 130 may be electrically connected to the memory 110 to control the overall operation of the electronic device 100 . In particular, the processor, by executing at least one instruction stored in the memory 110, information about the user data obtained by the user terminal from each of a plurality of user terminals registered in one user account through the communication unit 120 and the user It is possible to acquire information on the usage pattern of the terminal, and learn a neural network model for determining whether the user normally uses the user terminal based on the acquired information on the user data and the information on the usage pattern. When access to user data is requested from the first user terminal among the plurality of user terminals, the processor 130 may determine whether the user terminal is normally used by inputting information related to access to user data in the neural network model. . If it is determined that the access to user data is abnormal use of the user terminal, the electronic device 100 may provide a message regarding abnormal use of the user terminal. Alternatively, if it is determined that the access to user data is abnormal use of the user terminal, the electronic device 100 may provide a message for suggesting a security update for the first user terminal.

However, if it is determined that the access to the user data is the normal use of the user terminal, the electronic device 100 may train the neural network model on the access to the user data based on the data.

For example, the processor 130 obtains a usage pattern of an application installed in the user terminal and a web application accessed by the user terminal from each of the plurality of user terminals through the communication unit 120 , and uses the application installed in the user terminal. It is possible to learn the first neural network model for determining the abnormal use of the application based on the pattern and the usage pattern of the web application accessed by the user terminal.

As another example, the processor 130 obtains access history and hash information for user data stored in each of the plurality of user terminals through the communication unit 120 , and the history and hash of the user data transmitted to the outside by each of the plurality of user terminals It is possible to acquire information and learn a second neural network model for determining abnormal use of user data based on the access history and hash information of stored user data and the history and hash information of user data transmitted to the outside.

As another example, the processor 130 acquires network IP and URL information connected to receive or transmit user data from each of the plurality of user terminals through the communication unit 120 , and based on the network IP and URL information, abnormal network A third neural network model for determining use may be trained.

Meanwhile, the processor 130 may determine the user's private data among the user data obtained from the user terminal. In addition, when the management authority for the private data is set in the electronic device 100 , the processor 130 may track the transmission path and the copy path of the private data and store it in the memory 110 .

And, when a deletion request for the private data is received from the user, the processor 130 may control the communication unit 120 to transmit a deletion command for the private data to the outside based on the stored transmission path and the copy path.

In addition, when a transmission request for private data is received from an external terminal through the communication unit 120 , the processor 130 removes some files of the private data so that the private data cannot be reproduced to the external terminal through the communication unit 120 . When a request for reproduction of private data is received from an external terminal through the communication unit 120 , the processor 130 may transmit information about some files to an external terminal through the communication unit 120 according to a user input. Alternatively, when a transmission request for private data is received from an external terminal through the communication unit 120, the processor 130 encrypts the private data so that the private data cannot be reproduced and transmits it to the external terminal through the communication unit 120, and the communication unit When a request to reproduce the private data is received from the external terminal through 120 , the processor 130 may transmit information for decrypting the encrypted private data to the external terminal through the communication unit 120 . As a result, the external terminal processes and reproduces the private data based on information about some files or information for decoding, so that the user can be provided with a more improved protection function for the private data.

In addition, when a transmission request for private data is received from the external terminal through the communication unit 120 , the processor 130 may transmit temporary authorization information for the private data together with the private data to the external terminal through the communication unit 120 . . In this case, the temporary authorization information may include information on a storage period and a reproduction period for the private data.

In particular, the function related to artificial intelligence according to the present disclosure is operated through the processor 130 and the memory 110 . The processor 130 may include one or a plurality of processors. In this case, one or more processors may be a general-purpose processor such as a CPU, an AP, a digital signal processor (DSP), or the like, a graphics-only processor such as a GPU, a VPU (Vision Processing Unit), or an artificial intelligence-only processor such as an NPU. One or a plurality of processors control to process input data according to a predefined operation rule or artificial intelligence model stored in the memory 110 . Alternatively, when one or more processors are AI-only processors, the AI-only processor may be designed with a hardware structure specialized for processing a specific AI model.

The predefined action rule or artificial intelligence model is characterized in that it is created through learning. Here, being made through learning means that a basic artificial intelligence model is learned using a plurality of learning data by a learning algorithm, so that a predefined action rule or artificial intelligence model set to perform a desired characteristic (or purpose) is created means burden. Such learning may be performed in the device itself on which the artificial intelligence according to the present disclosure is performed, or may be performed through a separate server and/or system. Examples of the learning algorithm include, but are not limited to, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning.

The artificial intelligence model may be composed of a plurality of neural network layers. Each of the plurality of neural network layers has a plurality of weight values, and a neural network operation is performed through an operation between an operation result of a previous layer and a plurality of weight values. The plurality of weights of the plurality of neural network layers may be optimized by the learning result of the artificial intelligence model. For example, a plurality of weights may be updated so that a loss value or a cost value obtained from the artificial intelligence model during the learning process is reduced or minimized. The artificial neural network may include a deep neural network (DNN), for example, a Convolutional Neural Network (CNN), a Deep Neural Network (DNN), a Recurrent Neural Network (RNN), a Restricted Boltzmann Machine (RBM), There may be a Deep Belief Network (DBN), a Bidirectional Recurrent Deep Neural Network (BRDNN), or a Deep Q-Networks, but is not limited to the above-described example.

Meanwhile, according to an embodiment of the present disclosure, a neural network model for determining whether a user terminal is normally used may be provided for each type of usage pattern. 3A is a diagram for explaining an embodiment of determining whether a user terminal is normally used by using a plurality of neural network models, according to an embodiment of the present disclosure. Specifically, as shown in FIG. 3 , the electronic device 100 learns a usage pattern (eg, time, place, etc.) of an application or web application to detect an abnormal use of the application or web application. Download of the neural network model 310, the second neural network model 320, APK (AndroidApplicationPackage) and JS library (JavaScript library) for learning the access of IP and URL of the network to receive data to detect abnormal use of the network The third neural network model 330 for detecting abnormal use of user data by learning A fifth neural network model 350 for detecting abnormal use of the network by learning access to IP and URL of the network for transmission, a sixth neural network model for detecting abnormal use of user data by detecting upload of privacy data ( 360) may be included.

That is, the electronic device 100 may learn a neural network model for each type of use pattern of the electronic device 100 , and determine whether a feature value output through the neural network model is outside a threshold range to determine whether the electronic device 100 . can detect abnormal use of Also, since the use pattern for each user terminal may be different, the electronic device 100 may learn a neural network model for each user terminal.

Meanwhile, according to another embodiment of the present disclosure, the neural network model for determining whether the user terminal is normally used may be implemented as an integrated neural network model irrespective of the type of use pattern. 3B is a diagram for explaining an embodiment of determining whether a user terminal is normally used using an integrated neural network model, according to an embodiment of the present disclosure; Specifically, the integrated neural network model 370 may be composed of a plurality of layers, as shown in FIG. 3B , and a usage pattern (eg, time, place, etc.) of an application or web application through the plurality of layers. Abnormal use of an application or web application can be detected by learning, and an abnormal use of the network can be detected by learning the access to the IP and URL of the network to receive data, APK (AndroidApplicationPackage) and JS library (JavaScript library) ) to detect abnormal use of user data, learn access to privacy API (Application Programming Interface) to detect abnormal application use, and the IP and URL of the network to transmit data. By learning access, it can detect abnormal use of the network, and it can detect the upload of privacy data to detect abnormal use of user data.

That is, the electronic device 100 may learn the integrated neural network model 370 , and detect abnormal use of the electronic device 100 by determining whether a feature value output through the integrated neural network model is out of a threshold range. be able to In this case, the electronic device 100 may determine the type of the abnormal use pattern by determining the type of the layer to which the abnormal feature value is output. Also, since the use pattern for each user terminal may be different, the electronic device 100 may learn the integrated neural network model 370 for each user terminal.

On the other hand, the plurality of individual neural network models 310 to 360 and the integrated neural network model 370 shown in FIGS. 3A and 3B are only an embodiment, and the plurality of individual neural network models 310 to 360 are integrated with at least two or more. It may be implemented as a neural network model, and each individual neural network model may be implemented by being divided into two or more individual neural network models.

4 is a block diagram illustrating a PRIX-PROTECT module 216 for managing privacy data, according to an embodiment of the present disclosure. The PRIX-PROTECT module 216 may include a remote data management module 410 , a remote data activity tracking module 420 and a remote data processing module 430 as shown in FIG. 4 .

The remote data management module 410 is a module for remotely managing or controlling private data transmitted to a user terminal and an external terminal. In particular, the remote data management module 410 may manage authorization information for remotely managing private data. In this case, the authorization information may include information on authorizations such as deletion, copying, and movement of private data. In particular, the remote data management module 410 may grant temporary authorization information to at least one of user terminals of the same user account as the electronic device 100 as well as authorization information of the electronic device 100 . In this case, the temporary permission information is permission information granted for a preset period, and when the preset period arrives, the permission for the private data may be deleted.

In addition, when a command for processing the private data to be deleted or non-reproducible is input, the remote data management module 410 may transmit a command for processing the private data to be deleted or non-reproducible based on the stored transmission path and copy path. have. Accordingly, when the owner of the private data wants to delete his/her own private data, by transmitting a deletion command to the external terminal through the remote data management module 410, the private data stored in the external terminal of another person other than the user's own terminal is transmitted. Data can be deleted.

Also, the remote data management module 410 may provide a remote management UI for managing image data stored in the external terminal 30 of another user. Specifically, FIG. 5A may be a remote management UI 510 provided by the electronic device 100 , and FIG. 5B may be an image gallery UI 520 provided by the external terminal 30 . In this case, the UI elements 510-1 to 510-8 corresponding to the plurality of image data included in the remote management UI 510 may include transmission information, copy information, and reproduction information.

Also, when a deletion command is input by selecting at least one of the plurality of UI elements 510-1 to 510-8, the remote data management module 410 transmits a command to delete the selected image data to the external terminal 30 . can For example, when a deletion command is input by selecting the third image data and the seventh image data among the plurality of UI elements 510-1 to 510-8, the remote data management module 410 sends the data to the external terminal 30. A command to delete the selected third and seventh image data may be transmitted, and the external terminal 30 is a UI corresponding to the third image data and the seventh image data included in the image gallery UI 520 illustrated in FIG. 5B . A blank space may be displayed on the elements 520-3 and 520-7, and an indicator (expired (deleted)) guiding deletion may be displayed.

Also, according to the permission information set by the user of the electronic device 100 , at least one of the image gallery UIs 520 provided by the external terminal 30 may be processed to be impossible to reproduce. For example, when the external terminal 30 does not have a reproduction right for the fourth image data 510 - 4 , a UI element corresponding to the fourth image data among a plurality of UI elements included in the image gallery UI 520 . (520-4) may be displayed as a blank space.

The remote data activity tracking module 420 may collect the transmission path and the copy path of the private data to track information about the terminal device in which the private data is stored. In this case, the remote data activity tracking module 420 may track not only transmission and copying of private data, but also activities such as reproduction and editing, and may store the tracked information.

The remote data processing module 430 is a module for processing important information of the private data, and may delete some files of the private data or encrypt the private data in order to prevent it from being immediately reproduced even if the private data is transmitted to an external terminal. In addition, when a reproduction request is received from an external terminal later, the remote data processing module 430 may transmit some deleted files or information about decryption to the external terminal.

In one embodiment, the remote data processing module 430 separately transmits a Quantization/Entropy Coding Table for data/file configuration for data/file configuration, or separates low-frequency components and high-frequency components of an image/video frame and transmits them separately. , it is possible to prevent private data from being directly replayed on an external device.

That is, the user of the electronic device 100 can remotely manage and control private data using the modules shown in FIG. 4 , and thus can receive a more improved privacy protection function.

6 is a sequence diagram for explaining an embodiment of determining normal use of a plurality of user terminals based on user data and a usage pattern of the user terminals, according to an embodiment of the present disclosure.

First, the electronic device 100 may collect user data and information on a usage pattern from each of the first user terminal 10-1 and the second user terminal 10-2 (S605 and S610). In this case, the first user terminal 10 - 1 and the second user terminal 10 - 2 may be terminal devices registered in the same user account as the electronic device 100 . In this case, the user data is data generated by the user terminal, and may be data including information related to the user. For example, the user data may include image data, text data, schedule data, and the like. In addition, the information on the usage pattern may include a usage pattern for an application or web application installed in the user terminal, a usage pattern related to access or transmission of user data, a network connection pattern, and the like.

The electronic self-government 100 is a first user terminal 10-1 corresponding to the first user terminal 10-1 based on the user data received from each of the first and second user terminals 10-1 and 10-2 and the information on the usage pattern. The first neural network model and the second neural network model corresponding to the second user terminal 10-2 may be learned (S615). In this case, each of the first and second neural network models may be implemented as an individual neural network model or an integrated neural network model classified for each type of use pattern, as described with reference to FIGS. 3A and 3B .

The first user terminal 10-1 may detect access to user data (S620). For example, the first user terminal 10 - 1 may detect access to user data in order to upload image data to an external server using a specific application.

The first user terminal 10 - 1 may transmit information on access to user data to the electronic device 100 ( S625 ).

The electronic device 100 may determine whether the access to the user data is an abnormal use using the first neural network model ( S630 ). That is, the electronic device 100 may determine whether the access to the user data is an abnormal use by using the first neural network model corresponding to the first user terminal 10 - 1 . In this case, if it is determined that the access to the user data for the first user terminal is not abnormal, the electronic device 100 may learn the first neural network model based on the access to the user data for the first user terminal.

The second user terminal 10 - 2 may detect access to user data ( S635 ). The second user terminal 10 - 2 may transmit information on access to user data to the electronic device 100 ( S640 ).

The electronic device 100 may determine whether the access to the user data is an abnormal use by using the second neural network model (S645). That is, the electronic device 100 may determine whether the access to the user data is an abnormal use by using the second neural network model corresponding to the second user terminal 10 - 2 . In this case, if it is determined that the access to the user data for the second user terminal is an abnormal use, the electronic device 100 may provide a message about the abnormal use of the user terminal (S650).

7 is a sequence diagram for integrating and managing privacy data according to an embodiment of the present disclosure.

The external terminal 30 may transmit a private data transmission request to the electronic device 100 (S710). For example, the external terminal 30 may request transmission of image data including the user of the electronic device 100 .

The electronic device 100 may process the requested private data to be non-reproducible ( S720 ). Specifically, the electronic device 100 may remove some files of the requested private data, but encrypt the requested private data and process the requested private data to be non-reproducible.

The electronic device 100 may transmit the processed private data to the external terminal 30 (S730). In this case, the processed private data may be stored in the external terminal 30 in a non-reproducible form.

The external terminal 30 may receive a reproduction command for reproducing the private data (S740). The external terminal 30 may transmit a playback request to the electronic device 100 in response to the playback command (S750).

The electronic device 100 may obtain reproduction information for processing to be reproducible ( S760 ). In detail, the electronic device 100 may acquire information on some deleted files or information on decryption as reproduction information for reproducing the processed private data.

The electronic device 100 may transmit the acquired reproduction information to the external terminal 30 (S770). The external terminal 30 may reproduce the private data based on the received reproduction information (S780).

Also, the electronic device 100 may receive a deletion command for the private data (S790). The electronic device 100 may transmit a deletion command to the external terminal 30 based on the input command (S795).

The external terminal 30 may delete the private data in response to the deletion command (S797).

Accordingly, the user of the electronic device 100 may receive a more improved privacy protection function by performing remote management on the private data.

8 is a flowchart illustrating a method of controlling an electronic device according to an embodiment of the present disclosure.

First, the electronic device 100 may obtain information about user data acquired by the user terminal and information on a usage pattern of the user terminal from each of a plurality of user terminals registered in one user account ( S810 ).

The electronic device 100 may learn a neural network model for determining whether the user normally uses the user terminal based on the acquired information on the user data and the information on the usage pattern ( S820 ). In this case, a pattern of using the user terminal may be different for each user of the electronic device 100 . For example, while the user of the electronic device 100 takes or uploads a photo using a first user terminal (eg, a smart phone), the user takes or uploads a picture using a second user terminal (eg, a smart watch). Without it, only body information or time information can be checked. Therefore, since the usage pattern is different depending on the user terminal, it is possible to provide a more improved privacy protection function by learning the neural network model for each user terminal.

When access to user data is requested from a first user terminal among a plurality of user terminals, the electronic device 100 may determine whether the user terminal is being used normally by inputting information related to access to user data in the neural network model. There is (S830).

If it is determined that the access to user data is the abnormal use of the user terminal (S840-Y), the electronic device 100 may provide a message about the abnormal use of the user terminal (S850). However, if it is determined that the access to the user data is a normal use of the user terminal (S840-N), the electronic device 100 may re-learn the neural network model based on information related to the access to the user terminal.

9 is a detailed block diagram illustrating the configuration of an electronic device according to an embodiment of the present disclosure. As shown in FIG. 9 , the electronic device 100 includes a communication unit 120 , a memory 110 , a microphone 140 , a display 150 , a speaker 160 , a sensor 170 , and a processor 130 . can do. Meanwhile, it goes without saying that some of the components of the electronic device 100 shown in FIG. 9 may be added or omitted depending on the type of the electronic device 100 . In addition, since the characteristics of the memory 110, the communication unit 120, and the processor 130 shown in FIG. 9 have been described with reference to FIG. 2, a redundant description will be omitted.

The microphone 140 is a component for receiving a user's voice, and may be provided in the electronic device 100 , but this is only an exemplary embodiment. It can be connected wirelessly. In particular, the microphone 140 may receive a user voice for controlling or managing user data (or private data) stored in the electronic device 100 or the user terminal 10 .

The display 150 may display an image or UI received from the outside. In particular, the display 150 may be implemented as various types of displays such as a liquid crystal display (LCD), an organic light emitting diode (OLED) display, a plasma display panel (PDP), and the like. The display 150 may also include a driving circuit, a backlight unit, and the like, which may be implemented in the form of an a-si TFT, a low temperature poly silicon (LTPS) TFT, or an organic TFT (OTFT). Meanwhile, the display 150 may be implemented as a touch screen combined with a touch sensor, a flexible display, a three-dimensional display, or the like. Also, according to an embodiment of the present invention, the display 150 may include a bezel housing the display panel as well as a display panel for outputting an image. In particular, according to an embodiment of the present invention, the bezel may include a touch sensor (not shown) for detecting user interaction. In particular, the display 150 may provide at least one of a message UI for providing information on abnormal use and a message UI for suggesting an update of a security solution. Also, the display 150 may display a remote management UI for remotely managing private data.

The speaker 160 may be a component that outputs various types of audio data received externally, as well as various notification sounds or voice messages. In this case, the electronic device 100 may include an audio output device such as the speaker 160 , but may include an output device such as an audio output terminal. In particular, the speaker 160 may provide a voice message for providing information on abnormal use or a voice message for suggesting an update of a security solution.

The sensor 170 may acquire various information related to the electronic device 100 . In particular, the sensor 170 may include a GPS for obtaining location information of the electronic device 100 , and a biometric sensor (eg, a heart rate sensor) for obtaining biometric information of a user using the electronic device 100 . , PPG sensor, etc.) and various sensors such as a motion sensor for detecting the motion of the electronic device 100 .

In addition, the electronic device 100 may include an input interface for receiving a user command for controlling the electronic device 100 . In this case, the input interface may be implemented as a device such as a button, a touch pad, a mouse, and a keyboard, or may be implemented as a touch screen capable of performing the above-described display function and manipulation input function together. Here, the button may be various types of buttons such as a mechanical button, a touch pad, a wheel, etc. formed in an arbitrary area such as a front part, a side part, or a rear part of the main body of the electronic device 100 .

On the other hand, the drawings attached to the present disclosure are not intended to limit the technology described in the present disclosure to specific embodiments, and various modifications, equivalents, and/or alternatives of the embodiments of the present disclosure are provided. should be understood as including In connection with the description of the drawings, like reference numerals may be used for like components.

In the present disclosure, expressions such as “have,” “may have,” “include,” or “may include” indicate the presence of a corresponding characteristic (eg, a numerical value, function, operation, or component such as a part). and does not exclude the presence of additional features.

In this disclosure, expressions such as “A or B,” “at least one of A and/and B,” or “one or more of A or/and B” may include all possible combinations of the items listed together. . For example, "A or B," "at least one of A and B," or "at least one of A or B" means (1) includes at least one A, (2) includes at least one B; Or (3) it may refer to all cases including both at least one A and at least one B.

As used in the present disclosure, expressions such as "first," "second," "first," or "second," may modify various elements, regardless of order and/or importance, and refer to one element. It is used only to distinguish it from other components, and does not limit the components.

A component (eg, a first component) is "coupled with/to (operatively or communicatively)" to another component (eg, a second component) When referring to "connected to", it will be understood that the certain element may be directly connected to the other element or may be connected through another element (eg, a third element). On the other hand, when it is said that a component (eg, a first component) is "directly connected" or "directly connected" to another component (eg, a second component), the component and the It may be understood that other components (eg, a third component) do not exist between other components.

The expression "configured to (or configured to)" as used in this disclosure depends on the context, for example, "suitable for," "having the capacity to" ," "designed to," "adapted to," "made to," or "capable of." The term “configured (or configured to)” may not necessarily mean only “specifically designed to” in hardware. Instead, in some circumstances, the expression “a device configured to” may mean that the device is “capable of” with other devices or parts. For example, the phrase "a coprocessor configured (or configured to perform) A, B, and C" may include a processor dedicated to performing the operations (eg, an embedded processor), or executing one or more software programs stored in a memory device. By doing so, it may mean a generic-purpose processor (eg, a CPU or an application processor) capable of performing corresponding operations.

The electronic device according to various embodiments of the present disclosure may include, for example, at least one of a smartphone, a tablet PC, a desktop PC, a laptop PC, a netbook computer, a server, a PDA, a medical device, and a wearable device. In some embodiments, the electronic device may include, for example, at least one of a television, a refrigerator, an air conditioner, an air purifier, a set-top box, and a media box (eg, Samsung HomeSync™, Apple TV™, or Google TV™).

Meanwhile, the term user may refer to a person who uses an electronic device or a device (eg, an artificial intelligence electronic device) using the electronic device.

Various embodiments of the present disclosure may be implemented as software including instructions stored in a machine-readable storage media readable by a machine (eg, a computer). As a device that can be called and operated according to the called command, it may include an electronic device (eg, the electronic device 100) according to the disclosed embodiments. When the command is executed by the processor, the processor directly or A function corresponding to the instruction may be performed using other components under the control of the processor. The instruction may include code generated or executed by a compiler or an interpreter. A device-readable storage medium includes: It may be provided in the form of a non-transitory storage medium, where the 'non-transitory storage medium' does not include a signal and means that the data is tangible and semi-permanent or Temporarily stored is not distinguished, for example, a 'non-transitory storage medium' may include a buffer in which data is temporarily stored.

According to an embodiment, the method according to various embodiments disclosed in the present disclosure may be provided by being included in a computer program product. Computer program products may be traded between sellers and buyers as commodities. The computer program product may be distributed in the form of a machine-readable storage medium (eg, compact disc read only memory (CD-ROM)) or online through an application store (eg, Play Store™). In the case of online distribution, at least a portion of a computer program product (eg, a downloadable app) is at least temporarily stored in a storage medium such as a memory of a server of a manufacturer, a server of an application store, or a relay server, or , can be created temporarily.

Each of the components (eg, a module or a program) according to various embodiments may be composed of a singular or a plurality of entities, and some sub-components of the aforementioned sub-components may be omitted, or other sub-components may be various It may be further included in the embodiment. Alternatively or additionally, some components (eg, a module or a program) may be integrated into a single entity to perform the same or similar functions performed by each corresponding component prior to integration. According to various embodiments, operations performed by a module, program, or other component may be sequentially, parallel, repetitively or heuristically executed, or at least some operations may be executed in a different order, omitted, or other operations may be added. can

110: memory 120: communication unit
130: processor 140: microphone
150: display 160: speaker
170: sensor

Claims (20)

In an electronic device,
a communication unit including a circuit;
a memory storing at least one instruction; and
a processor connected to the communication unit and the memory to control the electronic device;
The processor, by executing the at least one instruction,
Obtaining information on user data obtained by a user terminal and information on a usage pattern of the user terminal from each of a plurality of user terminals registered in one user account through the communication unit,
Learning a neural network model for determining whether a user normally uses a user terminal based on the acquired information on the user data and the information on the usage pattern,
When access to the user data is requested from a first user terminal among the plurality of user terminals, information related to access to the user data is input to the neural network model to determine whether a normal user terminal is used,
When it is determined that the access to the user data is an abnormal use of the user terminal, the electronic device provides a message about the abnormal use of the user terminal. According to claim 1,
The processor is
Obtaining a usage pattern of an application installed in a user terminal and a usage pattern of a web application accessed by the user terminal from each of the plurality of user terminals through the communication unit,
An electronic device for learning a first neural network model for determining abnormal use of an application based on a usage pattern of an application installed in the user terminal and a usage pattern of a web application accessed by the user terminal. According to claim 1,
The processor is
Obtaining access history and hash information for user data stored in each of the plurality of user terminals through the communication unit, and obtaining history and hash information for user data transmitted by each of the plurality of user terminals to the outside,
An electronic device for learning a second neural network model for determining abnormal use of user data based on the access history and hash information of the stored user data and the history and hash information of the user data transmitted to the outside. According to claim 1,
The processor is
Obtaining network IP and URL information connected to receive or transmit user data from each of the plurality of user terminals through the communication unit,
An electronic device for learning a third neural network model for determining abnormal use of the network based on the network IP and URL information. According to claim 1,
The processor is
If it is determined that the access to the user data is abnormal use of the user terminal, the electronic device provides a message for suggesting a security update for the first user terminal. According to claim 1,
The processor is
determining the user's private data among the user data;
When a management right for the private data is set in the electronic device, the electronic device tracks and stores a transmission path and a copy path of the private data in the memory. 7. The method of claim 6,
The processor is
When a deletion request for the private data is received from a user, the electronic device controls the communication unit to transmit a deletion command for the private data to the outside based on the stored transmission path and the copy path. 7. The method of claim 6,
The processor is
When a transmission request for the private data is received from the external terminal through the communication unit, some files of the private data are removed so that the private data cannot be reproduced and transmitted to the external terminal through the communication unit,
When a request for reproduction of the private data is received from the external terminal through the communication unit, the electronic device transmits information on the partial file to the external terminal through the communication unit. 7. The method of claim 6,
The processor is
When a transmission request for the private data is received from an external terminal through the communication unit, the private data is encrypted so that the private data cannot be reproduced and transmitted to the external terminal through the communication unit,
When a request for reproduction of the private data is received from the external terminal through the communication unit, the electronic device transmits information for decrypting the encrypted private data to the external terminal through the communication unit. 7. The method of claim 6,
The processor is
When a transmission request for the private data is received from the external terminal through the communication unit, the electronic device transmits temporary authorization information granted for a preset period for the private data together with the private data to the external terminal through the communication unit. . A method for controlling an electronic device, comprising:
obtaining information on user data obtained by a user terminal and information on a usage pattern of the user terminal from each of a plurality of user terminals registered in one user account;
learning a neural network model for determining whether a user normally uses a user terminal based on the acquired information on the user data and the information on the usage pattern;
when access to the user data is requested from a first user terminal among the plurality of user terminals, inputting information related to access to the user data into the neural network model to determine whether a normal user terminal is used; and
When it is determined that the access to the user data is abnormal use of the user terminal, providing a message about the abnormal use of the user terminal; Control method comprising a. 12. The method of claim 11,
The obtaining step is
Obtaining a usage pattern of an application installed in a user terminal and a usage pattern of a web application accessed by the user terminal from each of the plurality of user terminals,
The learning step is
A control method for learning a first neural network model for determining abnormal use of an application based on a usage pattern of an application installed in the user terminal and a usage pattern of a web application accessed by the user terminal. 12. The method of claim 11,
The obtaining step is
Obtaining access history and hash information for user data stored in each of the plurality of user terminals, and obtaining history and hash information for user data transmitted to the outside by each of the plurality of user terminals,
The learning step is
A control method for learning a second neural network model for determining abnormal use of user data based on the access history and hash information of the stored user data and the history and hash information of the user data transmitted to the outside. 12. The method of claim 11,
The obtaining step is
Obtaining network IP and URL information connected to receive or transmit user data from each of the plurality of user terminals,
The learning step is
A control method for learning a third neural network model for determining abnormal use of the network based on the network IP and URL information. 12. The method of claim 11,
If it is determined that the access to the user data is abnormal use of the user terminal, providing a message for suggesting a security update for the first user terminal; 12. The method of claim 11,
determining the user's private data among the user data;
and tracking and storing a transmission path and a copy path of the private data when a management right for the private data is set in the electronic device. 17. The method of claim 16,
and, when a deletion request for the private data is received from a user, transmitting a deletion command for the private data to the outside based on the stored transmission path and the copy path. 17. The method of claim 16,
when a transmission request for the private data is received from an external terminal, removing some files of the private data so that the private data cannot be reproduced and transmitting the request to the external terminal; and
and transmitting information on the partial file to the external terminal when a request for reproduction of the private data is received from the external terminal. 17. The method of claim 16,
when a transmission request for the private data is received from an external terminal, encrypting the private data so that the private data cannot be reproduced and transmitting the private data to the external terminal; and
and transmitting information for decrypting the encrypted private data to the external terminal when a request for reproducing the private data is received from the external terminal. 17. The method of claim 16,
When a transmission request for the private data is received from an external terminal, transmitting temporary authorization information granted for a preset period for the private data together with the private data to the external terminal.

Images