KR102473506B1 - Method and apparatus for providing watermarked data using a neural network - Google Patents

Abstract

Embodiments of the present invention provide a method and an apparatus for allowing a server to use a neural network to transmit watermarked data to at least one device. The method according to one embodiment of the present invention comprises the steps of: obtaining original data based on document information inputted into the server; determining pattern information through a pattern generation model using the neural network, based on the original data, a name for at least one receiving person, and a name for a transmitting person which is inputted in advance, wherein the at least one receiving person is selected from the list of the receiving persons pre-stored in the server; allowing the pre-stored list of the receiving persons to include the names and the contact information of the receiving persons, allowing the pattern information to include information on a first object and a second object constituting a watermark and information on a pattern configured by combination of the first object and the second object, and inserting the watermark including the first object and the second object into the original data, based on the pattern information and setting information on the pre-inputted watermark; and transmitting the watermarked data to at least one device related to the at least one receiving person, based on the contact information on the receiving person. For example, the first object can be generated based on the name of the at least one receiving person. For example, the second object can be generated based on the name of the transmitting person. For example, the watermark transmitted by the at least one device can be determined differently for each of the at least one device.

Description

Method and apparatus for providing watermarked data using a neural network

Embodiments of the present disclosure relate to a technology for providing watermarked data, and to a technology for providing watermarked data using a neural network.

As the demand for online documents, photos, and video data increases with the development of digital technology, intellectual property rights problems arise due to illegal copying or distribution of digital contents. In the case of digital content, since a copy can be produced without loss of quality, unauthorized copying and distribution of digital content may be more problematic. Accordingly, a copyright protection technology for protecting the ownership of digital data and preventing illegal copying and distribution is required.

To this end, a commonly used technology is digital watermarking, and digital watermarking is a technology of inserting unique information of a copyright holder into content for the purpose of eradicating commercial theft, such as illegal copying, distribution, and counterfeiting of various digital content. Here, the embedded copyright holder's unique information is referred to as a watermark.

In particular, a watermark may be visibly inserted into a secure document to prevent leakage of the secure document. For example, official letters sent from the convenience store headquarters to store owners across the country, cooperation letters sent from the franchise headquarters to each business site, documents related to case progress or judgments sent by departments from the prosecution, in-house documents sent from companies to employees, and various documents sent from inside the police station. By visually embedding a watermark on documents that require security, such as documents, scenarios sent by movie or drama teams, various contracts, electronic books, and completed novels just before contracts, leakage of documents can be prevented in advance.

Meanwhile, when a plurality of security documents are sent to a plurality of recipients, it may be inconvenient to insert a watermark into each security document transmitted to each recipient. In addition, when only a part of the watermark-embedded security document is leaked, if a part of the watermark is included in the part of the security document, it may be difficult to check the leaked path with only a part of the watermark instead of the entire watermark.

Therefore, a method of easily setting a recipient through a pre-stored recipient list and providing watermarked data composed of a recipient's name and a sender's name as a pattern through a neural network is required.

Embodiments of the present disclosure may provide a method and apparatus for providing watermarked data using a neural network.

Technical tasks to be achieved in the embodiments are not limited to those mentioned above, and other technical tasks not mentioned will be considered by those skilled in the art from various embodiments to be described below. can

A method in which a server transmits watermarked data to at least one device using a neural network by a server according to an embodiment includes obtaining original data based on document information input to the server; , Pattern information is determined through a pattern generation model using the neural network based on the original data, the name of at least one recipient, and the name of the sender input in advance, and the at least one recipient is pre-stored in the server. is selected from a list of recipients, the pre-stored recipient list includes a name of the recipient and contact information about the recipient, and the pattern information includes information about a first object and a second object constituting a watermark and the first object and information on a pattern composed of a combination of the second object, and a watermark including the first object and the second object is created based on the pattern information and pre-input setting information for the watermark, and the original watermark and transmitting the watermarked data to at least one device associated with the at least one recipient based on contact information for the recipient. For example, the first object may be created based on the name of the at least one recipient. For example, the second object may be created based on the name of the sender. For example, a watermark transmitted to the at least one device may be determined differently for each of the at least one device.

According to an embodiment, the information about the first object and the second object includes information about the size of the first object and the second object and information about the shape of the first object and the second object. can do. For example, a pattern composed of a combination of the first object and the second object may include two or more patterns. For example, the size of the first object and the second object may be determined based on the size of text included in the original data.

According to an embodiment, a plurality of first areas may be determined for the original data through Density-Based Spatial Clustering of Applications with Noise (DBSCAN) technique based on the distribution of text included in the original data. For example, one or more second areas in which the watermark can be inserted may be determined among the plurality of first areas. For example, the watermark may be inserted among the one or more second areas.

Additionally, for example, among the plurality of first areas, an area in which a ratio of text included in the first area is greater than or equal to a preset ratio may be determined as the second area. For example, the ratio of the text included in the first area may be the ratio occupied by the texts included in the first area with respect to the first area. For example, the ratio of text included in the first area may be determined based on the size of the first area, the average size of text included in the first area, and the number of texts included in the first area. . For example, the server may insert the watermark into an area where the ratio of text included in the first area is greater than or equal to a preset ratio among the plurality of first areas.

For example, the preset ratio may be determined by the following equation.

In the above equation, r _w is the preset ratio, T _s is the average size of text included in the original data, n _T is the number of texts included in the original data, and D _s is the The size of the document related to the original data, n is the number of the plurality of first areas, A _i is the size of the ith first area, and A _o is the size of the currently selected first area.

According to an embodiment, a range of transparency for the watermark may be differently determined based on the clarity of the original data. For example, the minimum value of the transparency may be determined by the following equation.

In the above equation, a _min is the minimum value of the transparency, r _t is the ratio of text included in the original data, B _b is the average brightness value for the background included in the original data, and B _t is the An average brightness value for text included in the original data, S may be the sharpness of the original data, and a may be a weight related to the watermark.

Additionally, for example, sharpness of the original data may be determined by the following equation.

In the above equation, S is the sharpness of the original data, k is the total number of gradients, G _xj is a set of j-th gradient values in the x-direction, and G _yj is a set of j-th gradient values in the y-direction have. Here, the S value may be a value between 0 and 100. For example, when the sharpness of the original data is less than the preset sharpness, the server may increase the sharpness of the original data by using a sharpening filter.

According to an embodiment, the neural network may include an input layer, one or more hidden layers, and an output layer. For example, each training data consisting of a plurality of original data, data with a plurality of watermarks inserted, and correct answer pattern information is input to the input layer of the neural network, passes through one or more hidden layers and an output layer, and is output. It is output as a vector, and the output vector is input to a loss function layer connected to the output layer, and the loss function layer outputs a loss value using a loss function that compares the output vector with the correct vector for each learning data. Parameters of the neural network may be learned in a direction in which the loss value decreases, and the pattern generation model may be generated.

According to embodiments, the server can identify the recipient and the sender even when a part of the document is extracted by providing a watermark including a pattern composed of the name of the recipient and the name of the sender through the neural network. In addition, since the watermark includes two or more patterns, the recipient and the sender can specify and check an outflow path for the same documents.

According to the embodiments, the server determines the area where the watermark can be inserted according to the distribution of text included in the original data, and determines the transparency of the watermark according to the sharpness of the original data, thereby more effectively inserting the watermark into the document. can be inserted into

According to embodiments, the server may transmit watermarked data to devices associated with recipients based on a pre-stored recipient list, thereby transmitting data with different watermarks embedded in each recipient-related device, e-mail address, It can be transmitted efficiently according to the messenger ID or phone number.

Effects obtainable from the embodiments are not limited to the effects mentioned above, and other effects not mentioned are clearly derived and understood by those skilled in the art based on the detailed description below. It can be.

BRIEF DESCRIPTION OF THE DRAWINGS Included as part of the detailed description to aid understanding of the embodiments, the accompanying drawings provide various embodiments and, together with the detailed description, describe technical features of the various embodiments.
1 is a diagram illustrating a configuration of an electronic device according to an exemplary embodiment.
2 is a diagram showing the configuration of a program according to an embodiment.
3 illustrates a method of transmitting watermarked data to at least one device using a neural network according to an exemplary embodiment.
4 is an example of a basic screen including a recipient list according to an embodiment.
5 is an example of a screen related to watermark setting according to an exemplary embodiment.
6 is an example of a screen for reserving an entire message transmitted together with watermarked data according to an embodiment.
7 is an example of a screen before sending an entire message according to an embodiment.
9 is an example of a process of setting a watermark to be inserted into original data according to an embodiment.
10 is a block diagram illustrating a configuration of a server according to an exemplary embodiment.

The following embodiments combine elements and features of the embodiments in a predetermined form. Each component or feature may be considered optional unless explicitly stated otherwise. Each component or feature may be implemented in a form not combined with other components or features. In addition, various embodiments may be configured by combining some components and/or features. The order of operations described in various embodiments may be changed. Some components or features of one embodiment may be included in another embodiment, or may be replaced with corresponding components or features of another embodiment.

In the description of the drawings, procedures or steps that may obscure the gist of various embodiments are not described, and procedures or steps that can be understood by those skilled in the art are not described. did

Throughout the specification, when a part is said to "comprising" or "including" a certain element, it means that it may further include other elements, not excluding other elements, unless otherwise stated. do. In addition, terms such as “… unit”, “… unit”, and “module” described in the specification mean a unit that processes at least one function or operation, which is hardware or software or a combination of hardware and software. can be implemented as Also, “a or an”, “one”, “the” and like terms are used herein in the context of describing various embodiments (particularly in the context of the claims below). Unless otherwise indicated or clearly contradicted by context, both the singular and the plural can be used.

Hereinafter, embodiments according to various embodiments will be described in detail with reference to the accompanying drawings. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The detailed description set forth below in conjunction with the accompanying drawings is intended to describe exemplary embodiments of various embodiments, and is not intended to represent a single embodiment.

In addition, specific terms used in various embodiments are provided to help understanding of various embodiments, and the use of these specific terms may be changed into other forms without departing from the technical spirit of various embodiments. .

1 is a diagram illustrating a configuration of an electronic device according to an exemplary embodiment.

1 is a block diagram of an electronic device 101 within a network environment 100, according to various embodiments. Referring to FIG. 1 , in a network environment 100, an electronic device 101 communicates with an electronic device 102 through a first network 198 (eg, a short-range wireless communication network) or through a second network 199. It may communicate with at least one of the electronic device 104 or the server 108 through (eg, a long-distance wireless communication network). According to one embodiment, the electronic device 101 may communicate with the electronic device 104 through the server 108 . According to an embodiment, the electronic device 101 includes a processor 120, a memory 130, an input module 150, an audio output module 155, a display module 160, an audio module 170, a sensor module ( 176), interface 177, connection terminal 178, haptic module 179, camera module 180, power management module 188, battery 189, communication module 190, subscriber identification module 196 , or the antenna module 197 may be included. In some embodiments, in the electronic device 101, at least one of these components (eg, the connection terminal 178) may be omitted or one or more other components may be added. In some embodiments, some of these components (eg, sensor module 176, camera module 180, or antenna module 197) are integrated into a single component (eg, display module 160). It can be. The electronic device 101 may also be referred to as a client, terminal, or peer.

The processor 120, for example, executes software (eg, the program 140) to cause at least one other component (eg, hardware or software component) of the electronic device 101 connected to the processor 120. It can control and perform various data processing or calculations. According to one embodiment, as at least part of data processing or operation, the processor 120 transfers instructions or data received from other components (e.g., sensor module 176 or communication module 190) to volatile memory 132. , processing commands or data stored in the volatile memory 132 , and storing resultant data in the non-volatile memory 134 . According to one embodiment, the processor 120 may include a main processor 121 (eg, a central processing unit or an application processor) or a secondary processor 123 (eg, a graphic processing unit, a neural network processing unit ( NPU: neural processing unit (NPU), image signal processor, sensor hub processor, or communication processor). For example, when the electronic device 101 includes the main processor 121 and the auxiliary processor 123, the auxiliary processor 123 may use less power than the main processor 121 or be set to be specialized for a designated function. can The secondary processor 123 may be implemented separately from or as part of the main processor 121 .

The secondary processor 123 may, for example, take the place of the main processor 121 while the main processor 121 is in an inactive (eg, sleep) state, or the main processor 121 is active (eg, running an application). ) state, together with the main processor 121, at least one of the components of the electronic device 101 (eg, the display module 160, the sensor module 176, or the communication module 190) It is possible to control at least some of the related functions or states. According to one embodiment, the auxiliary processor 123 (eg, image signal processor or communication processor) may be implemented as part of other functionally related components (eg, camera module 180 or communication module 190). have. According to an embodiment, the auxiliary processor 123 (eg, a neural network processing device) may include a hardware structure specialized for processing an artificial intelligence model.

AI models can be created through machine learning. Such learning may be performed, for example, in the electronic device 101 itself where the artificial intelligence model is performed, or may be performed through a separate server (eg, the server 108). The learning algorithm may include, for example, supervised learning, unsupervised learning, semi-supervised learning or reinforcement learning, but in the above example Not limited. The artificial intelligence model may include a plurality of artificial neural network layers. Artificial neural networks include deep neural networks (DNNs), convolutional neural networks (CNNs), recurrent neural networks (RNNs), restricted boltzmann machines (RBMs), deep belief networks (DBNs), bidirectional recurrent deep neural networks (BRDNNs), It may be one of deep Q-networks or a combination of two or more of the foregoing, but is not limited to the foregoing examples. The artificial intelligence model may include, in addition or alternatively, software structures in addition to hardware structures.

The memory 130 may store various data used by at least one component (eg, the processor 120 or the sensor module 176) of the electronic device 101 . The data may include, for example, input data or output data for software (eg, program 140) and commands related thereto. The memory 130 may include volatile memory 132 or non-volatile memory 134 .

The program 140 may be stored as software in the memory 130 and may include, for example, an operating system 142 , middleware 144 , or an application 146 .

The input module 150 may receive a command or data to be used by a component (eg, the processor 120) of the electronic device 101 from the outside of the electronic device 101 (eg, a user). The input module 150 may include, for example, a microphone, a mouse, a keyboard, a key (eg, a button), or a digital pen (eg, a stylus pen).

The sound output module 155 may output sound signals to the outside of the electronic device 101 . The sound output module 155 may include, for example, a speaker or a receiver. The speaker can be used for general purposes such as multimedia playback or recording playback. A receiver may be used to receive an incoming call. According to one embodiment, the receiver may be implemented separately from the speaker or as part of it.

The display module 160 may visually provide information to the outside of the electronic device 101 (eg, a user). The display module 160 may include, for example, a display, a hologram device, or a projector and a control circuit for controlling the device. According to one embodiment, the display module 160 may include a touch sensor set to detect a touch or a pressure sensor set to measure the intensity of force generated by the touch.

The audio module 170 may convert sound into an electrical signal or vice versa. According to one embodiment, the audio module 170 acquires sound through the input module 150, the sound output module 155, or an external electronic device connected directly or wirelessly to the electronic device 101 (eg: Sound may be output through the electronic device 102 (eg, a speaker or a headphone).

The sensor module 176 detects an operating state (eg, power or temperature) of the electronic device 101 or an external environmental state (eg, a user state), and generates an electrical signal or data value corresponding to the detected state. can do. According to one embodiment, the sensor module 176 may include, for example, a gesture sensor, a gyro sensor, an air pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a bio sensor, It may include a temperature sensor, humidity sensor, or light sensor.

The interface 177 may support one or more designated protocols that may be used to directly or wirelessly connect the electronic device 101 to an external electronic device (eg, the electronic device 102). According to one embodiment, the interface 177 may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

The connection terminal 178 may include a connector through which the electronic device 101 may be physically connected to an external electronic device (eg, the electronic device 102). According to one embodiment, the connection terminal 178 may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (eg, a headphone connector).

The haptic module 179 may convert electrical signals into mechanical stimuli (eg, vibration or motion) or electrical stimuli that a user may perceive through tactile or kinesthetic senses. According to one embodiment, the haptic module 179 may include, for example, a motor, a piezoelectric element, or an electrical stimulation device.

The camera module 180 may capture still images and moving images. According to one embodiment, the camera module 180 may include one or more lenses, image sensors, image signal processors, or flashes.

The power management module 188 may manage power supplied to the electronic device 101 . According to one embodiment, the power management module 188 may be implemented as at least part of a power management integrated circuit (PMIC), for example.

The battery 189 may supply power to at least one component of the electronic device 101 . According to one embodiment, the battery 189 may include, for example, a non-rechargeable primary cell, a rechargeable secondary cell, or a fuel cell.

The communication module 190 is a direct (eg, wired) communication channel or a wireless communication channel between the electronic device 101 and an external electronic device (eg, the electronic device 102, the electronic device 104, or the server 108). Establishment and communication through the established communication channel may be supported. The communication module 190 may include one or more communication processors that operate independently of the processor 120 (eg, an application processor) and support direct (eg, wired) communication or wireless communication. According to one embodiment, the communication module 190 is a wireless communication module 192 (eg, a cellular communication module, a short-range wireless communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module 194 (eg, : a local area network (LAN) communication module or a power line communication module). Among these communication modules, a corresponding communication module is a first network 198 (eg, a short-range communication network such as Bluetooth, wireless fidelity (WiFi) direct, or infrared data association (IrDA)) or a second network 199 (eg, legacy It may communicate with the external electronic device 104 through a cellular network, a 5G network, a next-generation communication network, the Internet, or a telecommunications network such as a computer network (eg, a LAN or a WAN). These various types of communication modules may be integrated as one component (eg, a single chip) or implemented as a plurality of separate components (eg, multiple chips). The wireless communication module 192 uses subscriber information (eg, International Mobile Subscriber Identifier (IMSI)) stored in the subscriber identification module 196 within a communication network such as the first network 198 or the second network 199. The electronic device 101 may be identified or authenticated.

The wireless communication module 192 may support a 5G network after a 4G network and a next-generation communication technology, for example, NR access technology (new radio access technology). NR access technologies include high-speed transmission of high-capacity data (enhanced mobile broadband (eMBB)), minimization of terminal power and access of multiple terminals (massive machine type communications (mMTC)), or high reliability and low latency (ultra-reliable and low latency (URLLC)). -latency communications)) can be supported. The wireless communication module 192 may support a high frequency band (eg, mmWave band) to achieve a high data rate, for example. The wireless communication module 192 uses various technologies for securing performance in a high frequency band, such as beamforming, massive multiple-input and multiple-output (MIMO), and full-dimensional multiplexing. Technologies such as input/output (FD-MIMO: full dimensional MIMO), array antenna, analog beam-forming, or large scale antenna may be supported. The wireless communication module 192 may support various requirements defined for the electronic device 101, an external electronic device (eg, the electronic device 104), or a network system (eg, the second network 199). According to one embodiment, the wireless communication module 192 is a peak data rate for eMBB realization (eg, 20 Gbps or more), a loss coverage for mMTC realization (eg, 164 dB or less), or a U-plane latency for URLLC realization (eg, Example: downlink (DL) and uplink (UL) each of 0.5 ms or less, or round trip 1 ms or less) may be supported.

The antenna module 197 may transmit or receive signals or power to the outside (eg, an external electronic device). According to one embodiment, the antenna module 197 may include an antenna including a radiator formed of a conductor or a conductive pattern formed on a substrate (eg, PCB). According to one embodiment, the antenna module 197 may include a plurality of antennas (eg, an array antenna). In this case, at least one antenna suitable for a communication method used in a communication network such as the first network 198 or the second network 199 is selected from the plurality of antennas by the communication module 190, for example. can be chosen A signal or power may be transmitted or received between the communication module 190 and an external electronic device through the selected at least one antenna. According to some embodiments, other components (eg, a radio frequency integrated circuit (RFIC)) may be additionally formed as a part of the antenna module 197 in addition to the radiator.

According to various embodiments, the antenna module 197 may form a mmWave antenna module. According to one embodiment, the mmWave antenna module includes a printed circuit board, an RFIC disposed on or adjacent to a first surface (eg, a lower surface) of the printed circuit board and capable of supporting a designated high frequency band (eg, mmWave band); and a plurality of antennas (eg, array antennas) disposed on or adjacent to a second surface (eg, a top surface or a side surface) of the printed circuit board and capable of transmitting or receiving signals of the designated high frequency band. can do.

At least some of the components are connected to each other through a communication method between peripheral devices (eg, a bus, general purpose input and output (GPIO), serial peripheral interface (SPI), or mobile industry processor interface (MIPI)) and signal ( e.g. commands or data) can be exchanged with each other.

According to an embodiment, commands or data may be transmitted or received between the electronic device 101 and the external electronic device 104 through the server 108 connected to the second network 199 . Each of the external electronic devices 102 or 104 may be the same as or different from the electronic device 101 . According to an embodiment, all or part of operations executed in the electronic device 101 may be executed in one or more external electronic devices among the external electronic devices 102 , 104 , or 108 . For example, when the electronic device 101 needs to perform a certain function or service automatically or in response to a request from a user or another device, the electronic device 101 instead of executing the function or service by itself. Alternatively or additionally, one or more external electronic devices may be requested to perform the function or at least part of the service. One or more external electronic devices receiving the request may execute at least a part of the requested function or service or an additional function or service related to the request, and deliver the execution result to the electronic device 101 . The electronic device 101 may provide the result as at least part of a response to the request as it is or additionally processed. To this end, for example, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technology may be used. The electronic device 101 may provide an ultra-low latency service using, for example, distributed computing or mobile edge computing. In another embodiment, the external electronic device 104 may include an internet of things (IoT) device. Server 108 may be an intelligent server using machine learning and/or neural networks. According to one embodiment, the external electronic device 104 or server 108 may be included in the second network 199 . The electronic device 101 may be applied to intelligent services (eg, smart home, smart city, smart car, or health care) based on 5G communication technology and IoT-related technology.

The server 108 is connected to the electronic device 101 and can provide a service to the connected electronic device 101 . In addition, the server 108 may proceed with a membership sign-up procedure, store and manage various types of information of users subscribed as members, and provide various purchase and payment functions related to services. In addition, the server 108 may share execution data of service applications executed in each of the plurality of electronic devices 101 in real time so that the service can be shared among users. This server 108 may have the same configuration as a conventional web server or service server in terms of hardware. However, in terms of software, it may include a program module that is implemented through any language such as C, C++, Java, Python, Golang, or kotlin and performs various functions. In addition, the server 108 is generally connected to an unspecified number of clients and/or other servers through an open computer network such as the Internet, and receives requests from clients or other servers to perform tasks and derives and provides work results. It means a computer system and the computer software (server program) installed for it. In addition, the server 108, in addition to the above-described server program, a series of application programs operating on the server 108 and various databases (DB: Database, hereinafter referred to as It should be understood as a broad concept including DB"). Accordingly, the server 108 classifies member registration information and various information and data about games, stores them in a DB, and manages them. This DB may be implemented inside or outside the server 108 . In addition, the server 108 may be implemented using server programs that are provided in various ways according to operating systems such as Windows, Linux, UNIX, and Macintosh on general server hardware, As a representative example, a web service can be implemented using IIS (Internet Information Server) used in a Windows environment and CERN, NCSA, APPACH, TOMCAT, etc. used in a Unix environment. In addition, the server 108 may interoperate with an authentication system and a payment system for user authentication of services or payment for purchases related to services.

The first network 198 and the second network 199 are a connection structure capable of exchanging information between nodes such as terminals and servers, or a network connecting the server 108 and the electronic devices 101 and 104. means (Network). The first network 198 and the second network 199 include Internet, Local Area Network (LAN), Wireless Local Area Network (Wireless Local Area Network), Wide Area Network (WAN), Personal Area Network (PAN), and 3G , 4G, LTE, 5G, Wi-Fi, etc., but are not limited thereto. The first network 198 and the second network 199 may be closed first networks 198 and second networks 199 such as LAN and WAN, but are preferably open such as the Internet. The Internet includes protocols such as the TCP/IP protocol, TCP, and User Datagram Protocol (UDP), and various services that exist in the upper layer, such as HTTP (HyperText Transfer Protocol), Telnet, FTP (File Transfer Protocol), and DNS (Domain Name System). ), Simple Mail Transfer Protocol (SMTP), Simple Network Management Protocol (SNMP), Network File Service (NFS), and Network Information Service (NIS). ) structure.

A database may have a general data structure implemented in a storage space (hard disk or memory) of a computer system using a database management program (DBMS). The database may have a data storage form in which data can be freely searched for (extracted), deleted, edited, added, and the like. Databases are relational database management systems (RDBMS) such as Oracle, Informix, Sybase, and DB2, or object-oriented database management such as Gemston, Orion, and O2. It can be implemented according to the purpose of an embodiment of the present disclosure using a system (OODBMS) and XML Native Databases such as Excelon, Tamino, and Sekaiju, and its functions may have appropriate fields or elements to achieve.

2 is a diagram showing the configuration of a program according to an embodiment.

2 is a block diagram 200 illustrating a program 140 according to various embodiments. According to one embodiment, the program 140 includes an operating system 142, middleware 144, or an application 146 executable in the operating system 142 for controlling one or more resources of the electronic device 101. can include The operating system 142 may include, for example, Android™, iOS™, Windows™, Symbian™, Tizen™, or Bada™. At least some of the programs 140 are, for example, preloaded in the electronic device 101 at the time of manufacture, or when used by a user, an external electronic device (eg, the electronic device 102 or 104), or a server ( 108)) can be downloaded or updated. All or part of program 140 may include a neural network.

The operating system 142 may control management (eg, allocation or reclamation) of one or more system resources (eg, process, memory, or power) of the electronic device 101 . Operating system 142 may additionally or alternatively include other hardware devices of electronic device 101 , such as input module 150 , sound output module 155 , display module 160 , audio module 170 . , sensor module 176, interface 177, haptic module 179, camera module 180, power management module 188, battery 189, communication module 190, subscriber identification module 196, or It may include one or more driver programs for driving the antenna module 197.

The middleware 144 may provide various functions to the application 146 so that the function or information provided from one or more resources of the electronic device 101 may be used by the application 146 . The middleware 144 includes, for example, the application manager 201, the window manager 203, the multimedia manager 205, the resource manager 207, the power manager 209, the database manager 211, and the package manager 213. ), connectivity manager 215, notification manager 217, location manager 219, graphics manager 221, security manager 223, call manager 225, or voice recognition manager 227. can

The application manager 201 may manage the life cycle of the application 146 , for example. The window manager 203 may manage one or more GUI resources used in a screen, for example. The multimedia manager 205 identifies, for example, one or more formats necessary for reproducing media files, and encodes or decodes a corresponding media file among the media files using a codec suitable for the selected format. can be done The resource manager 207 may manage a source code of the application 146 or a memory space of the memory 130 . The power manager 209 may manage, for example, the capacity, temperature, or power of the battery 189, and determine or provide related information necessary for the operation of the electronic device 101 by using corresponding information among them. . According to an embodiment, the power manager 209 may interoperate with a basic input/output system (BIOS) (not shown) of the electronic device 101 .

The database manager 211 may create, search, or change a database to be used by the application 146, for example. The package manager 213 may manage installation or update of applications distributed in the form of package files, for example. The connectivity manager 215 may manage, for example, a wireless connection or a direct connection between the electronic device 101 and an external electronic device. The notification manager 217 may provide a function for notifying a user of occurrence of a designated event (eg, an incoming call, message, or alarm), for example. The location manager 219 may manage location information of the electronic device 101, for example. The graphic manager 221 may manage, for example, one or more graphic effects to be provided to a user or a user interface related thereto.

Security manager 223 may provide system security or user authentication, for example. The telephony manager 225 may manage, for example, a voice call function or a video call function provided by the electronic device 101 . The voice recognition manager 227 transmits, for example, the user's voice data to the server 108, and at least partially based on the voice data, a command corresponding to a function to be performed in the electronic device 101; Alternatively, text data converted at least partially based on the voice data may be received from the server 108 . According to one embodiment, the middleware 244 may dynamically delete some existing components or add new components. According to one embodiment, at least part of the middleware 144 may be included as part of the operating system 142 or may be implemented as separate software different from the operating system 142 .

The application 146 includes, for example, a home 251, a dialer 253, an SMS/MMS 255, an instant message (IM) 257, a browser 259, a camera 261, and an alarm 263. , Contacts (265), Voice Recognition (267), Email (269), Calendar (271), Media Player (273), Albums (275), Watch (277), Health (279) (e.g. exercise or blood sugar) measurement of biometric information) or environmental information 281 (eg, measurement of atmospheric pressure, humidity, or temperature information). According to an embodiment, the application 146 may further include an information exchange application (not shown) capable of supporting information exchange between the electronic device 101 and an external electronic device. The information exchange application may include, for example, a notification relay application configured to transmit designated information (eg, a call, message, or alarm) to an external electronic device, or a device management application configured to manage an external electronic device. have. The notification relay application, for example, transmits notification information corresponding to a designated event (eg, mail reception) generated in another application (eg, the email application 269) of the electronic device 101 to an external electronic device. can Additionally or alternatively, the notification relay application may receive notification information from an external electronic device and provide the notification information to the user of the electronic device 101 .

The device management application is, for example, a power source (eg, turn-on or turn-off) of an external electronic device that communicates with the electronic device 101 or some component thereof (eg, a display module or a camera module of the external electronic device). ) or functions (eg brightness, resolution, or focus). The device management application may additionally or alternatively support installation, deletion, or update of an application operating in an external electronic device.

Throughout this specification, a neural network, a neural network, and a network function may be used interchangeably. A neural network may consist of a set of interconnected computational units, which may be generally referred to as “nodes”. These “nodes” may also be referred to as “neurons”. A neural network includes at least two or more nodes. Nodes (or neurons) constituting neural networks may be interconnected by one or more “links”.

In a neural network, two or more nodes connected through a link may form a relative relationship of an input node and an output node. The concept of an input node and an output node is relative, and any node in an output node relationship with one node may have an input node relationship with another node, and vice versa. As described above, the input node to output node relationship can be created around the link. More than one output node can be connected to one input node through a link, and vice versa.

In a relationship between an input node and an output node connected through one link, the value of the output node may be determined based on data input to the input node. Here, a node interconnecting the input node and the output node may have a weight. The weight may be variable, and may be changed by a user or an algorithm in order for the neural network to perform a desired function. For example, when one or more input nodes are interconnected by respective links to one output node, the output node is set to a link corresponding to values input to input nodes connected to the output node and respective input nodes. An output node value may be determined based on the weight.

As described above, in a neural network, two or more nodes are interconnected through one or more links to form an input node and output node relationship in the neural network. Characteristics of the neural network may be determined according to the number of nodes and links in the neural network, an association between the nodes and links, and a weight value assigned to each link. For example, when there are two neural networks having the same number of nodes and links and different weight values between the links, the two neural networks may be recognized as different from each other.

3 illustrates a method of transmitting watermarked data to at least one device using a neural network according to an exemplary embodiment. One embodiment of FIG. 3 may be combined with various embodiments of the present disclosure.

Referring to FIG. 3 , in step S301, a server (eg, the server 108 of FIG. 1) may obtain original data based on document information input to the server. Here, the document information may be information about a document including text data or image data. For example, document information may include word processor software files (eg, doc files, docx files, HWP files), pdf (portable document format) files, and image files. For example, the document information may be received from at least one device (eg, the electronic device 101 of FIG. 1 ). Here, the original data may include values related to pixels of the document. For example, the original data may include text information included in the document and background information included in the document. For example, the text information may include values related to the number of texts, the size of the texts, and the color space of the texts. For example, the background information may include a value related to a color space of the background. Here, the color space may be a space concept expressing a color system in three dimensions. For example, the color space may be expressed in various ways such as RGB (red, green, blue), HSV (hue, saturation, value), YUV, and the like. For example, RGB may be a method expressed as coordinate values for three types of colors, red, green, and blue. For example, HSV may be a method that considers hue, saturation, and brightness information (eg, value). For example, in YUV, color can be expressed using two axes, U and V, with a brightness component on the y-axis. Here, the U-axis may be a value obtained by subtracting the brightness component from blue, and the V-axis may be a value obtained by subtracting the brightness component from red.

In step S302, the server may determine pattern information through a pattern generation model using the neural network based on the original data, at least one recipient name, and a pre-input sender name.

For example, the at least one recipient may be selected from a list of recipients pre-stored in the server. For example, the pre-stored recipient list may include a name for the recipient and contact information for the recipient. For example, when a corrected name for the at least one recipient is input to the server, the name for the selected recipient may be changed to the corrected name. For example, the contact information for the recipient may include an email address, a messenger ID, and a phone number. For example, the pre-stored recipient list may be entered or updated via a spreadsheet program file (eg, an Excel file).

For example, the pattern information may include information about the first object and the second object constituting the watermark and pattern information composed of a combination of the first object and the second object. For example, the information about the first object and the second object may include information about the size of the first object and the second object and information about the shape of the first object and the second object. have. For example, the first object may be created based on the name of the at least one recipient. For example, the first object may have the name of the recipient in any one of various shapes such as a triangle shape, a square shape, a circle shape, and a star shape. For example, the second object may be created based on the name of the sender. For example, the second object may have the name of the sender in any one of various shapes such as a triangle shape, a square shape, a circle shape, and a star shape. For example, a pattern composed of a combination of the first object and the second object may include two or more patterns. Here, the pattern may include various patterns such as a horizontal pattern, a vertical pattern, a diagonal pattern, a dot pattern, and a lattice pattern. That is, for example, a pattern composed of a combination of the first object and the second object may be created as a horizontal pattern and a vertical pattern.

For example, the neural network may include an input layer, one or more hidden layers, and an output layer. For example, each training data consisting of a plurality of original data, data with a plurality of watermarks inserted, and correct answer pattern information is input to the input layer of the neural network, passes through one or more hidden layers and an output layer, and is output. It is output as a vector, and the output vector is input to a loss function layer connected to the output layer, and the loss function layer outputs a loss value using a loss function that compares the output vector with the correct vector for each learning data. Parameters of the neural network may be learned in a direction in which the loss value decreases, and the pattern generation model may be generated.

Therefore, the server configures the pattern included in the watermark even when a part of the document is extracted by providing a watermark including a pattern composed of the first object and the second object based on the pattern information determined through the neural network. It is possible to check the recipient and the sender through the first object and the second object. In addition, since the watermark includes two or more patterns, an outflow path can be specified through the watermark even for the same documents by the recipient and the sender.

For example, the server may vectorize the size of text included in the original data, a color space value related to the text, and a color space related value related to the background through data preprocessing. For example, the server may vectorize a pattern composed of sizes and shapes of first and second objects included in watermarked data through data preprocessing and a combination of the first and second objects.

For example, the size of the first object and the second object may be determined based on the size of text included in the original data. For example, the sizes of the first object and the second object may be determined to be smaller than the size of text included in the original data by a preset ratio. As a result, the sizes of the first object and the second object are not always fixed, and may be different according to the size of text included in the original data, and a watermark may be formed in a more natural pattern.

In step S303, the server may embed a watermark including the first object and the second object into the original data based on the pattern information and previously input watermark setting information. For example, the server may determine a phrase set as a watermark as a pre-input phrase. For example, the setting information for the watermark may include the location, shape, color, slant, font, size, and transparency of the watermark. For example, setting information for the watermark may be pre-entered into the server. For example, the watermark setting information may be received by the server from a device connected to the server.

Additionally, for example, if a pre-input phrase does not exist, the server may determine a phrase set as the watermark based on the document information. For example, the server may determine one of a plurality of categories based on the document information. For example, the plurality of categories may include official documents, planning documents, draft documents, reports, specifications, contracts, and business letters. For example, words related to each of a plurality of categories may be pre-stored in the server. For example, a plurality of phrases for each of a plurality of categories may be pre-stored in the server. For example, the server may determine a plurality of first words that are matched with words related to each of the plurality of categories from among words included in the document information, and a second most frequent second word among the plurality of first words. words can be determined. Thereafter, the server may randomly set a phrase for the watermark among a plurality of phrases corresponding to the category including the second word.

For example, when the document information is an image file, the server may recognize texts included in the image file through various optical character recognition (OCR) techniques. For example, the server may pre-process metadata such as brightness or color of the image, determine a location where texts in the image exist, set a bounding box for the texts at that location, and , texts within the bounding box can be recognized. Thereafter, the server may determine one category from among the plurality of categories by detecting a word matching words related to each of the plurality of categories among the recognized texts. That is, for example, the server may determine a plurality of first words that match words related to each of a plurality of categories from among the recognized texts, and determine a second word having the highest frequency among the plurality of first words. can Thereafter, the server may randomly set a phrase for the watermark among a plurality of phrases corresponding to the category including the second word. For this reason, even if a phrase for a watermark is not input, the server can automatically provide an appropriate watermark phrase according to the document type.

For example, based on the distribution of text included in the original data, a plurality of first regions of the original data may be determined through the Density-Based Spatial Clustering of Applications with Noise (DBSCAN) technique. For example, the server may determine the plurality of first areas as n areas through a clustering technique using a neural network based on the distribution of text included in the original data. For example, clustering may refer to unsupervised learning that groups data having similar properties into a certain number of clusters. For example, n areas may be determined through a DBSCAN technique based on text included in the original data. For example, DBSCAN assumes that if a certain point belongs to a cluster, it must be located close to many other points in the cluster, and for this calculation, the radius and minimum points are can be used For example, the diameter may be a radius based on a specific data element, and may be referred to as a dense area. For example, the minimum element could indicate how many elements are needed around the core point to specify a core point. For example, the diameter and minimum element may be preset. In addition, each element of the data set can be divided into a core, a border, and an outlier point.

For example, the server may vectorize coordinates corresponding to each text included in the original data through data preprocessing. For example, the server may regard the coordinates of the vectorized text as one element, and the server may check the size of the diameter for each element and search for how many elements are around. Thereafter, if m or more elements exist within the diameter range, the server may determine the corresponding element as a key element. And, the server may determine an element included within a range of a diameter from the core element as a boundary element. Also, the server may determine an element not included within a diameter range from the core element as an outlier element, and the outlier element may be excluded from the corresponding cluster. In addition, when the distance between core elements is smaller than the diameter, the server may classify the corresponding elements into the same cluster.

For this reason, the server can more efficiently embed the watermark into the original data by determining an area where the watermark can be inserted according to the distribution of text included in the original data.

For example, one or more second areas in which the watermark can be inserted may be determined among the plurality of first areas. For example, among the plurality of first areas, an area in which a ratio of text included in the first area is greater than or equal to a preset ratio may be determined as the second area. For example, the ratio of the text included in the first area may be the ratio occupied by the texts included in the first area with respect to the first area. For example, the ratio of text included in the first area may be determined based on the size of the first area, the average size of text included in the first area, and the number of texts included in the first area. . For example, the server may insert the watermark into an area where the ratio of text included in the first area is greater than or equal to a preset ratio among the plurality of first areas.

For example, the preset ratio may be determined by Equation 1 below.

In Equation 1, r _w is the preset ratio, T _s is the average size of text included in the original data, n _T is the number of texts included in the original data, and D _s is the size of the document related to the original data, n is the number of the plurality of first areas, A _i is the size of the ith first area, and A _o is the size of the currently selected first area. have. Therefore, the criterion for the text ratio is not always the same, and the area to insert the watermark can be more accurately determined by changing it in consideration of the text ratio of the entire document related to the original data.

Alternatively, for example, the server may insert the watermark into at least one first area corresponding to the input value based on an input value for at least one first area among the plurality of first areas. For example, the server may transmit information on the plurality of first areas to a device connected to the server, and may receive an input value for at least one first area from a device connected to the server.

For example, a watermark related to the sender may be additionally inserted into the watermark-embedded data. For example, the server may receive information about a watermark related to a sender from a device connected to the server. For example, the information on the watermark related to the sender may include a text of the watermark related to the sender, a location, shape, color, slant, font, size, and transparency of the watermark related to the sender. For example, the server may additionally insert a watermark associated with the sender into the watermark-embedded data based on information on the watermark associated with the sender.

For example, a different range of transparency for the watermark may be determined based on the clarity of the original data. For example, the minimum value of the transparency may be determined by Equation 2 below.

In Equation 2, a _min is the minimum value of transparency, r _t is the ratio of text included in the original data, B _b is an average brightness value for the background included in the original data, and B _t is An average brightness value of text included in the original data, S may be sharpness of the original data, and a may be a weight related to the watermark. Here, the brightness value may be a value for brightness information in the above-described HSV. Here, the text ratio may be a ratio of texts included in the original data to a document related to the original data. Here, the transparency may have a value between 0 and 100. For example, the transparency may be expressed as a two-digit hexadecimal number located in front of a code value in which a value related to a color space (eg, an RGB value) is expressed in hexadecimal. For example, if the transparency is 1%, it may be converted into '03', if the transparency is 11%, it may be converted into '1C', and if the transparency is 85%, it may be converted into 'D9'. For example, a may be a constant of 1 or greater. For example, a may be 2. For example, the a may be set differently according to the pattern of the first object and the second object constituting the watermark and the size of the watermark.

For example, sharpness of the original data may be measured using a pixel value included in the document information. In this case, for example, an edge-based method for estimating the sharpness by measuring the spreading degree of an edge in the document may be used. For example, the server applies a Gaussian blur filter to the document, compares a gradient value for the document to which the Gaussian blur filter is applied, and a gradient value for original data of the document to determine sharpness. can be measured Here, the gradient may be an instantaneous rate of change in all directions from all input values of the multivariate function. For example, the gradient magnitude can be defined as the root mean square of the image direction gradient along two orthogonal directions. For example, the gradient may be calculated by processing the document with a linear filter such as a Roberts filter, a Sobel filter, and a Prewitt filter. That is, a region with high sharpness may have a high gradient value when a pixel value rapidly changes at the boundary, and a low gradient value may be determined when a blur occurs at the boundary.

For example, sharpness of the original data may be determined by Equation 3 below.

In Equation 3, S is the sharpness of the original data, k is the total number of gradients, G _xj is a set of j-th gradient values in the x-direction, and G _yj is a set of j-th gradient values in the y-direction can be Here, the S value may be a value between 0 and 100.

Accordingly, the server may provide a watermark more suitable for the provided document by adjusting the range of transparency of the watermark according to the measured sharpness of the document.

Additionally, for example, when the sharpness of the original data is less than the preset sharpness, the server may increase the sharpness of the original data using a sharpening filter. For example, the server applies a Laplacian filter to the original data to detect an area where the pixel value is greatly changed, and adds the area where the pixel value is greatly changed to the original data. Sharpness can be increased. Here, the Laplacian filter is a filter to which second-order derivative is applied, and may be a filter for detecting a boundary. Alternatively, for example, the server may obtain a high-frequency component of the original data based on an unsharp mask filter and add the high-frequency component to the original data to increase sharpness of the original data. . Here, the unsharp mask filter may be applied to the original data to obtain smooth original data. For example, the unsharp mask filter may be an average value filter. That is, an arithmetic mean value of one pixel in the original data and pixels surrounding the corresponding pixel may be stored as a result data pixel value through the average filter. For example, the unsharp mask filter may be a Gaussian filter. That is, in the Gaussian filter, a value at the center may have a high weight, and a value farther from the center may have a smaller weight. Through this, the weighted arithmetic average value of the original data may be stored as the result data pixel value. Accordingly, the server may insert a watermark after supplementing the document sharpness of the low sharpness document without using the low sharpness document as it is.

In step S304, the server may transmit the watermarked data to at least one device related to the at least one recipient based on the contact information of the recipient.

For example, the server may transmit a message including the watermarked data to at least one device related to the at least one recipient based on the contact information for the recipient. For example, the server may transmit a message received from a device connected to the server to a device related to the at least one recipient including data in which the watermark is inserted.

For example, a watermark transmitted to the at least one device may be determined differently for each of the at least one device. For example, when the name of the recipient is different for each of the at least one device, the watermark transmitted to the at least one device may be different for each of the at least one device.

For example, the server may transmit the watermarked data to the at least one device based on at least one of an email address, a messenger ID, and a phone number. For example, the server may transmit watermarked data to the first device through an email address and transmit watermarked data to the second device through a messenger ID. For example, the server may transmit a multimedia messaging service (MMS) message including watermarked data to at least one device through the phone number. Here, the MMS message is a message capable of sending multimedia such as photos and videos.

According to an embodiment of the present disclosure, the server may transmit information related to an interface for embedding a watermark to a device connected to the server (eg, the electronic device 101 of FIG. 1 ). For example, the information related to the interface for embedding the watermark includes information related to a basic interface including a list of recipients, information related to an interface for setting the watermark, and a message transmitted together with data in which the watermark is inserted. Information related to an interface for reserving the message and information related to an interface to be confirmed before sending the message may be included.

Hereinafter, an example of embedding a watermark in a specific document based on a server and a device connected to the server and transmitting the specific document with the watermark inserted to a device associated with at least one recipient together with a message will be described. Also, the interface described in this disclosure is one embodiment, and is not limited to the above embodiment, and may be implemented as various interfaces.

4 is an example of a basic screen including a recipient list according to an embodiment. 5 is an example of a screen related to watermark setting according to an exemplary embodiment. 6 is an example of a screen for reserving an entire message transmitted together with watermarked data according to an embodiment. 7 is an example of a screen before sending an entire message according to an embodiment. The embodiments of FIGS. 4 to 7 can be combined with various embodiments of the present disclosure.

Referring to FIG. 4 , the basic screen including the recipient list may include the recipient's name, watermark content, and recipient's contact information. For example, a device connected to the server may input the recipient list stored in the Excel file to the server through an Excel file. Also, for example, a device connected to the server may change the recipient list pre-stored in the server through an Excel file.

For example, a device connected to the server may search for at least one recipient from a recipient list pre-stored in the server and select the at least one recipient. For example, a device connected to a server may use a phrase for a watermark previously entered into the server. For example, a device connected to a server may change and use a phrase for a watermark pre-input in the server. For example, a device connected to a server may pre-enter a form for a document to the server.

Additionally, for example, when a pre-input phrase for a watermark does not exist in the server, a device connected to the server may receive a phrase for the watermark from the server. For example, a phrase for the watermark may be determined by the server based on document information.

Referring to FIG. 5 , a screen related to setting a watermark may include detailed settings for the watermark and a preview window for a document in which the watermark is inserted. For example, the watermark may include a first watermark composed of a name for the recipient and a name for the sender, and a second watermark set by the sender. For example, detailed settings for the first watermark may include settings for position, shape, color, inclination, font, size, and transparency. For example, detailed settings for the second watermark may include settings for position, shape, color, slant, font, size, and transparency. For example, a preview window for a watermark-embedded document may include a first watermark and a second watermark-embedded document image.

Referring to FIG. 6 , a screen for reserving an entire message to be transmitted together with watermarked data may include the entire message, a reservation date, a reservation time, and whether or not to send a reservation. For example, a device connected to the server may transmit the watermark-embedded document to at least one device together with the entire message. For example, when reservation delivery is set, a device connected to the server may transmit the watermark-embedded document together with the entire message to at least one device at the time when reservation delivery is set.

Referring to FIG. 7 , the screen before sending the entire message may include a recipient list and a window for previewing a document having a watermark corresponding to a recipient included in the recipient list. For example, a device connected to the server may display a document having a watermark corresponding to a recipient included in the recipient list on the screen.

8 is an example of a watermark according to an embodiment. The embodiments of FIG. 8 can be combined with various embodiments of the present disclosure.

Referring to FIG. 8 , the watermark may include a pattern composed of a name of a recipient (eg, Gildong Hong) and a name of a sender (eg, Shimchungi). Therefore, even when a part of the watermark-embedded document is extracted, the recipient and the sender can be identified through the name of the recipient and the name of the sender.

Additionally, for example, the name of the recipient and the name of the sender constituting the watermark may be formed in any one of various shapes such as a triangle shape, a square shape, a circle shape, and a star shape. . For example, a pattern composed of a combination of the name of the recipient and the name of the sender may include two or more patterns. Here, the pattern may include various patterns such as a horizontal pattern, a vertical pattern, a diagonal pattern, a dot pattern, and a lattice pattern. That is, for example, a pattern composed of the name of the recipient and the name of the sender may be generated as a horizontal pattern and a vertical pattern.

9 is an example of a process of setting a watermark to be inserted into original data according to an embodiment. The embodiment of FIG. 9 can be combined with various embodiments of the present disclosure.

Referring to FIG. 9 , in step S910, the server may obtain original data based on document information input to the server.

In step S920, the server may determine whether the sharpness of the original data is less than the preset sharpness. For example, the server may determine sharpness of the original data by Equation 3 above.

In step S930, if the sharpness of the original data is less than the preset sharpness, the server may increase the sharpness of the original data by applying a sharpening filter to the original data.

In step S940, the server may determine a range of transparency for the watermark based on the sharpness of the original data. For example, the minimum value in the range of transparency may be determined by Equation 2 above. For example, the maximum value in the range of transparency may be determined by Equation 4 below.

In Equation 4, a _max is the maximum value of transparency, r _t is the ratio of text included in the original data, B _b is the average brightness value of the background included in the original data, and B _t is An average brightness value of text included in the original data, and S may be a sharpness value of the original data.

For example, when the sharpness of the original data is greater than or equal to the preset sharpness, the server may determine a range of transparency for the watermark based on the sharpness of the original data.

In step S950, the server may determine a plurality of first areas of the original data. For example, the server may determine a plurality of first areas of the original data through the DBSCAN technique based on the distribution of text included in the original data.

In step S960, the server may determine whether the ratio is greater than or equal to a preset ratio for each of the plurality of first areas. For example, for each of the plurality of first areas, the server may determine whether the ratio of text included in the first area is greater than or equal to a preset ratio. For example, the preset ratio may be determined by Equation 1 above.

In step S970, the server may determine a first area equal to or greater than a preset ratio as a second area. For example, the second area may be an area where the watermark can be inserted. For example, the second area may be one or more.

In step S980, the server may insert a watermark into one or more second areas. For example, a watermark may be inserted into a second area corresponding to an input value received from a device connected to the server among one or more second areas.

The embodiment of FIG. 9 is an example, and the server may independently perform a process of determining a preset sharpness and a process of determining a preset ratio. Or, for example, the server may perform the above determination processes in parallel.

10 is a block diagram illustrating a configuration of a server according to an exemplary embodiment. One embodiment of FIG. 10 may be combined with various embodiments of the present disclosure.

As shown in FIG. 10 , the server 1000 may include a processor 1010, a communication unit 1020, and a memory 1030. However, not all components shown in FIG. 10 are essential components of the server 1000 . The server 1000 may be implemented with more components than those shown in FIG. 10 , or the server 1000 may be implemented with fewer components than those shown in FIG. 10 . For example, the server 1000 according to some embodiments may further include a user input interface (not shown), an output unit (not shown), etc. in addition to the processor 1010, the communication unit 1020, and the memory 1030. .

The processor 1010 typically controls the overall operation of the server 1000. The processor 1010 may include one or more processors to control other elements included in the server 1000 . For example, the processor 1010 may generally control the communication unit 1020 and the memory 1030 by executing programs stored in the memory 1030 . In addition, the processor 1010 may perform the functions of the server 1000 described in FIGS. 3 to 9 by executing programs stored in the memory 1030 .

According to an embodiment, the processor 1010 may obtain original data based on document information input to the server 1000 . The processor 1010 may receive the document information from a device connected to the server 1000 through a network through the communication unit 1020 .

The processor 1010 may determine pattern information through a pattern generation model using the neural network based on the original data, at least one recipient name and pre-input sender name through the memory 1030 . For example, the at least one recipient may be selected from a list of recipients pre-stored in the server.

For example, the pre-stored recipient list may include a name for the recipient and contact information for the recipient. For example, the pattern information may include information about the first object and the second object constituting the watermark and pattern information composed of a combination of the first object and the second object.

The processor 1010 may embed a watermark including the first object and the second object into the original data based on the pattern information and previously input watermark setting information through the memory 1030. .

The processor 1010 may transmit the watermark-embedded data to at least one device related to the at least one recipient based on the contact information for the recipient through the communication unit 1020 .

For example, the first object may be created based on the name of the at least one recipient. For example, the second object may be created based on the name of the sender. For example, a watermark transmitted to the at least one device may be determined differently for each of the at least one device.

For example, the information about the first object and the second object may include information about the size of the first object and the second object and information about the shape of the first object and the second object. have. For example, a pattern composed of a combination of the first object and the second object may include two or more patterns.

The processor 1010 may determine the size of the first object and the second object based on the size of text included in the original data.

The processor 1010 may determine a plurality of first areas of the original data through Density-Based Spatial Clustering of Applications with Noise (DBSCAN) technique based on the distribution of text included in the original data. The processor 1010 may determine one or more second areas in which the watermark may be inserted among the plurality of first areas. The processor 1010 may embed the watermark on at least one area among the one or more second areas.

The processor 1010 may differently determine a range of transparency for the watermark based on the sharpness of the original data through the memory 1030 . For example, the processor 1010 may determine the minimum value of the transparency by Equation 2 described above, and may determine the maximum value of the transparency by Equation 4 described above.

For example, the neural network may include an input layer, one or more hidden layers, and an output layer. The processor 1010 inputs each learning data composed of a plurality of original data, data with a plurality of watermarks inserted, and correct answer pattern information to the input layer of the neural network through the memory 1030, and the one or more hidden You can create output vectors by passing layers and output layers. The processor 1010 inputs the output vector to a loss function layer connected to the output layer through the memory 1030, and a loss function that compares the output vector with the correct answer vector for each training data through the loss function layer The pattern generation model may be generated by outputting a loss value using and learning parameters of the neural network in a direction in which the loss value decreases.

The communication unit 1020 may include one or more components that allow the server 1000 to communicate with other devices (not shown) and servers (not shown). The other device (not shown) may be a computing device such as the server 1000 or a sensing device, but is not limited thereto. The communication unit 1020 may receive a user input from another electronic device or data stored in an external device from an external device through a network.

For example, the communication unit 1020 may transmit/receive a message for establishing a connection with at least one device. The communication unit 1020 may transmit information generated by the processor 1010 to at least one device connected to the server. The communication unit 1020 may receive information from at least one device connected to the server. The communication unit 1020 may transmit information related to the received information in response to information received from at least one device.

The memory 1030 may store programs for processing and control of the processor 1010 . For example, the memory 1030 may store information input to a server or information received from another device through a network. Also, the memory 1030 may store data generated by the processor 1010 . For example, the memory 1030 may include pattern information generated by the processor 1010, information about a first object, a second object, information about a pattern composed of a combination of the first object and the second object, and information about a watermark. , a plurality of first areas, one or more second areas, and sharpness of original data may be stored. The memory 1030 may store information input to or output from the server 1000 . For example, the memory 1030 may store a preset ratio for the size of text, a preset ratio for text ratio, a preset sharpness for original data, a pre-set name for the sender and a pre-set watermark for the original data. Settings information can be saved.

The memory 1030 is a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (eg SD or XD memory, etc.), RAM (RAM, Random Access Memory) SRAM (Static Random Access Memory), ROM (Read-Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), PROM (Programmable Read-Only Memory), magnetic memory, magnetic disk , an optical disk, and at least one type of storage medium.

The embodiments described above may be implemented as hardware components, software components, and/or a combination of hardware components and software components. For example, the devices, methods and components described in the embodiments may include, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate (FPGA). array), programmable logic units (PLUs), microprocessors, or any other device capable of executing and responding to instructions. A processing device may run an operating system (OS) and one or more software applications running on the operating system. A processing device may also access, store, manipulate, process, and generate data in response to execution of software. For convenience of understanding, there are cases in which one processing device is used, but those skilled in the art will understand that the processing device includes a plurality of processing elements and/or a plurality of types of processing elements. It can be seen that it can include. For example, a processing device may include a plurality of processors or a processor and a controller. Other processing configurations are also possible, such as parallel processors.

Software may include a computer program, code, instructions, or a combination of one or more of the foregoing, which configures a processing device to operate as desired or processes independently or collectively. The device can be commanded. Software and/or data may be any tangible machine, component, physical device, virtual equipment, computer storage medium or device, intended to be interpreted by or provide instructions or data to a processing device. , or may be permanently or temporarily embodied in a transmitted signal wave. Software may be distributed on networked computer systems and stored or executed in a distributed manner. Software and data may be stored on one or more computer readable media.

The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer readable medium. The computer readable medium may include program instructions, data files, data structures, etc. alone or in combination. Program commands recorded on the medium may be specially designed and configured for the embodiment or may be known and usable to those skilled in computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic media such as floptical disks. - includes hardware devices specially configured to store and execute program instructions, such as magneto-optical media, and ROM, RAM, flash memory, and the like. Examples of program instructions include high-level language codes that can be executed by a computer using an interpreter, as well as machine language codes such as those produced by a compiler. The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the embodiments, and vice versa.

As described above, although the embodiments have been described with limited drawings, those skilled in the art can apply various technical modifications and variations based on the above. For example, the described techniques may be performed in an order different from the method described, and/or components of the described system, structure, device, circuit, etc. may be combined or combined in a different form than the method described, or other components may be used. Or even if it is replaced or substituted by equivalents, appropriate results can be achieved.

Therefore, other implementations, other embodiments, and equivalents of the claims are within the scope of the following claims.

Claims (12)

A method in which a server transmits watermarked data to at least one device using a neural network, the method comprising:
obtaining original data based on document information input to the server;
Determining pattern information through a pattern generation model using the neural network based on the original data, the name of at least one recipient, and the name of the sender input in advance - The at least one recipient is pre-stored in the server selected from recipient list-;
The pre-stored recipient list includes names of recipients and contact information about recipients;
The pattern information includes information about a first object and a second object constituting a watermark and information about a pattern composed of a combination of the first object and the second object,
inserting a watermark including the first object and the second object into the original data based on the pattern information and previously input watermark setting information; and
Transmitting the watermarked data to at least one device related to the at least one recipient based on the contact information for the recipient,
The first object is created based on the name of the at least one recipient,
The second object is created based on the name of the sender,
The watermark transmitted to the at least one device is determined differently for each of the at least one device,
A range of transparency for the watermark is determined differently based on the clarity of the original data;
The minimum value of the transparency is determined by the following equation,

In the above equation, a _min is the minimum value of the transparency, r _t is the ratio of text included in the original data, B _b is the average brightness value of the background included in the original data, and the B _t Is the average brightness value of text included in the original data, S is the sharpness of the original data, and a is a weight related to the watermark.
Way.
According to claim 1,
The information about the first object and the second object includes information about the size of the first object and the second object and information about the shape of the first object and the second object,
The pattern composed of the combination of the first object and the second object includes two or more patterns,
The size of the first object and the second object is determined based on the size of text included in the original data.
Way.
According to claim 1,
A plurality of first regions are determined for the original data through a Density-Based Spatial Clustering of Applications with Noise (DBSCAN) technique based on a distribution of text included in the original data;
One or more second areas in which the watermark can be inserted are determined among the plurality of first areas;
Among the one or more second areas, the watermark is inserted.
Way.
delete According to claim 1,
The neural network includes an input layer, one or more hidden layers and an output layer;
Each learning data consisting of a plurality of original data, data with a plurality of watermarks inserted, and correct answer pattern information is input to the input layer of the neural network, passes through the one or more hidden layers and output layers, and is output as an output vector. , The output vector is input to a loss function layer connected to the output layer, and the loss function layer outputs a loss value using a loss function that compares the output vector with the correct answer vector for each learning data, and outputs a loss value, Parameters of the network are learned in a direction in which the loss value becomes smaller, and the pattern generation model is generated.
Way.
In a server including at least one processor, at least one memory and a communication unit,
The at least one processor is:
Acquiring original data based on document information input to the server;
Determining pattern information through a pattern generation model using a neural network based on the original data, at least one name of a recipient, and a pre-input name of a sender;
the at least one recipient is selected from a list of recipients pre-stored in the server;
The pre-stored recipient list includes names of recipients and contact information about recipients;
The pattern information includes information on a first object and a second object constituting a watermark and pattern information composed of a combination of the first object and the second object,
inserting a watermark including the first object and the second object into the original data based on the pattern information and pre-input setting information for the watermark;
Transmitting the watermarked data to at least one device related to the at least one recipient based on the contact information for the recipient;
The first object is created based on the name of the at least one recipient,
The second object is created based on the name of the sender,
The watermark transmitted to the at least one device is determined differently for each of the at least one device,
A range of transparency for the watermark is determined differently based on the clarity of the original data;
The minimum value of the transparency is determined by the following equation,

In the above equation, a _min is the minimum value of the transparency, r _t is the ratio of text included in the original data, B _b is the average brightness value of the background included in the original data, and the B _t Is the average brightness value of text included in the original data, S is the sharpness of the original data, and a is a weight related to the watermark.
server.
According to claim 6,
The information about the first object and the second object includes information about the size of the first object and the second object and information about the shape of the first object and the second object,
The pattern composed of the combination of the first object and the second object includes two or more patterns,
The size of the first object and the second object is determined based on the size of text included in the original data.
server.
According to claim 6,
A plurality of first regions are determined for the original data through a Density-Based Spatial Clustering of Applications with Noise (DBSCAN) technique based on a distribution of text included in the original data;
One or more second areas in which the watermark can be inserted are determined among the plurality of first areas;
Among the one or more second areas, the watermark is inserted.
server.
delete According to claim 6,
The neural network includes an input layer, one or more hidden layers and an output layer;
Each learning data consisting of a plurality of original data, data with a plurality of watermarks inserted, and correct answer pattern information is input to the input layer of the neural network, passes through the one or more hidden layers and output layers, and is output as an output vector. , The output vector is input to a loss function layer connected to the output layer, and the loss function layer outputs a loss value using a loss function that compares the output vector with the correct answer vector for each learning data, and outputs a loss value, Parameters of the network are learned in a direction in which the loss value becomes smaller, and the pattern generation model is generated.
server.
A computer program stored in a computer readable recording medium in order to execute the method of claim 1 in combination with hardware.
A computer readable recording medium storing a computer program combined with hardware to execute the method of claim 1.

Images