KR20240007467A - Appratus and method for digital media watermarking using frame data - Google Patents

Abstract

The present invention relates to an apparatus and method for inserting a watermark to hide information in digital media, and more specifically, to a device and method for hiding information using a sequence of watermarks inserted on a frame-by-frame basis. It relates to devices and methods. A watermark insertion device according to an aspect of the present invention for solving the above-described problem includes a frame data generator for generating frame data, an insertion symbol generator for generating an insertion symbol, and watermark information based on the insertion symbol. a watermark information generation unit that generates watermark information, and a watermark insertion unit that generates watermarked frame data by inserting a watermark into the frame data based on the watermark information, wherein the insertion symbol is at least a first symbol. and a second symbol, and the watermark information may be generated differently depending on the symbol of the inserted symbol.

Description

Digital media watermarking device and method using frame data {APPRATUS AND METHOD FOR DIGITAL MEDIA WATERMARKING USING FRAME DATA}

The present invention relates to an apparatus and method for inserting a watermark to hide information in digital media, and more specifically, to a device and method for hiding information using a sequence of watermarks inserted on a frame-by-frame basis. It relates to devices and methods.

Recently, almost all tasks or services in life, including office work, payment services, certificate issuance, banking, and e-commerce, are performed online and in a computerized environment. In such an online and computerized environment, digital media becomes the object of information creation, viewing, storage, editing, and transmission. At this time, digital media may refer to various forms of digital information including images, text, and multimedia, such as digital documents, images, voices, videos, and 3D data.

However, due to the nature of digital information, these digital media are easily copied or transmitted without loss of the information, so they are quite vulnerable to leakage through illegal activities. Therefore, in order to provide a sound digital content use environment, a protection method, that is, a security system, that can prevent digital content from illegal activities is essential. For example, digital media can be protected through encryption and decryption systems, watermarking or fingerprinting systems, and screen capture prevention systems.

One of these methods is digital watermarking. Many commonly used digital watermarking methods include the process of embedding certain additional information by applying machine readable manipulation to a temporally or spatially determined location in digital media. The inserted additional information can be detected by analyzing watermarked digital media with a device or software configured to interpret the manipulation.

The additional information may be information indicating ownership of digital media, or may be information indicating that copying or viewing is restricted. Such additional information can be inserted into digital media in an invisible form, especially by performing manipulations that cannot be recognized by the viewer of the digital media but can be read by a machine.

Additionally, a forensic watermarking method that inserts traceability information as additional information into leaked images using digital watermarking has been recently used. Inserting and distributing recipient information in digital media can help identify individuals who use digital media in unauthorized ways or engage in illegal leakage of digital media.

In order to insert forensic watermarking, each time an original digital media is supplied to a specific recipient, a unique watermarked digital media with the recipient's information inserted is required. Because digital media is distributed to the public, it is not possible to have separate watermarked digital media for all recipients. Therefore, preferably, real-time forensic watermarking should be implemented by inserting a digital watermark in real time while the recipient is viewing digital media. However, because inserting a digital watermark requires significant computational resources, it can place a significant burden on the provider or recipient of digital media.

Accordingly, conventionally, for real-time forensic watermarking of digital media, copies of digital media watermarked in advance into several types are used, and transmission units (e.g., GOP (Group of Frame), packet, or A real-time insertion method of traceability information was used by combining different copies by chunk. The above-described method allows only digital media providers to easily implement real-time forensic watermarking by consuming minimal additional computational resources.

Each transmission unit illustrated above is composed of a fairly large number of frames and has a predetermined length in terms of time (quantity). Therefore, when using the conventional real-time forensic watermarking method described above, there is a problem that a considerably long amount of digital media must be obtained in order to detect information inserted through a watermark from the combination method. Additionally, there is a problem that it is vulnerable to a collusion attack in which copies of different digital media are merged based on the transmission unit.

A watermark insertion device according to an aspect of the present invention for solving the above-described problem includes a frame data generator for generating frame data, an insertion symbol generator for generating an insertion symbol, and watermark information based on the insertion symbol. a watermark information generating unit that generates watermark information, and a watermark inserting unit that inserts a watermark into the frame data based on the watermark information to generate watermarked frame data, wherein the inserted symbol is at least a first symbol. and a second symbol, and the watermark information may be generated differently depending on the symbol of the inserted symbol.

The watermark insertion device further includes a media acquisition unit for reading digital media including at least one frame data, and an insertion sequence acquisition unit for reading an insertion sequence including at least one insertion symbol. , the frame data generator is configured to sequentially read one frame data from the digital media to generate frame data, and the insertion symbol generator is configured to sequentially read one insertion symbol from the insertion sequence to generate an insertion symbol. It can be configured.

The insertion symbol sequence includes information related to the content represented by the frame data, identification information related to the viewing device on which the watermarking frame data is displayed or received, identification information related to the user of the viewing device, and the viewing device. It may include at least one piece of information related to connection for communication with the device.

The watermark information may include at least one of information related to the insertion power of the watermark or information related to the insertion method of the watermark.

The frame data generator further includes a frame separator for dividing the frame data into two or more regions, and the watermark information generator is configured to generate region watermark information for at least one of the two or more regions. It may further include a generator, and the watermark information may be generated based on the at least one area watermark information.

The area watermark information may include at least one of information related to the insertion power of the watermark or information related to the insertion method of the watermark.

The insertion symbol generator is configured to generate a plurality of insertion symbols including at least a first symbol and a second symbol, and the area watermark information generator is configured to insert the first symbol into a first area among the two or more areas. Then, the area watermark information can be generated by using the second symbol as an inserted symbol in the second area.

The watermark insertion device further includes a memory for storing at least one frame data or at least one watermark frame data as previous frame data, and the watermark information generating unit stores the frame data and at least one previous frame. It further includes a suitability evaluation unit that analyzes data and evaluates the suitability of watermark insertion for the frame data, wherein the watermark information includes information for not inserting a watermark when the suitability falls below a threshold. It can be.

The suitability evaluation unit determines the continuity of the frame data, the brightness change, the saturation change of the frame data, the motion vector size of the frame data, the scene transition included in the frame data, and the frame included in the frame data as a key frame ( It may be configured to determine the suitability based on at least one type of a reference frame (e.g., an I frame) or a reference frame (e.g., a P or B frame).

A watermark detection device according to an aspect of the present invention for solving the above-described problem includes a frame data generator for generating watermarked frame data, detecting a watermark from the watermarked frame data, and detecting the detected watermark. A watermark detection unit for generating watermark information based on a mark, and a symbol detection unit for generating a detection symbol based on the watermark information, wherein the detection symbol is based on the detected watermark information, at least It may belong to a set containing the first symbol and the second symbol.

The watermark information may include at least one of information related to the power of the detected watermark or information related to the method of the detected watermark.

The frame data generator further includes a frame separator that divides the watermarked frame data into two or more regions, and the watermark detector detects a region watermark for at least one region, and divides the watermark into two or more regions. It may further include a region watermark detection unit that generates region watermark information based on the watermark, and the watermark information may be generated based on the at least one region watermark information.

The area watermark information may include at least one of information related to the power of the detected area watermark or information related to the method of the detected area watermark.

The watermark detection device may further include a sequence restoration unit that generates a detection sequence based on at least one detection symbol.

The detection sequence includes information related to the content represented by the frame data, identification information related to the viewing device on which the watermarking frame data is displayed or received, identification information related to a user of the viewing device, and the viewing device. It may include at least one piece of information related to a connection for communication.

A watermark insertion method according to an aspect of the present invention for solving the above-described problem includes generating frame data, generating an insertion symbol, generating watermark information based on the insertion symbol, and generating watermarked frame data by inserting a watermark into the frame data based on watermark information, wherein the inserted symbol belongs to a set including at least a first symbol and a second symbol, Watermark information may be generated differently depending on the symbol of the inserted symbol.

The watermark information may include at least one of information related to the insertion power of the watermark or information related to the insertion method of the watermark.

A watermark detection method according to an aspect of the present invention for solving the above-described problem includes generating watermarked frame data, detecting a watermark from the watermark frame data, and detecting a watermark based on the detected watermark. Generating watermark information, and generating a detection symbol based on the watermark information, wherein the detection symbol includes at least a first symbol and a second symbol based on the detected watermark information. It may be a method belonging to the containing set.

The watermark information may include at least one of information related to the power of the detected watermark or information related to the method of the detected watermark.

A watermark insertion system according to an aspect of the present invention for solving the above-described problem includes a media acquisition module for reading digital media containing at least one frame data, and an insertion sequence containing at least one insertion symbol. an insertion sequence acquisition module, sequentially reading one frame data from the digital image, sequentially reading one insertion symbol from the insertion sequence, generating watermark information based on the insertion symbol, and a watermark insertion module that generates watermark frame data by inserting a watermark into the frame data based on mark information, and a media reconstruction module that sequentially combines the watermark frame data to output watermarked digital media. It may be a system in which the inserted symbol belongs to a set including at least a first symbol and a second symbol, and the watermark information is generated differently depending on the inserted symbol.

According to the present invention, watermarking can be performed on a frame-by-frame basis, making it possible to extract watermarks from digital media of shorter sections. Additionally, since it can be easily combined with other watermarking methods, complex watermarking This has the advantage of forming part of a robust watermarking system.

1 is a basic conceptual diagram of a digital media distribution system constructed according to the present invention;
Figure 2 is a basic conceptual diagram of a method of generating a watermark information sequence based on the frame sequence of digital media and the sequence of information to be inserted in the present invention;
3 is a basic conceptual diagram of a method of watermarking digital media using a watermark information sequence in the present invention;
Figure 4 is a basic conceptual diagram of the process of detecting information inserted from watermarked digital media in the present invention;
Figure 5 is a block diagram of an embodiment of a watermark insertion device according to the present invention;
6 is a block diagram of an embodiment of a watermark detection device according to the present invention;
7 is a conceptual diagram of frame data in which region division is performed;
Figure 8 is a block diagram of an embodiment further including a region division function of frame data in the watermark insertion device according to the present invention;
Figure 9 is an example of a method for inserting a watermark based on region division in a modified embodiment of the watermark insertion device according to the present invention;
Figure 10 is a block diagram of an embodiment of utilizing pre-processed digital media in the watermark insertion device according to the present invention, and
Figure 11 is an exemplary diagram of a method of inserting a watermark using a frame selection unit in a modified embodiment of the watermark insertion device according to the present invention.

Since the present invention can make various changes and have various embodiments, specific embodiments will be illustrated in the drawings and described in detail. However, this is not intended to limit the present invention to specific embodiments, and should be understood to include all changes, equivalents, and substitutes included in the spirit and technical scope of the present invention.

Terms such as first, second, etc. may be used to describe various components, but the components should not be limited by the terms. The above terms are used only for the purpose of distinguishing one component from another. For example, a first component may be referred to as a second component, and similarly, the second component may be referred to as a first component without departing from the scope of the present invention. The term “and/or” includes any combination of a plurality of related stated items or any of a plurality of related stated items and, unless otherwise indicated, is non-exclusive. When items are listed in the present application, it is merely an exemplary description to easily explain the spirit and possible implementation methods of the invention of the present application, and therefore, it is not intended to limit the scope of the embodiments of the present invention.

When a component is said to be "connected" or "connected" to another component, it is understood that it may be directly connected to or connected to the other component, but that other components may exist in between. It should be. On the other hand, when it is mentioned that a component is “directly connected” or “directly connected” to another component, it should be understood that there are no other components in between.

The terms used in this application are only used to describe specific embodiments and are not intended to limit the invention. Singular expressions include plural expressions unless the context clearly dictates otherwise. In this application, terms such as “comprise” or “have” are intended to designate the presence of features, numbers, steps, operations, components, parts, or combinations thereof described in the specification, but are not intended to indicate the presence of one or more other features. It should be understood that this does not exclude in advance the possibility of the existence or addition of elements, numbers, steps, operations, components, parts, or combinations thereof.

Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by a person of ordinary skill in the technical field to which the present invention pertains. Terms defined in commonly used dictionaries should be interpreted as having a meaning consistent with the meaning in the context of the related technology, and unless clearly defined in the present application, should not be interpreted in an idealized or excessively formal sense. No.

In describing the invention in this application, embodiments may be described or illustrated in terms of unit blocks that perform the described function or functions. In the present application, the blocks may be expressed as one or more devices, units, modules, units, etc. The blocks may be implemented in hardware using one or more logic gates, integrated circuits, processors, controllers, memories, electronic components, or information processing hardware implementation methods that are not limited thereto. Alternatively, the blocks may be implemented in software by an implementation method of application software, operating system software, firmware, or information processing software that is not limited thereto. One block may be implemented separately into a plurality of blocks that perform the same function, or conversely, one block may be implemented to simultaneously perform the functions of a plurality of blocks. The blocks may also be implemented physically separated or combined by any criteria. The blocks may be implemented to operate in an environment where their physical locations are not specified and are spaced apart from each other by a communication network, the Internet, a cloud service, or a communication method that is not limited thereto. All of the above implementation methods are within the scope of various embodiments that can be taken by a person skilled in the art of information and communication technology to implement the same technical idea, so any detailed implementation methods are all included in the technical idea scope of the invention in this application. It should be interpreted as

Hereinafter, preferred embodiments of the present invention will be described in more detail with reference to the attached drawings. In order to facilitate overall understanding when describing the present invention, the same reference numerals are used for the same components in the drawings, and duplicate descriptions for the same components are omitted. Additionally, it is assumed that the plurality of embodiments are not mutually exclusive, and that some embodiments may be combined with one or more other embodiments to form new embodiments.

Basic concept of the present invention

1 is a basic conceptual diagram of a digital media distribution system constructed according to the present invention.

The digital media source 110 is an initial source that supplies digital media 115, and depending on the embodiment, it is a device or system that reads and provides digital media from a recording medium on which digital media is recorded, or non-digital media. It may be a device or system that reads a recorded recording medium and provides it in the form of digital media, or it may be a device or system that provides information acquired in real time in the form of digital media. In another embodiment of the present invention, it may be a device or system that provides digital media to which at least one information protection method such as watermarking or encryption has already been applied.

In the present invention, the digital media 115 is considered information that requires protection. In one embodiment of the present invention, the digital media 115 may be information that may be exposed to illegal use or distribution, such as unauthorized copying, when distributed to the public. The present invention inserts (i.e., hides) predetermined information into the digital media 115 in the form of a watermark by allowing the digital media 115 to pass through the watermark insertion module 120 before being distributed, This can be used to create the first watermarked digital media 125 and distribute it to the first viewing device 130.

The predetermined information may include information related to the title, length, type, language, subtitles, copyright holder, distribution right holder, supply channel, and viewing age limit of the content included in the digital media 115. Alternatively, the predetermined information may include information for maintaining the security of the digital media 115. Specifically, it may include copy permission information, copy prevention information, and traceability information. Alternatively, the predetermined information may include information for controlling a device for receiving and viewing the first watermarked digital media 125, such as the first viewing device 130. Specifically, it may include metadata for additional services or terminal control commands that the first viewing device 130 can recognize as a signal.

The predetermined information is information inserted by real-time watermark insertion by the watermark insertion module 120, and is the first water distributed to the first viewing device 130 as a copy of the digital media 115. It may be traceability information that allows the marked digital media 125 to be distinguished and identified from other copies, such as the second watermarked digital media 125a distributed to the second viewing device 130a.

The traceability information may include identification information related to the first viewing device 130. Specific examples include the type of device hardware of the first viewing device 130, an identifier of the device hardware (e.g., device UUID), an identifier of device software, a version of device software, or other information about the first viewing device 130. ) may include information necessary for identification.

Additionally, the traceability information may include at least one of information related to a means for distributing the watermarked digital media 125 from the watermark insertion module 120 to the first viewing device 130. . The distribution means may use a conventional wired or wireless digital communication network, or may use hard-wired wiring between the watermark insertion module 120 and the first viewing device 130, or Physical recording media including hard disks, optical media, and flash memory can be used. Accordingly, the traceability information may include a communication network address, a network interface identifier, or network path information of the first viewing device 130 when the distribution means is a communication network, and the distribution means is a fixed wiring. In this case, it may include identification information of the wiring path, and if the distribution means is a physical recording medium, it may include a unique identifier for identifying a copy of a specific physical recording medium.

Additionally, the first device information 135 may include identification information related to the user 150 of the viewing device. Specific examples include receiving user identification information (e.g., service ID), personal identification information (e.g., name, identification information, etc.) of the user 150, and the watermarked digital media 125. It may include details of the usage fee paid by the user 150, financial transaction information performed by the user 150 to pay the usage fee, and other information necessary for identification of the user 150.

In order to generate the traceability information, the watermark insertion module 120 or a distribution system including the same provides identification-related information in the form of first device information 135 from the first viewing device 130. It can be configured to receive. For example, the first viewing device 130 uses its own communication network address and the unique identifier of the user 150 as the first device information 135 to use the watermark insertion module 120 or the distribution side including the same. The watermark insertion module 120 provides the first watermarked digital media 125 into which the traceability information including the first device information 135 is inserted to the viewing device 130. Can be configured to distribute. Thereafter, the first watermarked digital media may be provided (137) in a form that can be viewed by the user (150).

The operation of the watermark insertion module 120 described above can be commonly applied to each viewing device that wishes to view a copy of the digital media 115. For example, each of the traceability information for the first viewing device 130 and the traceability information for other viewing devices 130a, such as the second viewing device, may be uniquely distinguished information, and therefore the first The first watermarked digital media 125 distributed to the viewing device 130 and the second watermarked digital media 125a distributed to the second viewing device 130a are both watermarked using the same method. It may contain different traceability information embedded. Likewise, the second viewing device, etc. 130a may be configured to provide the second device information, etc. 135a to the watermark insertion module 120 or a distribution system including the same.

Meanwhile, digital media distributed by the watermark insertion module 120 may be subject to information detection by the watermark detection module 140, if necessary. The purpose or reason for the detection may be any, but for example, if the first watermarked digital media 125 into which the traceability information is inserted is unauthorizedly copied and distributed by the illegal act of the user 150. , it can be performed for the purpose of determining whether the user 150 is responsible by making the duplicate copy the object of detection.

The watermark detection module 140 detects the first watermarked digital media 115 distributed from the watermark insertion module 120 to the first viewing device 130, or the first viewing device 130 ) may target the detection results provided (137) in a viewable form to the user.

For example, the watermark detection module 140 may operate by obtaining the first watermarked digital media 125 in the form of a media file. Alternatively, the first watermarked digital media 125 may be obtained and operated on a communication network. Alternatively, for example, if the first watermarked digital media 125 includes digital image content, it may be operated by recording the result displayed 137 in the form of an image to the user 150. In addition, the digital media 125 including the first watermarked digital media 115 can be obtained directly or indirectly, and information satisfying the quality and quantity for detecting the watermark can be collected. Any method is fine.

The watermark detection module 140 detects 145 information inserted (i.e., hidden) in the first watermarked digital media 125 by the watermark insertion module 120 and provides restoration information 155. can be created. The restoration information 155 can be viewed as the same as the predetermined information that the watermark insertion module 120 can insert. If the restoration information 155 is traceability information, it can help identify a specific user 150 from the restoration information 155.

Basic concept of watermark insertion and detection according to the present invention

Next, the basic concept of the watermark insertion procedure based on the present invention will be described.

Figure 2 is a basic conceptual diagram of a method of generating a watermark information sequence based on the frame sequence of digital media and the sequence of information to be inserted in the present invention.

The digital media 210 to which the watermark is inserted may be digital media constructed based on frames 215, for example, digital video or audio encoded in a frame-based coding format. there is.

In the description related to FIG. 2, the frame is used as a common name for a standard unit designated to encode the content of a specific digital media, such as a frame or picture in any encoding or storage standard for the digital media. The frame may be, for example, a frame or picture representing an individual image in digital video media, a frame representing a unit encoding section in digital audio media, a paragraph representing a unit storage section in digital document media, or a frame representing a unit storage section in digital three-dimensional polygon media. It may correspond to individual objects or polygons constituting , and may correspond to any other encoding or storage unit.

Additionally, the frame may be limited to some components rather than all information components of a specific digital media. For example, in the case of digital video media using the YCbCr color expression method, the frame may be limited to only the brightness (Y) channel among arbitrary frames selected from the video media, and any other components may be selected and combined. There is no problem in implementing the technical idea of the present invention. In addition, the frame does not necessarily need to be interpreted as representing one frame, and may target a section consisting of two or more frames, or may be an information unit such as a slice, macroblock, or block that further divides one frame. do.

However, in order to facilitate the conceptual explanation of the present invention without denying the possibility of various implementations as described above, in FIG. 2, the digital media 210 is regarded as a digital video media according to an encoding standard such as MPEG, and the frame 215 is also The description will be made considering it as a frame according to the above encoding standard.

The insertion information 220, which is the content of the inserted watermark, is independent of any content, and it is sufficient that it corresponds to additional information to achieve a certain technical purpose through insertion. As an example of the additional information, the example related to the predetermined information inserted by the watermark insertion module 120 in relation to FIG. 1 may be applied.

The insertion information 220 may be an insertion sequence composed of a plurality of insertion symbols 225. The insertion symbol may be selected from a set of at least two types of distinct identification symbols. Additionally, the insertion symbol may be a number with base 2 or more. The insertion sequence 220 may be the result of encoding insertion information 227 in a form that can be interpreted by humans using the symbol construction method of the insertion symbol. For example, if the inserted symbol is a symbol that can have a value of 0 or 1 as binary digital information, the English letter "A" (227) corresponding to an example of the inserted information is encoded in ASCII. It is converted to the decimal number 65 and then converted back to the binary number 1000001, so that it can be composed of an insertion sequence represented by at least 7 insertion symbols. It can be easily seen that the information encoding method and the type and base of the inserted symbols can utilize all various technologies that can be applied by those skilled in the art, and do not constitute elements that limit the technical idea of the present invention.

However, in order to facilitate the conceptual explanation of the present invention without denying the possibility of various implementations as described above, in FIG. 2, the content of the insertion information 220 is an insertion sequence in which traceability information is digitally encoded, and the insertion symbol 225 is It is considered and explained as a single digit in digital binary notation with a symbol of 0 or 1.

As shown in FIG. 2, when the length of the insertion information 220 is N and consists of a total of N insertion symbols 225, one frame 215 corresponds to one insertion symbol 225 (230) It can be. That is, insertion information 220 of length N can be inserted into a frame sequence 217 of length N consisting of N frames. Figure 2 shows an example of inserting digital information 01001000 consisting of binary numbers by encoding the English letter "A" in an 8-bit space using ASCII code. In this example, the length N is 8.

Next, the process of inserting the insertion information 220 into each frame of the frame sequence 217 will be described.

For each symbol included in the insertion information 220, watermarking information with different properties may be generated depending on the type of symbol. Properties of the watermark information may include whether the watermark is inserted, the insertion power, or the type of watermarking algorithm used for insertion. Additionally, the watermark information may be in the form of a pattern, overlay image, attribute value, or command indicating a specific watermarking method using any one of the above attributes. Details will be described later in the description regarding the configuration of the watermark insertion device.

By combining the watermarking information generated for all symbols of the insertion information 220, an attribute for inserting the information of the insertion information 220 into the frame sequence 217 of the digital media 210 through watermarking A watermarking information sequence 260, which is a set of information, can be created.

The above-described procedure will be explained with respect to some frames shown in FIG. 2 as examples. Since the first symbol 251 corresponds to the first frame 241, and the value of the first symbol is “0”, first watermark information 261 with attributes corresponding to the symbol “0” will be generated. Since the second symbol 252 corresponds to the second frame 242 and the value of the second symbol is “1”, the second watermark information 262 has an attribute corresponding to the symbol “1”. can be created.

By repeating the same procedure as above N times (eight times in the example of FIG. 2), which is the length of the insertion information 220, watermark information consisting of a total of N pieces of watermark information to be applied to each frame belonging to the frame sequence 217 Sequence 260 may be created.

Figure 3 is a basic conceptual diagram of a method of watermarking digital media using a watermark information sequence in the present invention. The watermark information sequence 260 generated by the above-described method can be used in the process of inserting a watermark corresponding to each frame included in the original digital media 210.

Referring to what is shown in FIG. 3 by applying the example shown in FIG. 2, the first watermark information 260 included in the watermark information sequence 260 is included in the original digital media 210 as described above. It may indicate information related to the properties of a watermark to be inserted into the first frame 241, which is one of the frames 215.

When information on the first frame 241 is obtained from the original digital media 210 and digital watermarking 350 is performed on the first frame 241 using the first watermark information 261, A first watermarked frame 341 in which information on the first symbol 251 is inserted into the first frame 241 can be generated. In the same procedure, digital watermarking is repeated on the frames 215 according to the watermark information sequence 260 to obtain watermarked frames 315, and each watermarked frame 315 is connected according to the original order. Then, the watermarked digital media 310 can be finally obtained.

In conclusion, by the procedure described with reference to FIGS. 2 and 3, insertion information of length N can be inserted over N consecutive frames on digital media. This procedure can be done continuously by repeating every N frame, or it can be implemented so that a padding section unrelated to watermarking is formed for an arbitrary length of the number of repetitions every N frame and then inserted again, and digital It can also be implemented to cut N frames from a specific part of the media and insert them into a sequence of frames where at least one position is determined. Additionally, the length N of the inserted information is not fixed, and the padding section in which the last symbol or watermark of the inserted information is not inserted may be implemented to indicate the end of the inserted information. More detailed examples of transformation will be described again along with examples.

Next, the watermark detection procedure based on the present invention will be described.

Figure 4 is a basic conceptual diagram of the process of detecting information inserted from watermarked digital media in the present invention.

Watermarked digital media 310 may be subject to detection attempts to detect embedded information. For detection, the watermarked frames 315 belonging to the watermarked digital media 310 can be read sequentially and detection can be attempted on a frame-by-frame basis.

When the watermarked frame 341 obtained for analysis is input into the digital watermarking detection 450 algorithm, the type of watermark information 461 inserted into the watermarked frame 341 can be identified. Additionally, based on the properties of the watermark information 461, the detection symbol 410 indicated by the watermark information 461 can be restored. The detection sequence 420 can be obtained by sequentially acquiring a predetermined number of detection symbols, for example, N sequences of the same length as those mentioned in FIGS. 2 and 2 (425), and then combining them.

The detection sequence can be restored to detection information 427 by reversely applying the same method as when inserted. For example, if the detection symbol is a binary digital information symbol that can have a value of 0 or 1, the detection sequence “0100001” (420) can be obtained as a result of combining 8 detection symbols (425). When interpreted through ASCII encoding, it corresponds to the decimal number 65 and can be configured to be restored as the English letter "A".

Watermark insertion device and method of operation thereof

Figure 5 is a block diagram of an embodiment of a watermark insertion device according to the present invention.

The watermark insertion device 520 according to an embodiment of the present invention is combined with a media acquisition unit 510 that provides input, an insertion information acquisition unit 520, and a media recombination unit 590 that generates an output to create a watermark. The insertion module 500 can be configured. In one embodiment, the configuration of the module represents a structure separated according to the division of functions, and it is obvious that it can be implemented in various modifications, such as various implementation methods of the unit block as described above.

The media acquisition unit 510 may be configured to read digital media 501 as input to the watermark insertion module 500. The digital media 501 is the same as the digital media 115 and 210 in the description with reference to FIGS. 1 and 2, and the same example may also be applied to its embodiment.

The media acquisition unit 510 may be configured to receive digital media 501 via a digital communication network. Alternatively, the media acquisition unit 510 may be configured to read digital media 501 stored in a fixed or variable digital recording medium. Alternatively, the media acquisition unit 510 may be configured to process digital media 501 generated in real time from input devices such as cameras and recorders. The media acquisition unit 510 may also be modified and implemented in various embodiments to implement information input of the digital media 501.

The media acquisition unit 510 generates frame data 515 by sequentially separating frames, which are standard units constituting the information of the digital media 501, preferably from the starting point of the digital media 501 information. , and may be configured to provide this to the watermark insertion device 530.

As described above with respect to digital media 210 in the description of FIG. 2, the separated frame data 515 may be, for example, a frame or picture representing an individual image of digital video media, or a unit encoding of digital audio media. It may be a frame representing a section, a paragraph representing a unit storage section of digital document media, or an individual object or polygon constituting digital three-dimensional polygon media, and may be information of any other encoding or storage unit.

In the present invention, a representative embodiment assumes that the frame data 515 is a continuous frame sequence for a certain length, and sequential watermarks are inserted into each frame data 515 belonging to the frame sequence.

However, the media acquisition unit 510 may be configured to generate one frame data 515 by sequentially and continuously separating the digital media 501. However, the media acquisition unit 510 may be configured to separate the digital media 501 in a predetermined order including reverse order, or It may be configured to take and separate only one frame data 515 at intervals. Alternatively, it may be configured to separate two or more frame data 515 by one separation. The method for the media acquisition unit 510 to generate frame data 515 from the digital media 501 may be any method other than the above example, as long as it follows predetermined rules so that it can be recognized by a watermark detection device later. do.

The insertion information acquisition unit 520 may be configured to read insertion information 502 as input given to the watermark insertion module 500. The insertion information 502 includes predetermined information inserted into the digital media 115 in the description with reference to FIGS. 1 and 2, and the same example may also be applied to the embodiment. In particular, the insertion information 502 may be traceability information that can uniquely identify at least one of a viewing device or a recipient of digital media.

The insertion information 502 may be the predetermined information expressed in a form that can be interpreted by humans, but depending on the embodiment, the insertion information is encoded by expressing the predetermined information using at least one symbol. It may also be provided as a sequence. In encoding using the symbol, the symbol may be configured to be selected from a set of two or more distinct identification symbols. Additionally, the symbol may be a number with base 2 or more.

For example, if the insertion information 502 is encoded as binary digital information, the set of identification symbols may consist of {0, 1}. In this case, the English letter "A", which corresponds to an example of the insertion information, is converted to the decimal number 65 by ASCII encoding and then converted back to the binary number 1000001, so that it belongs to at least 7 {0, 1} It may consist of an insertion sequence represented by symbols (in this example, bits). It can be easily seen that the encoding method of the insertion sequence and the type and base of the symbols can utilize all various techniques that can be applied by those skilled in the art, and do not constitute elements that limit the technical idea of the present invention.

The insertion information acquisition unit 520 generates an insertion symbol 525 by separating symbols that are standard units of information expression from the insertion information 502 or the insertion sequence constituting the same, preferably sequentially from the starting point. , and may be configured to provide this to the watermark insertion device 530.

In the present invention, a representative embodiment assumes that the insertion symbol 525 is a continuous sequence of insertion information for a certain length, and sequential watermarks are inserted based on each insertion symbol 525 belonging to the insertion sequence. do.

However, the insertion information acquisition unit 520 may be configured to generate one insertion symbol 525 by sequentially and continuously separating the insertion information 502. However, the insertion information 502 may be separated in a predetermined order including reverse order, or It may be configured to take and separate only one insertion symbol 525 at regular intervals. Alternatively, it may be configured to separate two or more insertion symbols 525 by one separation. The method of the insertion information acquisition unit 520 generating the insertion symbol 525 from the insertion information 502 is different from the above example as long as it follows predetermined rules so that the information can be restored in the watermark detection device later. Any method is fine.

The watermark insertion device 530 may be a device that inserts a watermark into the frame data 515 using an insertion symbol 525, and includes a frame data generator 540, an insertion symbol generator 550, and a watermark. It may be composed of an information generation unit 560, a watermark insertion unit 570, and a memory 580.

The frame data generator 540 is configured to read the frame data 515 provided from the media acquisition unit 510 for the watermark insertion device 530 and provide it to the watermark insertion unit 570 (545). You can. Depending on the embodiment, the frame data generator 540 may be configured to provide input frame data 515 without processing (545), or may be directly connected to the media acquisition unit 510. By being composed of the same module, it may be configured to generate and provide frame data directly from digital media 501 (545). Alternatively, in another embodiment, the frame data generator 540 performs at least one procedure of modifying, correcting, processing, scaling, formatting, re-encoding, or inserting another watermark into the input frame data 515. It may be configured to apply and provide (545). If it is configured to generate frame data that is provided as input to the watermark insertion device 530 and is the subject of watermarking and provide it to the watermark insertion unit 570 (545), the frame data generation unit 540 includes Any additional functions may be added.

The insertion symbol generator 550 may be configured to read the insertion symbol 525 provided from the insertion information acquisition unit 520 for the watermark insertion device 530 and provide it to the watermark information generator (555). there is. Depending on the embodiment, the insertion symbol creation unit 550 may be configured to provide (555555) the input insertion symbol 525 without processing, or may be directly connected to the insertion information acquisition unit 520. Alternatively, it may be configured to generate and provide an insertion symbol directly from the insertion information 502 (555) by being composed of the same module. Alternatively, in another embodiment, the insertion symbol generator 550 applies at least one procedure of correction, correction, calculation, conversion, processing, or re-encoding to the input insertion symbol 525 and provides it (555). It may be configured to do so. If configured to generate a symbol provided as an input to the watermark insertion device 530 and inserted (i.e., hidden) through watermarking and provide the symbol to the watermark information generator 560 (555), the insertion symbol generator 550 ), any additional functions may be added.

The watermark information generation unit 560 may generate and provide watermark information 565 to indicate the operation method of the watermark insertion unit 570. The watermark information 565 may be generated differently depending on the value or type indicated by the insertion symbol 555 input to the watermark information generator 560. That is, the watermark information generation unit 560 may be configured to indicate a distinct watermark so that the watermark insertion unit 570 represents the insertion symbol 555.

The watermark information 565 is information corresponding to whether the watermark is inserted, insertion content, insertion method, insertion power, insertion transparency, insertion position, or other watermark insertion parameters. It may include at least one of:

The configuration of the watermark information 565 may vary depending on the type of watermark algorithm implemented in the watermark insertion unit 570. For example, in the case of a watermark algorithm that can insert distinct watermarks by adjusting the insertion strength, the embedding strength may be included in the watermark information 555. Alternatively, in the case of a watermark algorithm that can insert distinct watermarks by adjusting the insertion position, the insertion position may be included in the watermark information 555. Alternatively, if a plurality of watermark algorithm methods are implemented in the watermark insertion unit 570, information indicating one of the methods may be included in the watermark information 555.

Additionally, the watermark information 565 may include information that simply executes or does not execute the watermark, depending on the embodiment. For example, if the insertion symbol 555 is a binary number, it may include information instructing to perform watermarking if its value is 1, and not to perform watermarking if its value is 0.

In the process of generating the watermark information 565, frame data generated by the frame data generator 540 may be referenced (547). The referenced frame data 547 is also stored in the memory 580, which records and stores previously processed frame data 545 or watermarked frame data 575, and is stored in one or more memory 580 from the memory 580. These can be referenced simultaneously.

The watermark information generator 560 stores currently processed frame data 545, currently processed watermarked frame data 575, or memory 580 through reference to the frame data 547. It may be configured to analyze the content of the frame, including at least one of previously processed frame data or watermarking data.

Meanwhile, when inserting a watermark into digital media, especially when it is desired to maintain the invisibility of the watermark, the insertion efficiency of the watermark is lowered or non-visible in sections where certain changes occur in the digital media original content 501. There may be potential threats, such as visibility being violated and the watermark being exposed. Therefore, if it is analyzed that watermark insertion is not suitable for the current frame data 545 for reasons other than those mentioned above, the configuration of the watermark information to be provided to the watermark insertion unit 570 may be changed or , Alternatively, the watermark information can be configured to not insert the watermark.

Accordingly, the watermark information generation unit 560 may be configured to include or be linked to a suitability evaluation unit (not shown). The suitability evaluation unit may be configured to evaluate the reliability level of insertion of the watermark by the insertion symbol 555 into the currently processed frame data 545 by analyzing the content of the frame (e.g., detection of the specific change). You can. The suitability evaluation unit may evaluate the trust level as a quantitative value. Additionally, if the quantitative value of the trust level is less than a predetermined threshold, it may be configured to make a yes/no evaluation that watermark insertion is inappropriate.

In the present invention, the type of digital media is not limited, and the specific change represents a change that interferes with the useful operation of the watermark insertion unit 570 among the changes in frame data that are different depending on the content type of the digital media. It should be understood that this is something that ordinary technicians with watermark technology for each type of digital media are familiar with.

In one embodiment of the present invention, when the digital media corresponds to video content, the specific changes include continuity, brightness, saturation, color, motion vector size, scene transition, and key frames (such as I frames) of the frame data. ) or a type of reference frame (e.g., P or B frame).

In one embodiment of the present invention, when the digital media corresponds to audio content, the specific change is the continuity of the frame data, amplitude change, frequency change, and (e.g., divided into human vocalization, music, noise, etc.) ) may include a change in at least one of the phonetic forms.

Techniques for detecting each change in the above examples may follow those conventionally known to ordinary digital media engineers. For example, in order to detect a scene change in video content in the above embodiment, the method for detecting a scene change frame disclosed in Patent Publication No. 2017-0090868 may be used.

The watermark insertion unit 570 receives the frame data 545 and the watermark information 555 as input, and inserts a watermark into the frame data 545 based on the watermark information 555. , may be configured to generate watermarked frame data 575.

The watermark algorithm method used by the watermark insertion unit 570 may be any method known to provide the same function. In addition, the watermark algorithm method may be adopted in such a way that it is difficult for a person to identify whether a watermark is inserted from watermarked frame data, such as invisible or inaudible insertion, but depending on the embodiment, visible or audible insertion may be used. It is okay to use a watermark algorithm method that makes changes that can be identified by humans, such as insertion, and it is also okay to use a mixed method of invisible/inaudible methods and visible/audible methods.

The watermark algorithm used by the watermark insertion unit 570 may be implemented as either a real-time or non-real-time watermark insertion algorithm. When real-time is important, such as a digital media distribution environment through a communication network, an algorithm that guarantees real-time can be used. In an environment where completing the watermark before the watermark playback time is sufficient, an algorithm that does not guarantee real-time can be used. You can.

The watermark insertion unit 570 may be configured to use two or more watermark algorithm methods. The multiple watermark algorithms may be differently selected depending on the watermark information 565 and the insertion symbol 555, may be selectively applied according to other rules, or may be applied sequentially and repeatedly.

The watermark information 565 generated by the watermark information generation unit 560, or the watermark insertion algorithm of the watermark insertion unit 570, allows the watermarked digital media 595 to use a watermark key. It may be configured to include information encrypted by . The watermark key encrypts the information inserted through the watermark using a random key value, preventing the watermark from being read by an unauthorized third party, while preventing different watermarks from being inserted into the same digital media. It can help improve detection performance in cases.

The media reassembly unit 590 receives the watermarked frame data 575, which is the output of the watermark insertion unit 570, as an output of the watermark insertion device 530, and creates watermarked digital media 595. It can be configured to restore.

Previously, the media acquisition unit 510 separated the frame data 515 from the digital media 501 and provided it to the watermark insertion device 530, so it must be restored to the form of digital media in order to be distributed. The media reassembly unit 590 may be configured to reconstruct the watermarked frame data 575 in accordance with the separation method in the media acquisition unit 510. For example, when the media acquisition unit 510 separates the frame data 515 sequentially according to time, the media reassembly unit 590 is configured to combine the watermarked frame data 575 sequentially according to time. It can be.

The media manufacturing unit 590 may be configured to provide watermarked digital media 595 to another device, module, functional unit, or system. In one embodiment of the present invention, the media reassembly unit 590 may be connected to a transmission unit for a communication network and configured to transmit the watermarked digital media 595 to a specific viewing device. In another embodiment of the present invention, the media reassembly unit 590 is connected to a computing device including a recording function for an electronic recording medium, and is configured to record the watermarked digital media 595 on the recording medium. You can. In another embodiment of the present invention, the media reassembly unit 590 may be directly connected to a viewing device and configured to display watermarked digital media 595 to the user.

Watermark detection device and method of operation thereof

Figure 6 is a block diagram of an embodiment of a watermark detection device according to the present invention.

The watermark detection device 620 according to an embodiment of the present invention is combined with a media acquisition unit 610 that provides input and an information restoration unit 690 that generates output to form a watermark detection module 600. You can. In one embodiment, the configuration of the module represents a structure separated according to the division of functions, and it is obvious that it can be implemented in various modifications, such as various implementation methods of the unit block as described above.

The media acquisition unit 610 may be configured to read watermarked digital media 601 as input to the watermark detection module 600. The watermarked digital media 601 is the same as the watermarked digital media 125, 125a, 137, and 310 in the description with reference to FIGS. 1 and 2, and the same example can be applied to the embodiment as well. .

The media acquisition unit 610 may be configured to receive watermarked digital media 601 via a digital communication network. Alternatively, the media acquisition unit 610 may be configured to read watermarked digital media 601 stored in a fixed or variable digital recording medium. Alternatively, the media acquisition unit 610 may be configured to process watermarked digital media 601 collected via input devices such as cameras and recorders. The media acquisition unit 610 may also be modified and implemented in various embodiments to implement information input of the watermarked digital media 601.

The media acquisition unit 610 divides the frames, which are standard units constituting the information of the watermarked digital media 601, sequentially, preferably from the starting point of the watermarked digital media 601 information, and marks them as watermarks. It may be configured to generate frame data 615 and provide it to the watermark detection device 620.

As described above with respect to the watermarked digital media 215, 215a in the description of FIG. 2, and as described above with respect to the frame data 515 in the description of FIG. 5, the watermarked frame data 615 has any It may be information in an encoding or storage unit.

In the present invention, the frame data 615 is assumed to be a continuous watermarked frame sequence for a certain length, and sequential watermark detection is performed on each watermarked frame data 615 belonging to the watermarked frame sequence. A representative example is restoring information that was originally inserted into digital media through .

However, the media acquisition unit 610 may be configured to generate one watermarked frame data 615 by sequentially and continuously separating the digital media 601, but the watermark detection module 600 Any other method may be used as long as it follows predetermined rules for the operation of the media acquisition unit 520 or the media reassembly unit 590 in the corresponding watermark insertion module 500.

The watermark detection device 620 may be a device that detects and outputs a detection symbol 655 from watermarked frame data 615, and includes a frame data generation unit 630, a watermark detection unit 640, and a symbol detection unit ( 650), and a memory 680.

The frame data generator 630 reads the watermarked frame data 615 provided from the media acquisition unit 610 for the watermark detection device 620 and provides it to the watermark detection unit 640 (635). It can be configured. Depending on the embodiment, the frame data generator 520 may be configured to provide (635) the input watermarked frame data 615 without processing, or may be configured to provide (635) the input watermarked frame data 615 directly with the media acquisition unit (610). It may be configured to generate and provide 635 watermarked frame data directly from the watermarked digital media 601 by being connected or composed of the same module. Alternatively, in another embodiment, the frame data generator 630 may perform at least one of correction, correction, processing, scaling, format change, re-encoding, or detection of another watermark in the input watermarked frame data 615. It may be configured to apply one procedure and then provide it (635). If it is configured to generate watermarked frame data that is provided as input to the watermark detection device 630 and is the subject of detection and provide it to the watermark detection unit 640 (635), the frame data generation unit 630 includes Any additional functions may be added.

Additionally, according to an embodiment of the present invention, the watermarked frame data 635 may be provided to the watermark detection unit 640 and may be provided and stored in the memory 680. The memory 680 may be configured to store and provide (637) at least one previously watermarked frame data for the operation of the watermark detection unit (640).

The watermark detection unit 640 may be configured to receive the watermarked frame data 635 as input and generate watermark information 645 using a watermark detection algorithm.

The watermark detection algorithm used by the watermark detection unit 640 may be any method as long as it can read the watermark inserted by the watermark insertion module 500. Additionally, the watermark algorithm method used by the watermark detection unit 640 may be implemented as either a real-time watermark detection algorithm or a non-real-time watermark detection algorithm. According to one embodiment of the present invention, the watermark detection module 600 is configured to operate on watermarked digital media 601 that has already been collected and stored, and although real-time is not guaranteed, detection accuracy is improved. It could be an algorithm.

The watermark detection unit 640 may be configured to use two or more watermark detection algorithm methods. The multiple watermark detection algorithm may also be used under any conditions, method, or order as long as it can read the watermark by the multiple watermark algorithm used in the watermark insertion module 500.

The watermark information 645 generated by the watermark detection unit 640 includes whether or not the watermark is inserted, insertion content, insertion method, insertion power, insertion transparency, insertion location, or other watermark information. It may include at least one of information corresponding to the insertion parameter. In one embodiment of the present invention, the watermark information 645 may be composed of the same information as the watermark information 565 generated in the watermark insertion module 500, or may be inserted for ease of detection. It may be composed of information different from that of the module 500. Additionally, the watermark detection unit 640 may be configured to receive a watermark key and detect a watermark encrypted by the watermark key upon insertion.

In the process of generating the watermark information 645, watermarked frame data generated by the frame data generator 620 and stored in the memory 680 may be referenced (637). The watermark detection unit 640 can be used to supplement or strengthen watermark detection performance through reference 637 of the watermarked frame data.

Exemplarily explained through an embodiment of the present invention, when the watermark insertion module 500 inserts an invisible watermark by adjusting the image brightness for each frame of digital media consisting of video content, In order to separate the artificial change in image brightness created by the watermark from the natural change in image brightness that originally existed in the original digital media 501, at least two or more watermarked frame data 635 are stored in the memory 580. It can be configured to detect changes created by the watermark based on the trend curve of image brightness obtained from each frame data by storing and referencing 637 and analyzing it statistically. In addition, it is obvious that analysis and detection methods over multiple frames used in various digital watermark insertion and detection algorithms known to those skilled in the art can be used.

Additionally, the watermark detection unit 640 may include or be configured to be linked to a suitability evaluation unit (not shown). The suitability evaluation unit determines the currently processed watermarked frame data 635 by analyzing the content of the watermarked frame data (e.g., detecting specific changes as exemplified in the description of the watermark insertion module 500). It may be configured to evaluate the level of confidence that a watermark has been inserted. The suitability evaluation unit may evaluate the trust level as a quantitative value. Additionally, if the quantitative value of the trust level is less than a predetermined threshold, it is considered that the watermark has not been inserted, and the watermark detection unit 640 may be configured not to generate the watermark information 645.

In the present invention, the type of digital media is not limited, and the watermark detection algorithm method, detection performance enhancement method, and specific change are effective watermark information from the change pattern of different frame data depending on the content type of the digital media. It should be understood as representing all technical configurations configured to detect (640), which are well known to those skilled in the art in watermarking technology for each type of digital media.

The symbol detection unit 650 receives the watermark information 645, determines which detection symbol the watermark information 645 should be interpreted as, and outputs the detection symbol 645 to the watermark detection device 620. It can be configured to output as .

The detection symbol 645 may be any information unit that can reconstruct the insertion information 502 given to the watermark insertion module 500, but is preferably the same symbol as the insertion symbol 555. , It may be a symbol selected from a set of at least the same size. To explain the present invention by way of example, if the insertion symbol 555 is a symbol selected from the set of binary digital information {0, 1}, the detection symbol 655 is also {0, 1}. It can be selected from the set of 1}, or it can be selected from the set of {TRUE, FALSE} with the same size.

In one embodiment of the present invention, the watermark information 645 is the same as the watermark information 565 when inserted in the watermark insertion module 500, or the content of the watermark information 565 when inserted. This may be information that can be inferred. In this embodiment, the symbol detection unit 650 may be configured to generate a detection symbol 655 by categorizing and classifying the contents of the watermark information 645 to determine which detection symbol value it corresponds to. there is.

In another embodiment of the present invention, the watermark information 645 may be composed of information that can directly derive the value of the detection symbol 655 or the insertion symbol 555. In this case, the detection symbol 655 can be generated simply by re-outputting the watermark information 645 without special processing.

The information restoration unit 690 may be configured to receive the detection symbol 655, which is the output of the symbol detection unit 650, as an output of the watermark detection device 620, and restore it into restoration information 695. The restoration information may be considered the same as the restoration information 155 previously described in detail in FIG. 1. In addition, it may be regarded as information that is the same as or symmetrical to the inserted information 502 described above in the description of FIG. 5 or certain information included in the inserted information 502. The information of symmetrical content refers to a relationship in which one piece of information can be specified and corresponded to another piece of information, such as a public key versus a private key or the input and output of a hash function, but is limited to the relationship in the example above. It doesn't work.

Previously, in the watermark insertion module 500, the insertion symbol 525 was separated from the insertion information 502 to watermark the frame data 515. Therefore, the restoration information 695 corresponding to the insertion information 502 was used to watermark the frame data 515. ), the detection symbol 655 obtained from the watermarked frame data 615 must be reassembled.

The information restoration unit 690 may be configured to reconstruct the detection symbol 655 in accordance with the separation method in the insertion information acquisition unit 520. For example, when the insertion information acquisition unit 520 separates the insertion symbols 525 sequentially over time, the information recovery unit 690 combines the detection symbols 655 sequentially over time to restore information ( 695).

The restoration information 695 can be utilized by the user of the watermark detection module 600. In one embodiment of the present invention, further referring to the description of FIGS. 1 and 5 above, the insertion information 502 included in the watermarked digital media 125, 125a, 502, and 601 includes viewing of the digital media. Traceability information that can uniquely identify at least one of the device or the recipient may be included, and the traceability information can be detected and restored using the restoration information (155, 695). Accordingly, it can be helpful to identify which viewing device (130, 130a) or user (150) the watermarked digital media 601 was initially provided for.

Modified Embodiment

Hereinafter, some possible modified embodiments of the watermark insertion device and its operating method according to the present invention will be described.

According to an embodiment of the present invention, frame data subject to watermarking may be divided into a plurality of areas, and watermark information may be generated with different content in the plurality of areas so that different watermarks are inserted.

Figure 7 is a conceptual diagram of frame data in which region division is performed. When there is one frame data 730, the frame data 730 can be divided into an A area 710 and a B area 720. When the frame data corresponds to, for example, one frame of video digital media, the A area 710 and B area 720 may be established by spatially dividing the frame image. At this time, the method of dividing the area may be of any type. For example, the example shown in FIG. 7 shows an example of vertically dividing a frame image into A area 710 and B area 720 of the same size, but A area 710 and B area 720 ) may be defined differently.

The method of dividing the area can be applied in various ways, such as horizontal division, division by a dividing line defined as a polygon, or division by a dividing line defined by a curve including a circle and an ellipse.

The area division method may also be defined as a different concept depending on the type of digital media. For example, if the frame data originates from audio digital media, the division may be implemented, for example, by division by frequency band, and if the frame data originates from text digital media, the division may be implemented, for example, in rows. It can be implemented as cross division by (line), and if the frame data is derived from digital media including 3D object data, it can be implemented as division by object. The method of re-dividing the information units of digital media may be any method available to those skilled in the art in terms of other digital media types or other methods.

The region dividing method may further be configured to divide the frame data into two or more regions. Additionally, the area division method may be a method of dividing at least one area so that it overlaps with at least one other area, as long as the watermark insertion method allows.

Figure 8 is a block diagram of an embodiment in which the watermark insertion device according to the present invention further includes a region division function of frame data. FIG. 8 is an embodiment modified from FIG. 5, and parts that are not changed from FIG. 5 are denoted by the same symbols.

The watermark insertion module 800 according to a modified embodiment of the present invention may include a plurality of insertion information acquisition units. For example, referring to FIG. 8, a first insertion information acquisition unit 821 that receives first insertion information 801, and a second insertion information acquisition unit 822 that receives second insertion information 802. may include. The functions and operations of each of the plurality of insertion information acquisition units may be the same or similar to the insertion information acquisition unit 520 described with FIG. 5, and the embodiment of the insertion information acquisition unit 520 explained with FIG. 5 above. It may be implemented with reference to .

The watermark insertion device 830 included in the watermark insertion module 800 may include a plurality of insertion symbol generators. For example, referring to FIG. 8, a first insertion symbol generator 851 receives and transmits the first insertion symbol 825 (855), and receives and transmits the second insertion symbol 826 (856). It may include a second insertion symbol generation unit 852 that does. The function and operation of each of the plurality of insertion symbol generators may be the same or similar to the insertion symbol generator 550 described with FIG. 5, and the embodiment of the insertion symbol generator 550 described with FIG. 5 It may be implemented with reference to .

The watermark information generator 860 of the watermark insertion device 830 included in the watermark insertion module 800 may include a plurality of area watermark information generators. For example, referring to FIG. 8, a first area watermark information generator 861 receives a first insertion symbol (855) and generates first area watermark information, and provides a second insertion symbol (856). ) may include a second area watermark information generator 862 that receives the data and generates second area watermark information. The functions and operations of each of the plurality of area watermark information generators may be the same or similar to the watermark information generator 560 described with FIG. 5, and the watermark information generator 560 described with FIG. ) may be implemented with reference to the embodiment.

The first and second area watermark information generators 861 and 862 generate first area watermark information and second area watermark information for the first and second areas respectively formed by separating the frame data 845. It may be configured to generate information. In addition, the watermark information generator 860 generates watermark information 865 to be finally executed on the frame data 845 based on the first area watermark information and the second area watermark information. It can be configured.

The frame data generation unit 840 of the watermark insertion device 830 included in the watermark insertion module 800 may further include a frame separation unit 841. The frame separator 841 may be configured to generate region frame data 845 by dividing the frame data 515 into two or more regions by a designated method. However, the region frame data 845 may be the same as the undivided frame data 545 of FIG. 5, depending on the configuration of the watermark information generator 860.

In the embodiment shown in FIG. 8, insertion information, insertion symbol, and area watermark information are independently generated for each divided area of frame data. It is obvious that even if the same information is used in some of the independent generation processes or processes processed by non-separated functional units are mixed, the basic purpose of the present embodiment is not deviated. In addition, in FIG. 8, the frame was divided into two regions, but as the number of divisions increases, the third, fourth,... An insertion information acquisition unit, an insertion symbol creation unit, and an area watermark information creation unit may be further included.

Figure 9 is an exemplary diagram of a method for inserting a watermark based on region division in a modified embodiment of the watermark insertion device according to the present invention. According to the watermark insertion module 800 illustrated in FIG. 8, divided areas of frame data can be independently watermarked using different methods. Figures 9 (a) to (d) show various implementation methods of the present invention that can be implemented through the independent watermarking.

Figure 9(a) shows a first implementation method of inserting a watermark using a divided area. If there is insertion information of length N (8 in the example of FIG. 8), the same insertion information can be inserted into the divided area using a different watermark method. One insertion symbol is generated from one insertion information, the one insertion symbol is transformed into a plurality of different area watermark information by a mutually independent area watermark information generator, and the watermark is created by combining them. It may be configured to obtain frame data watermarked by information. That is, the watermarked A-area frame data 911 and the watermarked B-area frame data 912 can be configured to coexist in one frame data. For example, in one frame data 930, area A 931 and area B 932 may include different watermarks indicating the same insertion symbol.

Figure 9(b) shows a second implementation method of inserting a watermark using a divided area. If there is insertion information of length N (8 in the example of FIG. 8), a plurality of insertion symbols can be inserted into one frame data using the divided area. Frame data combining the watermarked A-area frame data 913 and the watermarked B-area frame data 914 may be configured to include two insertion symbol information at the same time. Therefore, in this case, the length of the frame sequence required to insert insertion information of length N is N/2.

Figure 9(c) shows a third implementation method of inserting a watermark using a divided area. When there is first insertion information of length N (8 in the example of FIG. 8) and second insertion information of length M (6 in the example of FIG. 8), they are inserted into one frame data using the divided area. It may be configured to insert an insertion symbol derived from other insertion information. Frame data combining the watermarked A-area frame data 915 and the watermarked B-area frame data 916 may be configured to include insertion symbols for two different insertion information at the same time.

The insertion period for the first insertion information of length N and the insertion period for the second insertion information of length M may be configured to be inserted asynchronously as in the third implementation method. Alternatively, it may be configured to be inserted synchronously as in the fourth implementation method shown in (d) of FIG. 9. In this case, padding (to prevent the watermark from being inserted in at least one area in order to synchronize the insertion cycle) padding) interval P may be defined.

Despite the description of the various embodiments above, those skilled in the art may use methods other than those exemplified above to achieve any technical goal that can be achieved by dividing frame data into regions and performing different watermarking. Since it is obvious that a method for applying the present invention can be easily constructed, the above embodiments do not limit the scope of the present invention.

Additionally, when the watermark insertion module 800 that divides the area as described above is used, the watermark detection module may also be implemented as a modified embodiment of the corresponding method. The watermark detection device 620 included in the watermark detection module 600 of FIG. 6 further includes a frame separation unit in which the frame data generator 630 divides the watermarked frame data into two or more areas, and The mark detection unit 640 further includes an area watermark detection unit that detects an area watermark for at least one area and generates area watermark information based on the detected area watermark, and provides watermark information 645 ) may be modified to be generated based on the at least one region watermark information.

According to another embodiment of the present invention, the watermark insertion module does not actually run a watermarking algorithm, but reads frame data from a plurality of preprocessed digital media that have undergone preprocessing, such as inserting a watermark in advance, It can be configured to achieve a practical watermarking effect by selectively reading based on the inserted information.

Figure 10 is a block diagram of an embodiment of utilizing pre-processed digital media in the watermark insertion device according to the present invention. FIG. 10 is an embodiment modified from FIG. 5, and parts that are not changed from FIG. 5 are denoted by the same symbols.

The watermark insertion module 1000 according to a modified embodiment of the present invention may include a plurality of media acquisition units, and the plurality of media acquisition units may be configured to acquire differently preprocessed digital media. For example, referring to Figure 10, a first pre-processed media acquisition unit 1011 for acquiring first pre-processed digital media 1001, and a second pre-processed media acquisition unit for acquiring second pre-processed digital media 1002. It may include part 1012. The functions and operations of each of the plurality of pre-processing media acquisition units may be the same or similar to the media acquisition unit 510 described with reference to FIG. 5. Refer to the embodiment of the media acquisition unit 510 described with FIG. 5. So it may be implemented.

However, the plurality of pre-processed media acquisition units 1011 and 1012 may be configured to read digital media that has been watermarked through pre-processing in advance instead of the original digital media 501. At this time, the first pre-processed digital media 1001 may be watermarked in advance by inserting a first symbol for all frame data, and the second pre-processed digital media 1002 may be watermarked in advance with a first symbol inserted for all frame data. The symbol may have been inserted and watermarked in advance. The plurality of pre-processed media acquisition units (1011, 1012) separate frames from the plurality of pre-processed digital media (1001, 1002) to generate first frame data (1005) and second frame data (1006), , and may be configured to provide this to the watermark insertion device 1030 (1045, 1046).

The watermark insertion device 1030 included in the watermark insertion module 1000 may include a frame selection unit 1070 instead of the watermark insertion unit. The frame selection unit may be configured to receive watermark information 565 from the watermark information generator 560 and select frame data in which the watermark indicated in the watermark information has been previously inserted. For example, if the insertion symbol 555 generated from the insertion information 520 is the first symbol, the first frame data 1005 derived from the preprocessed digital media 1001 that has been previously watermarked with only the first symbol is used. It can be configured to receive data (1045) and output it as watermarked frame data (575).

Figure 11 is an exemplary diagram of a method of inserting a watermark using a frame selection unit in a modified embodiment of the watermark insertion device according to the present invention.

The first pre-processed digital media 1111 and the second pre-processed digital media 1112 each have a watermark inserted into the same original digital media. The symbol “0” is continuously inserted into all frames belonging to the first pre-processed digital media 1111, and the symbol “1” is continuously inserted into all frames belonging to the second pre-processed digital media 1112. When the insertion symbol is determined to be “01000001” through the insertion information, the frame selection unit 1070 selects the frame data derived from the preprocessed digital media that matches the value of the insertion symbol as watermarked frame data and outputs it. It can be configured to do so.

In the modified embodiment shown in FIGS. 10 and 11, the first pre-processed digital media (1001, 1111) and the second pre-processed digital media (1002, 1112) can be prepared by watermarking them in advance. Thus, it is possible to prevent overload of computation due to watermark insertion in the watermark insertion module 1000. Therefore, when distributing one original digital media to multiple users in real time, tracking information corresponding to each user is provided in real time. It can be helpful to insert and provide.

In addition, in FIG. 10, the description is based on the set size to which the value of the inserted symbol belongs is 2 (e.g., {0,1}). However, when the size of the set increases, the third, fourth,... The pre-processed digital media, pre-processed media acquisition unit, and frame data generation unit may further be included or used.

Lastly, the embodiments described in FIGS. 7 to 9 and the embodiments described in FIGS. 10 to 11 may be used in a combined or mixed form. For example, the frame is divided into two and watermarked, but a watermark is inserted by the watermark insertion unit in the first area, and the content corresponding to the second area is selected from one of the digital media that has been pre-processed in advance, This combination may be implemented to be used as watermarked frame data.

The present invention has been described above with reference to the drawings and examples, and some predictable modified examples have been discussed. However, as already mentioned above, the scope of protection of the present invention is limited by the drawings or examples shown above. It does not mean that the present invention is intended to be modified, and those skilled in the art will understand that various modifications and changes can be made to the present invention without departing from the spirit and scope of the present invention as set forth in the claims below.

Claims (20)

A frame data generator that generates frame data;
an insertion symbol generator that generates an insertion symbol;
a watermark information generator that generates watermark information based on the inserted symbol; and
a watermark insertion unit configured to generate watermarked frame data by inserting a watermark into the frame data based on the watermark information;
The inserted symbol belongs to a set including at least a first symbol and a second symbol,
The watermark information is generated differently depending on the symbol of the insertion symbol.
According to claim 1,
a media acquisition unit that reads digital media including at least one frame data; and
It further includes an insertion sequence acquisition unit that reads an insertion sequence including at least one insertion symbol,
The frame data generator is configured to sequentially read one frame data from the digital media and generate frame data,
The insertion symbol generator is configured to generate an insertion symbol by sequentially reading one insertion symbol from the insertion sequence.
According to clause 2,
The insertion symbol sequence includes information related to the content represented by the frame data, identification information related to the viewing device on which the watermarking frame data is displayed or received, identification information related to the user of the viewing device, and the viewing device. A device, comprising at least one of information related to a connection to communicate with the device.
According to claim 1,
The watermark information includes at least one of information related to the insertion power of the watermark or information related to the insertion method of the watermark.
According to claim 1,
The frame data generator further includes a frame separator that divides the frame data into two or more regions,
The watermark information generator further includes a region watermark information generator that generates region watermark information for at least one of the two or more regions,
The watermark information is generated based on the at least one area watermark information.
According to claim 5,
The area watermark information includes at least one of information related to the insertion power of the watermark or information related to the insertion method of the watermark.
According to claim 5,
The insertion symbol generator is configured to generate a plurality of insertion symbols including at least a first symbol and a second symbol,
The area watermark information generator generates the area watermark information by using the first symbol as an insertion symbol in a first area among the two or more areas and using the second symbol as an insertion symbol in a second area. .
According to claim 1,
Further comprising a memory for storing at least one frame data or at least one watermark frame data as previous frame data,
The watermark information generator further includes a suitability evaluation unit that analyzes the frame data and at least one previous frame data to evaluate watermark insertion suitability for the frame data,
The watermark information is generated including information for not inserting a watermark when the suitability is less than a threshold.
According to claim 8,
The suitability evaluation unit determines continuity of the frame data, brightness change, saturation change of the frame data, motion vector size of the frame data, scene transition included in the frame data, and I/P of the frame included in the frame data. The device is configured to determine the fitness based on at least one of the /B types.
a frame data generator that generates watermarked frame data;
a watermark detection unit that detects a watermark from the watermarked frame data and generates watermark information based on the detected watermark; and
It includes a symbol detection unit that generates a detection symbol based on the watermark information;
The watermark detection device wherein the detected symbol belongs to a set including at least a first symbol and a second symbol based on the detected watermark information.
According to claim 10,
The watermark information includes at least one of information related to the power of the detected watermark or information related to the method of the detected watermark.
According to claim 10,
The frame data generator further includes a frame separator that divides the watermarked frame data into two or more regions,
The watermark detection unit further includes an area watermark detection unit that detects an area watermark for at least one area and generates area watermark information based on the detected area watermark,
The watermark information is generated based on the at least one area watermark information.
According to claim 12,
The area watermark information includes at least one of information related to the power of the detected area watermark or information related to a method of the detected area watermark.
According to claim 10,
The device further includes a sequence restoration unit that generates a detection sequence based on at least one detection symbol.
According to claim 14,
The detection sequence includes information related to the content represented by the frame data, identification information related to the viewing device on which the watermarking frame data is displayed or received, identification information related to a user of the viewing device, and the viewing device. A device comprising at least one of information related to a connection to communicate with.
generating frame data;
creating an insertion symbol;
generating watermark information based on the inserted symbol; and
generating watermarked frame data by inserting a watermark into the frame data based on the watermark information;
The inserted symbol belongs to a set including at least a first symbol and a second symbol,
The watermark information is generated differently depending on the symbol of the insertion symbol.
According to claim 16,
The watermark information includes at least one of information related to the insertion power of the watermark or information related to the insertion method of the watermark.
generating watermarked frame data;
detecting a watermark from the watermark frame data and generating watermark information based on the detected watermark; and
generating a detection symbol based on the watermark information;
The watermark detection device wherein the detected symbol belongs to a set including at least a first symbol and a second symbol based on the detected watermark information.
According to claim 18,
The watermark information includes at least one of information related to the power of the detected watermark or information related to the method of the detected watermark.
a media acquisition module that reads digital media containing at least one frame data;
an insertion sequence acquisition module that reads an insertion sequence containing at least one insertion symbol;
One frame data is sequentially read from the digital image, one insertion symbol is sequentially read from the insertion sequence, watermark information is generated based on the insertion symbol, and the frame is based on the watermark information. a watermark insertion module that inserts a watermark into data to generate watermark frame data; and
A media reconstruction module that sequentially combines the watermark frame data to output watermarked digital media,
The inserted symbol belongs to a set including at least a first symbol and a second symbol,
The watermark information is generated differently depending on the insertion symbol.