KR20030051712A - Halftone watermarking and related applications - Google Patents

Abstract

One method for halftone image watermarking assigns halftone watermark dot values to pseudorandom positions in an image and assigns multilevel pixels to adjacent pixel positions of these dots. Spread into the field. Another method modulates halftone thresholds to convert multilevel pixels into watermarked halftone pixels. This method can be used in connection with a robust watermark 110 that spreads through the image before inserting the halftone watermark. In a compatible watermark reader 124, the digital watermark signal strength is measured by analyzing bit errors in the extracted watermark message signal.

Description

Halftone watermarking and related applications

Digital watermarking is a process for modifying physical or electronic media for embedding machine readable code into media. The medium can be modified such that the inserted code can be hardly or hardly recognized by the user, and can also be detected through an automatic detection process. Most commonly, digital watermarking is applied to media signals such as images, audio signals, video signals, and the like. However, it also includes other forms of media, including documents (eg via line, word or character shifting), software, multi-dimensional graphics models and surface textures of objects. It can be applied to objects.

Digital watermarking systems typically detect and read embedded watermarks from two main components: an encoder that embeds a watermark into a host media signal, and a signal (suspicious signal) suspected of containing a watermark. Has a decoder The encoder inserts a watermark by changing the host media signal. The reading component analyzes the suspicious signal to detect whether a watermark is present. In applications where the watermark encodes information, the reader extracts this information from the detected watermark.

Several special watermarking techniques have been developed. The reader is assumed to be familiar with the literature in this field. Specific techniques for inserting and detecting unrecognizable watermarks in media signals are described in detail in Assignee's co-pending application No. 09 / 503,881 and US Pat. No. 5,862,260, which are incorporated herein by reference. It is integrated as.

The present invention relates to multimedia signal processing, and more particularly to image watermarking methods and related applications.

1 illustrates an example of an error diffusion mask used to generate halftone images from multilevel images per pixel.

2 illustrates another example of an error diffusion mask used to generate halftone images from multilevel images per pixel.

3 illustrates another example of an error diffusion mask used to generate halftone images from multilevel images per pixel.

4 is an application diagram of a halftone watermark application.

5 illustrates a method of generating a watermarked halftone image using a threshold mask.

6 illustrates a method of using a halftone screen as a direction signal to determine geometric distortion of a watermarked image signal.

The present invention provides halftone image watermark methods and systems.

One aspect of the present invention is a halftone image watermarking method. This method assigns a set of halftone watermark values to locations in the halftone image. It spreads the error associated with halftone watermark values at neighboring locations in the halftone image. The error is characterized by the difference between the multilevel pixel value at the position of the halftone watermark dot and the halftone watermark value of the halftone watermark dot. This method can be used in conjunction with other watermark embedding stages. For example, a robust watermark containing a key for the halftone watermark may be inserted into the image before inserting the halftone watermark.

Another aspect of the invention is a method of decoding a halftone watermark from an image. This decoding method uses a key to identify the positions and values of the halftone watermark, and uses the pixel values at the positions to determine whether the values at the positions correspond to the values specified by the key. Analyze In one application, a robust watermark embedded in an image has a key used to decode a halftone watermark from the image. The decoder reads a robust watermark from the scan of the halftone watermarked image and compensates for geometric distortion of the scanned image, optionally using an orientation signal. The decoder then extracts the key from the robust watermark and passes it to the detector, which also performs a high resolution scan of the suspicious image to determine whether a halftone watermark is present at the locations specified by the key. Check it.

Another aspect of the invention is another method of embedding a watermark in a halftone image. This method calculates a watermark image that includes an array of values corresponding to pixel locations in the halftone image. It inserts a watermark image into the halftone image by using the values of the watermark image to modulate the thresholds at pixel locations. Thresholds are used in halftone processing to convert multilevel pixel values in a multilevel per pixel image to halftone pixel values of the halftone image.

Another aspect of the invention is another method of embedding a watermark in a halftone image. This method calculates a watermark image at halftone resolution that includes an array of values corresponding to pixel locations within the target halftone image. It then combines the array of values of the watermark image with the corresponding multilevel pixel values of the multilevel image per pixel to produce an image per pixel watermarked at the resolution of the target halftone image. Finally, it performs halftone processing to convert the watermarked image per pixel into a watermarked halftone image.

Another aspect of the invention is a watermark decoder. The decoder has a watermark detector and a reader. This detector interprets portions of the image to detect the watermark signal embedded in the image. This image is scanned from the halftone printed image at a sufficiently high resolution to identify the inserted watermark image at the resolution of the halftone image. The watermark reader reads the watermark signal from parts of the image and decodes an auxiliary message comprising one or more symbols.

Another aspect of the invention is a method of embedding a digital watermark in a halftone image. This method redundantly encodes a multi-bit message and converts the encoded message into a multilevel watermark image per pixel. It then derives halftone thresholds from the per-pixel multilevel watermark image. These thresholds are then used to convert the target images into watermarked halftone images.

Multilevel pixels in the target image are used to select corresponding halftone thresholds from halftone thresholds derived from the watermark image. The selected thresholds are applied to corresponding multilevel pixels in the watermark image to produce a watermarked halftone image of the target image.

Another aspect of the invention is a method of measuring digital watermark intensity. In this method, a watermarked signal is processed to extract estimates of error correction encoded bits embedded in the watermarked signal. The error correction encoded bits are then decoded to calculate the message payload. The message payload is re-encoded to calculate error correction encoded bits. The magnitude of the watermark intensity is calculated from the error correction encoded bits and the estimates of the error correction encoded bits. In particular, in one implementation, the soft bit estimates decoded from the watermarked signal are multiplied by corresponding recalculated error correction encoded bits and summed to obtain the magnitude of the watermark signal strength. This measurement may be compared to a threshold to detect interference with watermarked signals such as compression, scanning and reprinting, photo-coping, and the like.

For print applications, the embedded digital watermark can be used to carry the decoded printer information later by the watermark detector and examine the digital scan of the printed object and determine whether the printed object is authentic. Can be used to

Further features will become apparent with reference to the following detailed description and the accompanying drawings.

(details)

The following description details the methods and related applications for watermarking halftone images. While watermarking methods apply to other forms of halftoning, the description provides specific examples applicable to the error diffusion techniques used to generate halftone images.

One form of error diffusion used to process halftone images is called Floyd-Steinberg error diffusion. In a typical implementation of 8 bits per pixel images, this error diffusion method takes a 0-255 level input image at a first resolution (eg 200 pixels per inch) and uses a higher resolution (eg inch). Generate a binary image with 600 dots per second). This description details the implementation of the image plane, where each pixel has a corresponding multilevel value, for example a luminance value, or other color channels, such as cyan, magenta, yellow, and the like. The description applies to color images having more than one multilevel value per pixel. In such cases, halftone processing is done for each of the color channels per pixel and produces a halftone image for each channel.

As a first step, the input image is upsampled to higher resolution. One method for generating binary 600 dots / inches image from an upsampled 0-255 level 600 pixels / inches (ppi) image is to threshold the pixel values so that It is to generate zero at the corresponding position of the binary image, otherwise the corresponding position may be set to one (1). Error diffusion takes this general approach in raster-scan order but also achieves improved performance by "diffusing" the error at each location to locations in the vicinity to be processed. This error diffusion is guided by the weighting mask shown in FIG.

"X" represents the pixel position currently being processed, and adjacent cells show how the error spreads from that position. For this algorithm, the sum of the spread errors is equal to the overall error at position X. For some particular binary positions, the precision of this method is not better than the simple thresholding method. However, if the pixels from the original 200 ppi image are compared with the corresponding 3x3 binary region, the number of cells with one correlates closely with the multilevel pixel value.

A more clear description of the basic error spreading algorithm uses three equations. See “Digital Color Halftoning,” page 359, by Kang (co-published by The International Society for Optical Engineering and IEEE Press 1999), incorporated herein by reference.

(m, n): Binary (upsampled) pixel position

p _i (m, n): Upsampled input strength, 0-255

p '(m, n): modified input strength

p ₀ (m, n): output binary value

e (m, n): error at position (m, n)

w _kl : error diffusion weighting coefficients

The weight mask shown in FIG. 1 is modified to show which errors spread to a given location. 2 shows an example of such variations.

In the following paragraphs, we describe a modified error diffusion method of inserting a watermark containing a set of binary values at specific dot positions of a binary image. This method starts with an upsampled binary host image, a list of dot locations in the binary image, and corresponding binary values for the watermark. This method assigns the corresponding values of the watermark to these locations and attempts to improve the image with an error diffusion algorithm at other (non-watermark) locations. It is assumed that the portion of the total pixels occupied by the watermark is small.

In the error diffusion algorithm introduced above, the error for a given position is always calculated after the algorithm has processed all preceding positions. Typically, the algorithm scans a rectangular image containing scanline rows of pixels scanning from left to right across each row starting from the top row. The modified error diffusion method calculates error values for all positions of the starting point and corrects them as this method proceeds. At the start point, the error of all positions not covered by the watermark is set to zero. For the position covered by the watermark, the error is

e (m, n) = p _i (m, n) -255W (m, n)

Is calculated as Where W (m, n) is the value of the watermark at position (m, n). A new set of error spreading weights is used. That is, the pattern of error diffusion to position X is shown in FIG.

In this way, the spread of errors occurs in two directions. Errors from watermarked locations spread in one direction (backwards in this case), and errors from processed locations spread in the other direction (fronts). As each location is processed, the error for that location is updated for later spreading. The calculation of the output binary value is not changed from the error spreading algorithm, except for the watermark positions. Where p ₀ (m, n) is set to W (m, n).

The appearance of the watermark can be improved by arranging the locations of the watermark dots in a pseudorandom pattern according to some additional human visual system frequency response criteria. One way to arrange the positions of the watermark dots is to use a minimum visual cost techique. In this way the watermark is selected by minimizing the following cost function, for example.

C = ∬ ｜ H (f _x , f _y ) V (f _x , f _y ) ｜ ² df _x df _y

Where H (f _x , f _y ) is the frequency spectrum of the watermark and V (f _x , f _y ) is the frequency response model of the human visual system. One possible such model is by Sullivan et al. And is given in Kang, section 5.6. The effect of this cost function is to weight it by human visual capability to detect the frequency content of the watermark. In this way, watermark patterns in which the human visual system moves frequency content into areas of the less sensitive frequency spectrum will have lower costs. This method of selecting values and positions of watermark dots provides a perceptually more uniform distribution of dots in a solid line region (e.g., a white region in a grayscale image). Search for a watermark with low cost can be done by various methods. One such method is simulated annealing.

The encoder may repeat the watermark in blocks of the host image, eg, adjacent blocks of pixels through the image. Each block may use the same or different keys. If the encoder uses different keys per block, each key can be associated with another key by a secret function, for example an encryption function.

To read the watermark, the watermark decoder uses key specifying where the watermark dots are located. The binary values of these dots are then determined. If a watermark is embedded in the printed image, the scanner or camera with a high resolution sufficient to read the halftone dots produces a digital image from which the decoder extracts the watermark.

There are many applications of this type of watermark. It may be used to carry a message containing usage control instructions or other metadata, such as an identifier or index of an image owner, in a database record that stores relevant information. It may also convey a fixed pattern used to determine whether a watermarked image (eg a printed halftone watermarked image) has been authenticated or changed. In such an application, the decoder compares the extracted watermark pattern with a known pattern and determines whether the watermarked image has been altered (e.g., copied, scanned and reprinted, compressed, etc.) based on this comparison. Determine. It can also specify where the image has changed by showing locations in the scanned image where no halftone watermark is present.

Since the watermark is applied to the halftone images, it can be applied as part of the print process in which the multilevel digital image per pixel is converted to a halftone image. For example, it may be integrated in a halftoning process implemented in a printer device or printer driver running on a computer. More generally, the watermarking method may be applied to applications in which halftone images are printed, including commercial print presses as well as personal print devices such as inkjet printers.

In some forms of image watermarking, the host image interferes with the watermark. The host image provides a communication channel for the watermark. In decoding the watermark from the host image, the host image may take into account noise that interferes with the watermark decoder's ability to accurately extract the watermark signal. To increase the chances of accurate recovery of the watermark, it is distributed over a large portion of the host image (possibly the entire image).

Watermarks of the type described in the previous paragraph can be inserted directly into the halftone image. The discussion below describes an error diffusion method for directly embedding this type of watermark in a halftone image.

In this method, the watermark to be inserted, W (m, n), takes values of 0-255. The error spreading process is changed such that the threshold used to calculate the binary output values is modulated by the next watermark signal.

The intensity level I controls how severely the watermark is inserted in the image.

As an alternative to modulating error diffusion thresholds, the watermark can be inserted without modulating the halftone processing. For example, a multilevel watermark signal per pixel is generated at the resolution of the target halftone image. The watermark encoder generates an multilevel watermark signal per pixel at the desired resolution of the halftone image or at slightly different resolutions and up or down sampling to match the resolution of the target halftone image. do. This watermarked signal is then added to the host image at the same spatial resolution to produce a composite watermarked image. Thereafter, error diffusion processing or some other type of halftone processing may then be applied directly to this composite image to produce a watermarked halftone image. This technique applies to various halftone processes, including ordered dithering (eg, blue noise masks, clustereddot halftones, etc.) as well as error diffusion halftone processes.

There are various ways to generate a watermark signal. One approach takes an auxiliary message that includes binary or M-ary symbols, applies error correction coding to it, and then encodes error correction. Apply spread spectrum modulate to a message. One way to spread the spectral modulation message is to spread each binary symbol in the message for pseudorandom numbers using an exclusive OR or multiplication operation. The binary message elements obtained in the spread spectrum modulated message signal are then mapped to spatial image locations. The watermark signal may be represented in a binary antipodal form, where the binary symbols are positive or negative. To increase the robustness, the spread spectrum modulated message signal can be repeated throughout the host image, for example by inserting the message signal into the host image of several blocks. In particular, the watermark encoder can insert the watermark signal into successive blocks of pixels, for example, over a portion of the host image or over the entire host image.

Perceptual modeling may be applied to the host image to calculate the gain vector with gain values corresponding to the message signal elements. For example, if an upsampled watermarked signal is added to the host signal, the gain values can be used to scale the binary opposite values of the message signal before adding them to the host signal. Each gain value may be a function of desired watermark visibility and detectability constrains. In particular, the preceptual model analyzes the image to determine a range in which the corresponding element of the watermark image can be hidden. One form of analysis is to calculate the local contrast within each pixel (eg, signal activity) in the vicinity of the neighbor and to select the gain for the pixel as a function of the local contrast. A detectability model analyzes the host signal to determine the range in which pixel values are biased relative to the value of the watermark signal at corresponding pixel locations. The gain is then adjusted up or down depending on the range in which the host image pixels are biased relative to the watermark signal.

This type of watermark is derived from a watermarked halftone image or other image representations of the watermarked image, such as a multilevel per pixel representation of a fairly high resolution image to represent the watermark signal. Can be read. To decode the watermark, the watermark decoder detects the presence and orientation of the watermark in the watermarked image. Thereafter, it performs the inverse of the embedding function to extract the estimated watermark message signal.

The message signal is strongly encoded using a combination of the following processes.

1. repeatedly encoding a message signal at various locations (eg, blocks of an image);

2. spread spectrum modulation of the message, including modulation techniques using M sequences and gold codes;

3. Error correction coding, such as convolution coding, turbo coding, BCH coding, Reed Solomon coding, and the like.

The watermark decoder,

1. gathering estimates of the same message element in each inserted example of the message;

2. performing spread spectrum demodulation;

3. Reconstruct the embedded message from the estimated watermark signal by decoding the error correction.

In one implementation, the decoder uses the direction signal component of the watermark to detect its presence and direction in the watermarked image. It then performs predictive filtering on the image sample values to estimate the original watermarked signal, and subtracts the original estimate from the watermarked signal to produce an estimate of the watermark signal. It performs error correction decoding and spread spectrum demodulation to reconstruct any message embedded in the watermarked signal.

More details on embedding an image watermark, and detecting and reading the watermark from a digitized version of the image after printing and scanning are incorporated herein by reference, the assignee's pending application number. See 09 / 503,881 and US Pat. No. 5,862,260. To create a watermark that is robust against topographical distortion, the watermark includes a directional watermark signal component. In addition, it includes a direction watermark signal from the watermark message signal and the watermark signal. These components may be added to the host image at the resolution of the halftone image before the host image is converted to a halftone image. Alternatively, these components may be combined to form a watermark signal used to modulate the error spread threshold used for halftone processing of the error spread type.

One type of watermark direction signal is an image signal comprising a set of impulse functions in the Fourier magnitude domain, each having a pseudorandom phase. In order to detect the rotation and scale of the watermarked image (eg, after printing and scanning), the watermark decoder converts the image to a Fourier size domain, and then resamples the log pole of the Fourier size image. (log polar resampling) is performed. A generalized matched filter correlates the re-sampled watermarked signal with a known direction signal to find the rotation and scale parameters that provide the highest correlation. The watermark decoder performs additional correlation operations between the watermarked signal and the phase information of the known direction signal to determine translation parameters that identify the origin of the watermark signal. Once the rotation, scale, and translation of the watermark signal has been determined, the reader then adjusts the image data to compensate for this distortion and extracts the watermark message signal as described above.

The halftone watermarks described above can be used in combination with one or more other watermarks. In one application, for example, a robust watermark is used to convey a key specifying the dot locations of a halftone watermark. In particular, a robust watermark's message payload carries a key that identifies certain dots (high resolution binary values) turned on or off in a specific pattern, these binary value bits are scanned for this image. It acts as a secondary fragile watermark that can be verified.

4 shows an implementation of this application. On the watermark embedding side, the watermark encoder 100 operates on the input image 102 to insert a robust watermark that withstands printing, scanning, and topographical distortions. Examples of watermarks of this type are disclosed in the above patents and patent applications incorporated by reference. At least a portion of this robust watermark message payload carries the key used to decode the halftone watermark at specified halftone dot locations in the halftone image.

In this implementation, the key is assigned to a random number generator 104 that identifies the locations of halftone watermark dots in the halftone image and also specifies the binary values (on or off) of the dots at those locations. Seed for the seed. Thereafter, the halftone converter 106 (e.g., error diffusion, blue noise masking, etc.) robustly waters, ensuring that halftone watermark dots are assigned correction values based on the output of the random number generator. Convert the marked image to a halftone image.

One example of such a process is to ensure that these watermark dot values are fixed, while the halftone converter spreads the error induced by the halfwatermark dots at these respective locations in the spatially neighboring perimeter. This approach reduces the significant impact of halftone watermarks. Other halftone methods can be used as long as they ensure that halftone watermark dots are set as specified by the key. The result is a halftone image 108 that can be printed using conventional printer techniques such as ink jet printing and the like 110. In practice, the watermark embedding process 100 can be implemented in a printer or a driver for a printer.

To verify the authenticity of the printed image, the scanner 120 captures the digital image from the printed image 122. Thereafter, as the robust watermark survives printing and scanning by a low resolution (eg, 100 dpi) scanner or digital camera, it can be recovered from the digital image captured from the printed image 122. Watermark reader 124 using the decoding operations described above detects a robust watermark, determines its direction, and then reads the watermark payload containing the key. This supplies a key to random number generator 126 that provides values of halftone watermark and halftone dot locations.

Secondary watermark verifier 128 then checks for a suitable high resolution scan of the printed image to determine if a halftone watermark is present. A "suitable high resolution scan" of a printed image is one in which the heartphone dots of the printed image are readable. This high resolution scan may be the same image captured by scanner 120 if it is at a sufficient resolution. Alternatively, individual image capture device 130 may be used to capture high resolution images representing halftone dots. Verifier 128 provides a signal indicative of the range in which the halftone watermark is present.

Based on the output of the detector, a number of actions can be taken. Some operations include recording information identifying a user (eg, device ID, address (eg network address), user ID, etc. of the user's computer or imaging device), and remote device via the communication link with this information. To transmit the results of the verification operation to, to display information about the rules governing the use of the image, to electronically permit or purchase rights, products, or services related to the user, or Connecting the decoding system to a licensing or electronic transaction server.

Tough and soft halftone watermarks can also convey other information. They are indexes to database records that store usage control instructions or other metadata such as the image owner's identifier, a computer address for establishing a remote connection (eg, IP address, URL, etc.), or related information. Can be used to convey a message containing (index). In one application, a message carries an identifier that is used to fetch related information into an image. In particular, the watermark decoder communicates the identifier in the form of a request to the database management system. The database management system and underlying database records can be implemented at a remote device connected to the watermark decoder device via a network connection (eg, a web server on the Internet connected via a TCP / IP connection). Mark decoder (eg, a local database running in a device such as a decoder or in a computer locally connected to the decoding device via a port, such as a serial, parallel, infrared Bluetooth wireless or other peripheral port). May be implemented within a system that includes In response to the request, the database looks for information related to the identifier or identifiers extracted from the robust watermark. One example is to use an identifier or address embedded in a robust watermark to fetch information from a web site related to the watermarked image. For more information on this use of robust watermarks, see US Pat. Nos. 5,841,978 and pending applications 09 / 571,422, 09 / 563,664, and 09 / 597,209, which are incorporated herein by reference. See arc.

As mentioned above, for many image watermark applications, as part of the watermark detection and reading process, it is necessary to determine the direction (scale, rotation, etc.) of the watermarked image before the process of reading the watermark. . One way of determining the orientation of a watermarked image relative to its orientation at the time of embedding the watermark is to use some form of calibration signal as part of the watermark. The calibration signal may be integrally related to the watermark message signal, such as a carrier signal or a synchronization code. Alternatively, this may be an individual signal, such as a registration template that is not detectable.

The following watermark method uses a halftone calibration image screen as a direction signal, which avoids the need to insert individual calibration signals. Halftone screens can be used to print halftone images that are not normally watermarked, or can be specifically adapted to generate and print watermarked halftone images.

This method is suitable for image halftone methods in which halftones are implemented through the use of threshold masks. Threshold masks are patterns that can be tiled in an iterative manner on an image. Most commonly, the threshold mask is a square matrix of numbers. For example, the threshold mask may be 128 × 128, and the image may consist of an array of 8 bit pixels. In this case, the threshold mask includes numbers in the range of 0 to 255, corresponding to 256 possible levels of 8 bit pixel value. If the number of elements in the mask exceeds the number of levels per pixel, the threshold mask is generally designed to include the same number of occurrences of each level. For example, in the current example where there are 16384 elements of a 128 × 128 mask, there are 64 entries of the threshold mask for each level (16384/256 = 64). It is possible to use a 16 x 16 mask with each element corresponding to 0 to 255 different levels. However, when tiled across an image, such a mask can introduce more visual artifacts than a larger mask. Of course, these masks are merely examples, and there are many alternative combinations of mask sizes, dimensions, levels for threshold masks used in halftone screen processes.

In typical halftone screen processing, a threshold mask is used to generate a binary pattern of ink dots used to represent a printed version of an image. If an image with 200 pixels per inch is printed using a resolution of 2400 dots per inch, the image is first upsampled to 2400 pixels per inch. The threshold mask is tiled and applied to the upsampled image to construct a halftone image. Each pixel of the halftone image is calculated from the corresponding pixel of the upsampled image and the value of the threshold mask tiled at that location. If the threshold mask value is less than or equal to the upsampled image pixel, then the halftone image pixel is set to one, otherwise it is set to zero.

Error diffusion halftoning may not be implemented in this way.

5 illustrates a method of generating a watermarked halftone image using a threshold mask. The watermark encoding process begins by converting any message into a watermark image signal 150. The method shown in FIG. 5 may be used in various ways of constructing a watermark image signal. For purposes of explanation, FIG. 5 refers to an example of using a spatial spread spectrum watermark. In this case, the watermark message payload is a spread spectrum that is modulated over the same time region as the threshold mask. Frequency domain techniques, in which a watermark signal modulates frequency coefficients to encode message symbols, and a watermark is characterized by its characteristics such as its autocorrelation, power, amplitude, signal peaks, etc. Alternative watermark encoding functions may be used, such as modulating statistical features. Each of these insertion functions may be characterized as adding a spatial watermark image with corresponding elements in the host image.

The encoder may modulate the watermark image signal with a gain that makes the watermark message easier to recover, less visible, or to balance recovery and imperceptibility. In the case of a spatial spread spectrum watermark image, the encoder converts the message to a binary antipodal signal (150), and the gain modulator adjusts the value of each element of the signal (152). The spatial resolution of the watermark image signal corresponds to the target halftone image.

The watermark image signal may be tiled such that the same payload may be repeated or changed for the image and different information is inserted in different regions of the image.

To prepare a host image for the watermark, the encoder takes a multilevel form per pixel of the image 154 and converts it to halftone resolution (156). This step typically involves upsampling a multilevel image at a higher resolution (eg, upsampling 200 dpi, which is 8 bits per pixel image, to 2400 dpi per pixel image). One way to embed a watermark is to add a spatial domain representation of the watermark image signal with the multilevel image signal, respectively, at halftone image resolution to produce a composite image.

The encoder then applies threshold masks 160 and 162 to the resulting composite image. Halftone screen processing switches pixel levels below or equal to the corresponding threshold mask level to zero, otherwise sets them to one. The result is a watermarked halftone image. The image can then be printed using various halftone printing processes, including inkjet printers, and printing process.

6 illustrates a method of using a halftone screen as a direction signal to determine topographical distortion of a watermarked image signal. In particular, this method is used to calculate the direction of the host signal at the time of embedding the watermark to facilitate recovery of the watermark message.

First, a scanner, camera or other imaging device captures a digital version of the watermark image 170 printed at a significantly higher resolution 172 to distinguish halftone pixels. In subsequent examples, the watermark decoder obtains blocks of the digital image 174 each having an area approximately corresponding to the size of the threshold mask (eg 128 × 128). Each digital image pixel is approximately equal in size to the halftone image pixel.

Next, the decoder filters the image blocks using lowpass filtering techniques, such as calculating average pixel values and replacing each pixel with an average. The decoder then applies the same threshold mask used to generate the halftone image for printing (178, 180). This processing results in a target orientation signal with orientation characteristics similar to the orientation image.

To determine the geometric distortion of the received image with respect to the original watermarked image, the decoder performs a series of correlation operations between the object direction signal and the received image. In the method shown in FIG. 6, the decoder transforms the received image block and the target direction signal 182 derived therefrom into the phase and domain 182, in particular, it is a 128 × 128 fast Fourier transform. And resample the Fourier size representation obtained with a log-polar coordinate system to obtain a Fourier-Mellin representation of the images. Correlation actuator 184, such as a generalized matched filter, correlates a Fourier-Mellin representation of the target signal (reference signal) with a Fourier-Mellin representation of the received image. The correlation operation produces an estimate of the scale and rotation parameters of the digital image.

The decoder transforms the received spatial digital image as the inverse of the estimated rotation and scale. The correlation operator correlates the spatial target direction signal with the obtained spatial image to obtain a translation estimate of the image with respect to the threshold mask.

If there are estimates for the scale, rotation, and translation parameters of the digital version of the watermarked image, the watermark message reader 186 demodulates the spread spectrum signal from the received image.

Each of the halftone watermark methods described above can be used for "fragile watermark" or "semi-fragile" watermark applications, the watermark being replaced with a watermarked image. It is analyzed to detect and characterize alterations. A soft watermark refers to a watermark that degrades when it is subjected to certain types of distortions but becomes unreadable. A semi-soft watermark is a variation on this theme where the watermark survives some type of distortion, while others do not.

One approach is to design a watermark so that it cannot degrade or read in response to certain types of distortions such as printing, scanning on consumer grade scanners, image compression, and the like. If the decoder is unable to detect or read the watermark, or if the measured column exceeds the threshold, then the decoder provides an output indicating that the image has been modified (eg, not authenticated). Degradation can be measured as the degree of recovery of the watermark signal, such as the degree of correlation between the known watermark signal and what is extracted from the received image. In such soft or semi-soft applications, the gain values of the halftone watermark methods described above can be selected to provide an accurate balance between read and easy for some types of distortions.

An enhanced approach is to design watermarks so that the types of variations can be distinguished. For example, the type of deformation may be characterized by the type of degradation due to the watermark signal. By measuring the amount of degradation for other aspects of the watermark signal, the decoder can match the observed degradation to a particular type of deformation.

The performance of such weak watermarking applications can be improved by selecting the frequency distribution of the embedded watermark to differentiate between different forms of distortion. There are two main considerations in designing watermark signal properties. The first is to put a portion of the watermark's energy into frequencies that are prone to degradation by the particular type of degradation to be detected (such as print and scan operations); The second is to put energy in frequencies that are less prone to degradation by the distortion that the watermark (such as smudges and normal wear and tear) must survive. By analyzing the frequency distribution of the watermark, the decoder can distinguish between forms of change. This approach is particularly useful for determining whether a printed image is original (e.g., reproduced through scan and print operations), as opposed to being solid and warning only through normal use.

The digital watermark can be embedded in the halftone image using the digital watermark signal as a halftone screen. The following discussion shows an example of the procedure:

1. For example, generating a digital watermark signal using the spread spectrum techniques described above. Optionally, the watermark signal may contain an orientation signal such as an orientation signal that produces a pattern when transformed into a frequency domain, such as a Fourier domain, or a pseudo random carrier signal that produces a pattern of signal peaks when transformed into an autocorrelation domain. It may include.

The resulting watermark signal is an array of luminance values or a multilevel gray scale image per pixel. This is a known block size, such as 64 X 64, 128 X 128, 256 X 256 pixel elements, which can be tiled contiguously to produce images of variable sizes.

2. Calculate the histogram of the watermark image block.

3. Use the histogram to set halftone threshold levels for corresponding gray values (or luminance values). This example uses gray levels, but the same technique is applied to luminance values as well as other color channels of color images. These halftone threshold levels compare the multilevel pixel value to a threshold and at that location set the halftone pixel to zero or one depending on whether the multilevel pixel values are less than or greater than the threshold, and thus 'on' the halftone pixel. Or later to convert the multilevel pixel value to an 'off' binary state.

For each discrete gray value between 0 and 255, for example, the method may include a threshold level with a tone density at which the resulting halftone image achieves the desired gray value when applying multi-level pixel values within the watermark image block. Select.

If the same watermark signal is embedded in several halftone images, the thresholds set in this procedure may continue to be used to generate watermarked halftone images as the same embedded signal. Here is how the procedure for creating a halftone image works:

1. Upsample or otherwise convert a multilevel image per target host pixel to halftone dot resolution. It also converts the upsample or otherwise per-pixel multilevel watermark block to halftone dot resolution and, if necessary, blocks adjacent blocks across the target image so that the watermark signal is coextensive over the target image. Tile with.

2. For each multilevel pixel value in the upsampled image, find the corresponding halftone threshold level corresponding to this level.

3. Apply a threshold to the corresponding per-pixel multilevel value in the watermark signal to take a halftone dot value of zero or one. When the halftone conversion procedure of steps 2 and 3 is completed, the result is a watermarked halftone image that can be printed on paper or other objects.

Each halftone dot in the watermarked halftone image is ink (minimum brightness) or non-ink (maximum brightness). Thus, a high luminance value in the target host image sets a threshold to make it easier for corresponding pixels in the watermark image to be set to a 'non-ink' state. Conversely, the low luminance value in the target image sets the threshold such that the corresponding pixel in the watermark image is more easily set to an 'ink present' state.

This procedure of converting an image into a watermarked halftone image can be applied to one or more color planes in the color halftone image.

If the target host image is a simple gray level image with only a few different gray levels, the above technique will be used to set thresholds for each of the gray levels in the host image. Also, while a host image may have several gray levels, these gray levels may be divided into ranges in which each range is assigned a single halftone threshold. Masks may then be calculated to define regions of the host image corresponding to each of the halftone thresholds. Finally, the halftone threshold corresponding to the particular mask is applied to the watermark signal regions covered by the mask to produce a watermarked halftone image.

Consider the following example where the host image has three gray levels. Where each has a corresponding mask defining the areas of the image in which these gray levels reside. The method now thresholds a multilevel watermark signal per pixel that spans the same time as the target image based on three different masks that correspond to regions of a particular tonal density within the target image:

Mask 1 has a tonal density D1 and uses a threshold T1.

Mask 2 has tone density D2 and uses threshold T2.

Mask 3 has a tone density D3 and uses a threshold T3.

There are many ways to generate digital watermark signals with this technology. One way:

1. Take the desired watermark message payload containing an N bit binary string.

2. Perform error correction encoding of the message, such as convolutional coding, turbo coding, and the like.

3. Spread each bit of the error correction encoded message onto the pseudo random number carrier signal, for example by taking an XOR of the bit value with each value within the pseudo random number carrier.

4. Map spread signal values to pixel locations within the watermark image block.

5. Convert the spread signal to a multilevel watermark signal per pixel. In this step, the spread signal values are for example binary of {1,0}, or {1, -1}, indicating an increase or decrease in corresponding sample values (e.g. luminance, gray level, intensity, etc.). to be. Each binary value corresponds to one or more neighboring pixels in the watermark signal. These binary values may be converted to multilevel values by up or down adjusting a multilevel pixel value (eg, 128 mid-level gray values of an 8-bit pixel) corresponding to the value of the spread signal. The gain of the watermark signal can be adjusted by applying the scale factor to the spread signal. Also, changes from each bit of the spread signal can be made to change smoothly onto corresponding pixels in the neighborhood.

There are a variety of techniques for generating spreading signals, such as convolving carriers and messages, multiplying binary antipodal carriers and binary antipodal message signals, and the like. The carrier can be used to synchronize the detector with the watermark in the watermarked signal. In addition, as described in detail above, a direction signal can be added to the spread signal to aid calibration.

To recover a watermark message, the watermark detector operates as follows:

1. Capture a digital version of a halftone image (eg from a scanner capture of a printed halftone image or from a web cam).

2. Detect and align images using any of the techniques previously described.

3. expect a watermark signal at each pixel in the image aligned from neighboring pixels; For example, using a filter to de-correlate the host signal from the watermark signal.

4. Error correction Correlate the carrier and watermark signals to recover encoded bit values. These may be "soft" bit values weighted by the carrier and the extension of correlation. In other words, rather than representing each estimate of an error correction encoded bit as hard 1 or 0, they are represented by some weighted likelihood between values corresponding to binary 1 or 0 (eg, -1 and 1, -128 and 128, etc.)

5. Perform error correction decoding on soft bit values using a turbo coding scheme or convolution compatible with the inserted message, for example to recover the message payload.

The procedure for reading the watermark can also be used to detect whether the watermarked halftone image has changed. In particular, it can be used to determine whether a printed halftone image has been copied, for example photocopy, or scanned and re-printed.

The detected intensity of the watermark signal can be compared with a copy detection threshold to determine if the printed object has been copied. One approach to measuring the strength of a watermark signal is as follows:

1. Use the message payload read from the watermark to regenerate the originally inserted bit sequence used for the watermark (including the over-encoded bits from the error correction coding).

2. Switch the original bit sequence so that 0 (zero) is represented by -1 and 1 (one) is represented by 1.

3. Multiply (element-wise) the soft-value bit sequence used to decode the watermark by the sequence of step 2.

4. Generate one or more measurements of the watermark intensity from the sequence resulting from the previous step. One such measure is the sum of squares of the values in the sequence. Another measure is the square of the sum of the values in the sequence. Other measurements are likewise possible. For example, soft bits associated with the high frequency components of the watermark signal can be analyzed to obtain a strong measurement due to the high frequency components. Such high frequencies are more susceptible to degradation due to photocopy, digital-to-analog and analog-to-digital conversion, scanning and re-copying, and the like.

5. Compare the intensity measurements with the thresholds to determine if the suspicious image was captured from a copy of the original or printed object. The threshold is based on the diversity of the copiers, scanners and printers used to produce the copy and the difference in the measured watermark intensity of the original printed object to the copied object on the subject printer platform used to create the original. It is derived by evaluation.

As an additional method of detecting copying, a watermark message payload can be used to identify the printer model and / or the resolution of the watermarked halftone image. Then, to verify that the printed object is genuine, the watermark detector extracts the message payload and determines the printer model and / or the printed resolution. A forensic scan of the printed object is then analyzed to verify that the object is printed at the resolution specified in the watermark payload. One particular way of analyzing the resolution of a typical scan is to examine its frequency content to determine if it is easy to print at a specified resolution.

The technique described above of measuring symbol errors to detect signal tampering can be applied to two or more different watermarks embedded at different spatial resolutions within the host media signal. Each of the watermarks may have the same or different message payloads. In the first case where the watermarks have the same message payloads, the message extracted from one of the watermarks can be used to measure bit errors in each of the other watermarks. For example, a message payload from a strong watermark inserted at low spatial resolution may be used to measure bit errors from a less robust watermark at higher spatial resolution. If the watermarks carry different message payloads, error correction bits, such as CRC bits, can be used within each message payload to ensure that the message has been correctly decoded before regenerating the originally inserted bit sequence.

Using two or more watermarks allows the threshold to be set based on the ratio of signal strengths of the watermarks relative to each other. In particular, the signal strength of the first watermark at high resolution (600-1200 dpi) is divided by the signal strength of the second watermark at lower resolution (75-100 dpi). In each case, the signal strength is measured using the measurement of symbol errors or some other measurement (eg a correlation measurement).

If the measured intensity exceeds the threshold, the detector assumes that the watermark signal is original and generates an authentication signal. This signal can be a simple binary value that indicates whether the object is real or a more complex image signal that indicates that bit errors have been detected in the scanned image.

The watermark and host signal can be tiled in particular to detect copying by printing / re-scanning and photocopying of the printed object. This involves inserting a watermark as selected scanning structures with specific spatial frequencies / resolutions that are prone to generating message symbol errors when the object is printed again. This detection procedure ensures that this enables automatic authentication and can be used with low quality camera devices such as web cameras and common image scanners, and that the watermark carries a message payload useful for a variety of applications. It has additional advantages in that it permits to service functions that determine.

The message payload may include an identifier or index in a database that stores information about the destination or link in a network resource (eg, a web page on the Internet). The payload may also include a hidden tracking identifier associated with a particular genuine item, batch of items, printer, or dispenser. This allows a genuine or fake object printed without authorization to be detected and tracked to a particular source, such as its printer, dispenser or batch number.

The payload may also carry printer type information or printer characteristics that enable the watermark reader to adapt its detection routines to the printer types that have occurred. For example, the payload may carry an identifier that specifies the type of halftoning used to generate the properties of the real image, in particular halftone screen. With this information, the reader can check authenticity by determining if the characteristics associated with the halftone screen are present in the printed object. Similarly, the reader can check the halftone screen properties indicating that another halftone screen procedure was used (e.g., forgery was generated using another halftone screen). One particular example is a payload that identifies a halftone screen type and a paper type. The reader extracts this payload from the robust watermark payload and then analyzes the halftone screen and paper attributes to see if they match the halftone type and paper type indicated in the watermark payload. For example, the halftone type may specify the type of halftone screen patterns used to produce the real image. If this halftone screen pattern is not detected (e.g. by the absence of a watermark inserted using the very particular type of halftone screen), the image is considered forged.

A related approach to analyzing halftone types is to find halftone properties such as tell-tale signals of stochastic halftone screens versus ordered dither matrix type screens. Dither matrix screens used in low end printers tend to produce evidence patterns such as patterns of peaks in the Fourier domain that differentiate between statistical screens such as error diffusion procedures that do not generate such evidence peaks and halftone procedures. . If the reader finds peaks where nothing is expected, the image is considered fake. Likewise, if the reader does not find the peaks where the peaks are expected, the image is also considered counterfeit. Before performing such an analysis, it is better to use the embedded digital watermark to realign the image in its original direction at the time of printing. The attributes attributed to the halftone screen can then be evaluated within the appropriate spatial frame of the reference. For example, if the originally aligned dither matrix printer produced an array of peaks in the Fourier domain, the peak positions can be checked more accurately after the image is rearranged.

conclusion

Having described and illustrated the principles of the technique with reference to specific implementations, it will be appreciated that the technique may be implemented in many other different forms. In order to provide an easy-to-understand disclosure without excessively lengthening the specification, Applicants incorporate the above referenced patents and patent applications by reference.

The methods, procedures, and systems described above may be implemented in hardware, software, or a combination of hardware and software. For example, auxiliary data encoding procedures may be implemented in a programmable computer or dedicated digital circuit. Similarly, auxiliary data decoding may be implemented in software, firmware, hardware, or a combination of software, firmware, hardware. The methods and procedures described above may be implemented as programs executed from a system's memory (a computer readable medium such as an electronic, optical or magnetic storage device).

Certain combinations of elements and features in the above detailed embodiments are exemplary only; Replacement and replacement of these and other teachings in patents / applications incorporated herein and by reference are also contemplated.

Claims (49)

In the method of halftone image watermarking, Assigning a set of halftone watermark values to locations in the halftone image; Diffusing an error related to the halftone watermark values to adjacent locations in the halftone image, wherein the error includes the multilevel pixel value at the location of the halftone watermark dot and the watermark dot; A method of halftone image watermarking, characterized as a difference between halftone watermark values. The method of claim 1, Halftone image water, wherein an error related to the halftone watermark values is spread in one direction of the image, and an error related to converting multilevel pixel values in the image into a halftone image is spread in another direction. Method of marking. The method of claim 1, Wherein the positions and values of a halftone watermark are specified by a key. The method of claim 3, wherein And the key is carried in a separate watermark embedded in the image. The method of claim 3, wherein And the key is used to seed pseudorandom processing that specifies the values and positions of the halftone watermark. The method of claim 1, Inserting a watermark into the multilevel pixel representation of the image prior to converting the image into the halftone image. The method of claim 6, Wherein the watermark embedded in the multilevel pixel representation of the image comprises a watermark direction signal used to detect the watermark in the image after experiencing geometric distortion. The method of claim 6, And wherein the watermark in the multilevel pixel representation carries a message comprising keyed positions of the halftone watermark. A computer readable medium having stored software for carrying out the method of claim 1. A method of decoding a halftone watermark from an image, the method comprising: Using a key to identify positions and values of the halftone watermark; Analyzing pixel values at the locations to determine whether the pixel values at the locations correspond to the values specified by the key. The method of claim 10, Scanning the printed version of the image at a suitable high resolution to read halftone watermark dots embedded in the printed version of the image; And then performing the operations of claim 10, wherein the halftone watermark is decoded from the image. The method of claim 10, Deriving the key from the data embedded in the image. The method of claim 12, And the key is derived from a watermark embedded in the image. The method of claim 13, And the watermark from which the key is derived is embedded with the watermark to survive printing and scanning. The method of claim 13, Detecting a direction signal and using the direction signal in the determined direction parameters of the watermark; Using the direction parameters to read a message embedded in the watermark, wherein the message comprises the key. A computer readable medium having software for performing the method of claim 10. In the method of embedding a watermark in a halftone image, Calculating a watermark image comprising an array of values corresponding to pixel locations in the halftone image; Embedding the watermark image in the halftone image by using the values of the watermark image to modulate thresholds at the pixel locations, the thresholds being multilevel in a multilevel image per pixel. A method of decoding a halftone watermark from an image, used to convert pixel values into halftone pixel values of the halftone image. The method of claim 17, Wherein the watermark image comprises a directional watermark signal that causes a watermark decoder to compensate for rotation, scale, and conversion. The method of claim 17, And the watermark image comprises a watermark message signal carrying a message of two or more symbols. A computer readable medium having stored software for performing the method of claim 17. In the method of embedding a watermark in an image, Calculating a watermark image comprising an array of values corresponding to pixel positions in the target halftone image at a resolution; Combining the array values of a watermark image and corresponding multilevel pixel values of a multilevel image per pixel to produce a multilevel image per pixel watermarked at the resolution of the target halftone image; Performing halftone processing to convert the multilevel image per watermarked pixel into a watermarked halftone image. The method of claim 21, And a combining step includes combining the array values of the watermark image and the corresponding pixel values of the multilevel image per pixel. The method of claim 21, The step of performing the halftone process includes performing an error diffusion process on the multilevel image per watermarked pixel, and embedding a watermark in the halftone image. The method of claim 21, And performing an ordered dithering process on the multilevel image per watermarked pixel. The method of claim 21, And the watermark image comprises a directional watermark signal that causes a watermark decoder to compensate for rotation, scale, and conversion. The method of claim 21, And the watermark image comprises a watermark message signal carrying two or more symbols. The computer readable medium having stored thereon software for performing the method of claim 21. In the watermark decoder, A watermark detector for analyzing portions of an image to detect a watermark signal embedded in an image, wherein the image is scanned from a halftone print image at a sufficiently high resolution to identify a workermark image inserted at the resolution of the halftone image. The watermark detector; And a watermark reader for reading a watermark signal from said portions of said image. The method of claim 28, The decoder is operable to use a key to identify positions and values of a halftone watermark, and to determine whether the pixel values at the positions correspond to the values specified by the key. And detect a change in the image by analyzing pixel values at positions. The method of claim 29, The key is decoded from a first watermark in the image and the halftone watermark is a second watermark in the image. The method of claim 28, The watermark signal is calculated by calculating a watermark image comprising array values corresponding to pixel positions in a halftone image and using the values of the watermark image to modulate thresholds at the pixel positions. And encoded by inserting the watermark image into a tone image, wherein the thresholds are used to convert multilevel pixel values in a per-pixel multilevel image into halftone pixel values of the halftone image. The method of claim 31, wherein The watermark decoder is operable to detect a change in the image by analyzing the watermark signal. Decoder. The method of claim 28, The watermark signal is used to calculate a watermark image comprising array values corresponding to pixel positions in a target halftone image at a resolution, and to produce a multilevel image per pixel watermarked at the resolution of the target halftone image. Combining the array values of the watermark image and corresponding multilevel pixel values of a multilevel image per pixel, and performing halftone processing to convert the watermarked multilevel image into a watermarked halftone image Watermark decoder. The method of claim 33, The watermark decoder is operable to detect a change in the image by analyzing the watermark signal. An image processor for determining geometric distortion of an image; The processor, A halftone screen threshold analyzer for producing a target direction signal by applying a halftone screen threshold mask to image data from the received image; A correlation operator for correlating the received image with the target direction signal in a correlation domain, the correlation operator including the correlation operator to generate one or more direction parameters that estimate the geometric distortion of the image Image processor to determine the geometric distortion of the image. The method of claim 35, wherein And the orientation parameter is used to reorder the image to facilitate decoding of the watermark from the reordered image. The method of claim 36, And the watermark carries a message of one or more symbols. The method of claim 36, And the watermark is analyzed to detect a change in the image. The method of claim 1, Values and positions of the halftone watermarks are selected using a human visual system model. A method of embedding a digital watermark in a halftone image, Extra encoding a multi-bit message; Converting the encoded message into a multilevel watermark image per pixel; Deriving halftone thresholds from the per-pixel multilevel watermark image; Converting the multilevel target image per pixel into a watermarked halftone image, Use the multilevel pixels in the target image to select corresponding halftone thresholds from the halftone thresholds derived from the watermark image, and to create the watermarked halftone image of the target image Converting the multilevel target image per pixel into a watermarked halftone image by applying the selected thresholds to corresponding multilevel pixels in the watermark image. How to insert in. The method of claim 40, And wherein the multi-bit message includes information about a printer to be used to print the watermarked halftone image. 42. The method of claim 41 wherein And the information about the printer includes a resolution of the printer. 42. The method of claim 41 wherein And the information about the printer includes information identifying a model of the printer. The method of claim 40, Varying the encoded message comprises modulating the message with a pseudo random carrier. The method of claim 40, Deriving the halftone thresholds creates a histogram of the watermark image, from which the halftone image is associated with the halftone threshold when a halftone threshold is applied to the watermarked image. Creating relevant multilevel pixel values with halftone thresholds to be made to a tone density corresponding to the multilevel pixel value. In the method of measuring the digital watermark intensity, Processing the watermarked signal to extract estimates of error correction encoded bits inserted into the watermarked signal; Decoding the error correction encoded bits to calculate a message payload; Re-encoding the message payload to calculate error correction encoded bits; Calculating a magnitude of a watermark intensity from the estimates of error correction encoded bits and error correction encoded bits. The method of claim 46, And the magnitude of the watermark intensity is compared with an intensity threshold to detect degradation of the watermark signal. The method of claim 47, Wherein the watermarked signal is a captured image of a printed object and the intensity threshold is used to determine whether the printed object is an original or a copy. The method of claim 46, The message payload includes printer information for a printer used to print the watermarked signal, wherein the printer information is used to determine whether a printed object having the watermarked signal is an original or a copy. The method of measuring digital watermark intensity.