JP2008178081A - Method and system using digital watermark - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To improve security and to prevent copying by applying digital watermarking to banknotes and other security documents. <P>SOLUTION: Machine-readable data are digital watermarked into banknotes 16. Such watermarking is optically sensed using an optically scanner 13 and is detected by various devices 11. In response, such various devices 11 intervene to prevent banknotes from being copied. This arrangement addresses various problems, e.g., the use of digital image editing tools to circumvent other banknote anti-copy systems. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

Field of Invention

  This application relates to the use of digital watermarks in banknotes and other securities.

Background and Summary of the Invention

  The problem of accidental counterfeiting of banknotes first occurred 20 years ago with the introduction of color photocopy. A number of techniques have been proposed to address this issue.

  US Pat. No. 5,659,628 (assigned to Ricoh) is one of several patents showing that a photocopier can be equipped to recognize banknotes and prevent photocopying of banknotes. It is. The Ricoh patent specifically suggests that the red indicia printed on Japanese yen bills is a suitable pattern for machine recognition. US Pat. Nos. 5,845,008 (assigned to OMRON) and US Pat. Nos. 5,724,154 and 5,731,880 (both assigned to Canon) sense the presence of a seal emblem in the banknote and respond accordingly. Shows other photocopiers to disable. U.S. Pat. No. 5,678,155 (assigned to Sharp) describes a large number of other features in banknotes such as logos, stamps, seals, face values, symbols including $ and yen, etc. It can be used as a basis for as well.

  Other techniques have proposed to prevent counterfeiting by uniquely marking the printed output from each color photocopier so that the copy can be traced back to the originating machine. For example, US Pat. No. 5,568,268 discloses adding an essentially undetectable pattern of yellow dots to the printed output, making this pattern machine specific. U.S. Pat. No. 5,557,742 discloses a related arrangement in which the photocopy machine serial number is printed on the output document, again in an essentially undetectable form (small yellow letters). is doing. U.S. Pat. No. 5,661,574 shows an apparatus for representing the bits constituting the serial number of a photocopier in the printed output of the photocopier. Here, the bit is a pixel value at a known location (for example, a yellow pixel) depending on whether the bit of the corresponding serial number is “1” or “0” (for example, +/− 30) represented by increasing or decreasing.

  Recent advances in personal computers, color scanning, and printing technology have greatly increased the level of accidental counterfeiting. High quality scanners are now readily available to many computer users, 300 dpi scanners are available for less than $ 100, and 600 dpi scanners are available for a little more. Similarly, photographic quality color jet printers are generally available from Hewlett-Packard Company, Epson, etc. for less than $ 300.

  These tools pose new threats. For example, banknotes can be tampered with (e.g., by whiteout, scissors or resclad techniques) to remove / extinguish the visible patterns on which prior art banknote detection techniques rely to prevent counterfeiting. As a result, a document altered in this way can be freely scanned or copied even in a photocopier designed to prevent processing of banknote images. The removed pattern can then be added back by, for example, a digital image editing tool to allow free duplication of the banknote.

  Digital watermarking technology provides a new way of dealing with the problem of counterfeiting, overcoming many of the newly emerging threats and bringing other benefits that were not previously available.

  Digital watermarking (sometimes referred to simply as “watermarking”) is a rapidly growing field of research through several different approaches. The applicant's research has been carried out in U.S. Pat. Nos. 5,841,978, 5,768,426, 5,748,783, 5,748,763, 5,745,604, 5,710,834, 5,636,292 and 5,721,788, and International Publication Pamphlet WO95 / 14289. , WO 96/36163 and WO 97/43736.

  Applicant's International Publication Pamphlet WO 95/14289 discloses a system for redundantly embedding a plurality of bit codes in various electronic or physical media using a noise-like signal. It shows that you can. In one embodiment, encoding is performed by changing the micro topology of the medium surface. Another embodiment is a scanner that embeds an identification code in the scanned output data.

  Applicant's International Publication WO 96/36163 discloses a reproduction kiosk for use in a retail photo store that allows a consumer to reproduce a photo. This kiosk examines consumer photos with embedded watermark data (for example, indicating that the photo is copyrighted) and copies such photos if such watermark data is detected. Stop. This application encodes information in vector graphics and very low-order bitmap graphics by slightly changing the outline of the line, for example, slightly up, down, left, or right. It is also disclosed that it can be realized.

  Other digital watermarking studies are described in U.S. Pat. Nos. 5,734,752, 5,646,997, 5,659,726, 5,664,018, 5,671,277, 5,687,191, 5,687,236, 5,687,587, 5,568,570, 5,572,247. No. 5574962, No. 5579124, No. 5581500, No. 5613004, No. 5629770, No. 5461426, No. 574363, No. 5488664, No. 5530759, No. 5539735, No. 49439973, No. 5337361, 5,404,160, 5,404,377, 5,31,098, 5,319,735, 5,337,362, 4,972,471, 5,161,210, 5,243,423, 5,091,966, No. 113437, No. 4939515, No. 5374976, No. 4855827, No. 4876617, No. 4996315, No. 4963998, No. 4996941, International Publication Pamphlet WO98 / 02864, European Patent Application Publication No. 822550, International Publication Pamphlet WO 97/39410, British Patent Publication No. 2196167, European Patent Application Publication No. 777197, European Patent Application Publication No. 734860, European Patent Application Publication No. 705025, European Patent Application Publication No. No. 766468, European Patent Application Publication No. 783222, International Publication Pamphlet WO95 / 20291, WO96 / 26494, WO96 / 42151, WO97 / 22206, WO97 / 26733 . Some of the above patents relate to visible watermark processing techniques. Other visible watermarking techniques (eg, data glyphs) are described in US Pat. Nos. 5,706,364, 5,689,620, 5,684,885, 5,680,223, 5,668,636, 5,640,647, 5,594,809. . (More typically, digital watermarks are steganographic, i.e. they are essentially insensitive to human vision and help hide the embedded data in front of the viewer. )

  However, the majority of watermarking research is not in the patent literature, but rather in published research. In addition to the patentee of the above patent, as some of the other researchers in this field (documents related to watermarks can be found by author search in the INSPEC database) Pitas, Eckhard Koch, Jian Zhao, Norishige Morimoto, Laourence Boney, Kineo Matsui, A. Z. Tiekel, Fred Mintzer, B.M. Macq, Ahmed H.M. Mention may be made of Teffi, Frederic Jordan, Naohisa Komatsu and Lawrence O'Gorman.

  In A Signal Theoretic Method for Creating Forgery-Proof Documents for Automatic Verification by Szepanski at the 1979 Carnahan Conference on Crime Countermeasures, the author Overprinting a document (eg, banknote) using a simple line pattern is a deterrent against counterfeiting, but does not convey meaningful information that can be used in an automated verification procedure. Therefore, he proposes instead to superimpose a pattern representing multiple bit data on the document. The document is divided into as many square areas as data bits to be represented and a carrier pattern is selected. In the region corresponding to the “1” bit, the carrier pattern is added to the underlying document sample, and in the region corresponding to the “0” bit, the pattern is reduced. Then, the document can be scanned using the automatic verification procedure, the correlation between the scanned data and the carrier pattern can be obtained, and the digital data can be extracted. In particular, the author contemplates a passport in which the passport photo is processed in this way to convey the owner's name and date of birth.

  Artisans are assumed to be familiar with the prior art.

  In this disclosure, it should be understood that reference to a watermark not only includes the assignee's watermark technique, but can be similarly performed using any other watermark technique such as those described above.

  Watermarks can be applied to countless forms of information. This disclosure focuses on bills, traveler's checks, passports, registered stock certificates, etc. (hereinafter collectively referred to as “security documents”), which are generally characterized by the use of line images. To do. However, it should be recognized that the principles discussed below can be used outside this particular domain.

  Most of the prior art in image watermarking is focused on images composed of pixels (eg, bitmapped images, JPEG / MPEG images, VGA / SVGA display devices, etc.). In most watermarking techniques, sub-liminal encoding of binary data through an image is performed with slight changes in luminance or color values of constituent pixels. (This encoding can be done directly in other regions, such as the pixel region or the DCT region.) In some systems, isolated pixels are changed according to one or more bits of binary data, and other In the system, multiple populations associated with a region of pixels (eg, locally adjacent or corresponding to a given DCT component) are thus changed. However, in all cases, the pixel functions as the final carrier for the embedded data.

  Pixel images are a relatively recent development, but line art dates back centuries. One well-known example is US banknotes. For example, in a dollar bill, line drawings are used in several different ways. One is to form a grid pattern that surrounds the edge of a bill (typically consisting of bright lines on a dark background). Others form grayscale images, such as the portrait of George Washington (typically consisting of dark lines on a light background).

  There are two basic ways to simulate grayscale in line drawings. One changes the relative spacing of the lines to lighten or darken the image area. FIG. 1A shows such an arrangement, with region B appearing darker than region A due to the closer spacing of the constituent lines. Another technique is to change the width of the constituent lines, with wider lines resulting in darker areas and narrower lines resulting in brighter areas. FIG. 1B shows such an arrangement. Again, region B appears darker than region A due to the wider constituent lines. These techniques are often used together.

  In accordance with one aspect of the present invention, the security of a bill (or other security document) uses digital data that is mechanically readable and generally undetectable to slightly change the distribution of ink on the surface of the bill. By marking the surface, it is improved. Digital data can have multiple bits and can be redundantly encoded across the banknote (rather than marking the banknote only in a single localized area). The encoding can use a code signal or a discrete cosine transform. In some embodiments, two watermarks can be used advantageously, in which case the two watermarks are encoded with different robustness (robustness) or according to different code signals. be able to. Some such banknotes may be further provided with holograms to further improve security.

  According to another aspect of the invention, a bill (or other security document) is encoded with multi-bit digital data to facilitate subsequent machine identification. This can be accomplished by receiving the initial banknote block with multi-bit digital data. Next, a pattern of data is generated and the digital data is encoded in this pattern. Next, the initial composition is adjusted according to this data, and the banknote is printed using the adjusted composition. Again, this arrangement desirably spreads multiple bits spatially across the bill.

  One such embodiment places a virtual grid of points superimposed on the line drawing image. These points are spaced apart in the vertical and horizontal directions. (The horizontal and vertical intervals need not be equal.) Virtual points may be overlapped at equal vertical and horizontal intervals of 250 μm in some or all of the bill. In the area of a banknote having a line drawing, the constituent lines of the line drawing meander in and between these virtual grid points.

  Consider each grid point as the center of the corresponding region. The brightness of a region is a function of the proximity of the line to the center point of any region within the region boundary and the line thickness.

  In order to change the brightness of the region, the outline of the line is slightly changed within the region. In particular, the line is slightly thickened to reduce the brightness, or thinner to increase the brightness. (In this example, dark lines on a light background are assumed.) Using these slight changes, binary data is encoded in line drawings according to known pixelated-based watermarking techniques. When such a bill is subsequently scanned by a scanner, the value of the pixel data generated by the scanner reflects the change in the luminance value described above, and the embedded watermark data can be decoded.

  In another embodiment, the line thickness is not changed. Instead, the position of the line is shifted slightly towards or opposite the central virtual grid point, causing an increase or decrease in the brightness of the corresponding region with the same effect.

  According to another aspect of the present invention, duplication of banknotes (or other security documents) is prevented by encoding the banknotes with multi-bit digital data. Thereafter, sampled image data corresponding to such bills is provided and analyzed to detect the presence of digital data in the image data. If such data is detected, take appropriate intervention.

  In one such embodiment, the sampled image data is analyzed to detect the presence of visible structural features on the banknote. Intervention actions can be triggered in response to the detection of visible structure and / or digital data.

  Interventions can include interfering with image data duplication, sending a message to a remote location to report processing of banknote data, and inserting legal tracking data into the image data.

  According to another aspect of the invention, a copy of a bill (or other security document) is examined for suspicious image data corresponding to the sampled optical brightness of the object, and whether the suspicious image data corresponds to a bill. Prevent by determining what. Both of these tasks are performed at the first site. If the suspicious image data corresponds to a banknote, the second remote site is contacted and a report regarding the detection of banknote related data is sent.

  In accordance with another aspect of the invention, the output from the printer of the personal computer system is marked to allow later identification. This method involves receiving (or intercepting) printer data sent from a personal computer to a printer, and digitally watermarking this data (which is desirably spread through the printer data). And watermarking. This watermark acts to mark the printer data as being printed by a particular printer and allows the resulting document to be matched to this printer.

  According to another aspect of the invention, an apparatus for processing (eg, recognizing or confirming) a banknote (or other security document) includes a scanner that generates image data corresponding to document input to the apparatus, And a processor for detecting the multi-bit digital watermark data. Detection of such watermark data indicates that the document is a banknote. If watermark data is detected, the control unit responds in a first way to alert the remote site or service to the operation of the device (e.g., stop replication, insert legal tracking information into the output data). Change or restrict). If no watermark is detected, the control unit responds in a second different way. Scanners, photocopiers, and printers are examples of devices that can use banknote recognition devices such as these.

  According to another aspect of the present invention, a cash processing system includes an input unit that receives a plurality of banknotes, and a feed mechanism that transports banknotes from the input unit through an optical detector. The optical detector generates digital image data corresponding to the bills transported. The image data is then processed by the processor, which detects multi-bit digital watermark data encoded steganographically into a line drawing in at least some portion of the banknote. The control unit responds in a manner that depends on the detected watermark data.

  The foregoing and other features and advantages of the present invention will become more readily apparent from the following detailed description, which proceeds with reference to the accompanying figures.

Detailed description

  Referring to FIG. 2, an exemplary embodiment of the present invention uses a grid 10 of virtual reference points arranged on a line drawing image. The spacing between the points is 250 μm in the exemplary arrangement, but wider or narrower spacing can of course be used.

  Each grid point is associated with a surrounding area 12 shown in FIG. As will be described later, the brightness (or reflectance) of each of these regions 12 is slightly changed to perform subliminal encoding of binary data.

  Region 12 can take a variety of shapes, and the rounded rectangular shape shown is merely a sample. (The shape shown has the advantage of being able to surround a significantly larger area with less visual artifacts than a square area.) In other embodiments, squares, rectangles, circles, ellipses Shapes can be used instead.

  4 is an enlarged view of a portion of FIG. 3, showing a line 14 passing through a grid of dots. The thickness of the line will, of course, depend on the particular image that the line is part of. The lines shown are about 25 μm wide, and of course thicker or thinner thicknesses can be used.

  In the first embodiment of the present invention shown in FIG. 5, the line width can be changed and controlled so as to change the brightness of the region through which the line passes. In order to increase the brightness (reflectance), the line is made thinner (ie, there is less ink in that region). To reduce brightness, the lines are made thicker (ie, more ink).

  Whether the brightness of a given area should be increased or decreased depends on the particular watermarking algorithm used. Any algorithm can be used by changing the brightness of the region 12 to change the brightness or color of the pixels in the pixelated image in other ways. (Some watermarking algorithms make these changes in transformed regions such as DCT, Weblet, or Fourier. However, these changes eventually appear as changes in brightness or color. )

  In one example algorithm, binary data is represented as a sequence of -1 and 1 instead of 0 and 1. (Binary data may have one data, but more typically has some data. In the illustrated embodiment, the data has 100 bits, some of which are error corrected or detected. Bit.)

  Each element in the binary data sequence is then multiplied by a corresponding element of a pseudorandom sequence consisting of -1 and 1 to generate an intermediate data signal. Each element of this intermediate data signal corresponds to an individual subsection (sub-part) of the image, such as region 12. Collectively, these elements form a pattern of binary data. (Normally, each element of the intermediate data signal is mapped to some of the regions.)

  The image in each such region (and optionally surrounding it) is analyzed to determine the relative ability to hide the embedded data and generate a corresponding scale factor. An exemplary scale factor may vary in the range of 0-3. The scale factor for that region is then multiplied by an element of the intermediate data signal corresponding to that region (ie, mapped) to generate a tweak (ie, bias). In the illustrated case, the resulting tweak is in the range of -3 to 3. Collectively, these elements form a pattern of gray scaled data.

  Next, the brightness of the area in the banknote is adjusted according to the corresponding tweak value. A -3 tweak value corresponds to a -5% change in brightness, -2 corresponds to a -2% change, -1 corresponds to a -1% change, 0 corresponds to no change, 1 Can correspond to a + 1% change, 2 can correspond to a + 2% change, and 3 can correspond to a + 5% change. (This example follows the basic technique described in the real-time encoder embodiment disclosed in US Pat. No. 5,710,834.)

  In FIG. 5, the watermarking algorithm has determined that the brightness of region A should be reduced by a certain percentage and the brightness of regions C and D should be increased by a certain percentage.

  In region A, the brightness is lowered by increasing the line thickness. In region D, the brightness is increased by reducing the line thickness, and so on (but to a lesser extent) in C.

  The line does not pass through region B and there is no opportunity to change the brightness of this region. However, this is not fatal in this method. This is because the exemplary watermarking algorithm redundantly encodes each bit of data in a plurality of subsections spread across the line drawing image.

  The change with respect to the line thickness in the areas A and D of FIG. 5 is exaggerated for the sake of explanation. While the illustrated variation is possible, most implementations typically adjust (increase or decrease) the line thickness by 3-50%.

  (Many watermarking operations usually work within a signal margin of about +/- 1% change in brightness and do the encoding. That is, the "noise" added by the encoding is below Only about 1% of the signal, the line usually does not occupy the entire area of the area, and a 10% change in the thickness of the line will only cause a 1% change in the brightness of the area. Banknotes differ from photos in that the artwork generally does not need to convey photorealism (photorealistic depictions), so banknotes are larger than those used in watermarking photos Can be encoded with energy, and the result of the encoding is still aesthetically pleasing, for example a local brightness change on the order of 10% is possible in banknotes, while in a photograph The level watermark energy, such as, in general, in some. Considered that unacceptable situation, localized luminance changes of even 20, 30, 50, or 100% is allowed.)

  In the illustrated embodiment, the change to line thickness is only in response to a tweak to be applied to one region. Therefore, if the line passes through any part of the region to which the 2% tweak should be applied, the thickness of the line in this region will change to produce a 2% luminance difference. In different embodiments, the change in line thickness is a function of the position of the line in the region. Specifically, the change in line thickness is a function of the distance between the center grid point of the region and the closest point of that line to that point. If the line passes through the grid points, a full 2% change occurs. Increasing the distance continuously adds a continuously smaller change. The manner in which the tweak size changes as a function of the position of the line within the region can be determined by using one of various interpolation algorithms such as bilinear, bicubic, cubic spline, custom curve, etc. it can.

  In another different embodiment, the change in line thickness in one region is a tweak weighted function for the adjacent region or the surrounding region. Accordingly, the thickness of a line in a certain region can be increased or decreased according to a tweak value corresponding to one or more adjacent regions.

  Combinations of the above embodiments can also be used.

  In the above-described embodiment, it may be necessary to make a trade-off between the tweak values of adjacent regions. For example, a line may pass along a boundary line between regions and may pass an equidistant point (equal distance zone) from four grid points. In such cases, the line may be given a contradictory tweak value, some areas will need to increase the line thickness, and other areas will need to decrease the line thickness. Sometimes. (Or it may be necessary to increase the thickness of the line on both sides, but it may be a different amount.) Area where the line does not pass through the equidistant zone but the change in line thickness has a different tweak value The same applies to a function in the vicinity of. In this case as well, the weight to be given to the tweaks from each region can be determined using a known interpolation function when determining the change to be made to the line thickness in the predetermined region.

  In the exemplary watermarking algorithm, the average change in brightness in the image is zero, and it is not clear that the image does not lighten or darken the entire image. Local changes in brightness are subtle in quantity and are localized in the proper position, so they are essentially invisible to the human eye (eg inconspicuous / subliminal).

  Another embodiment is shown in FIG. In FIG. 6, not the thickness of the line but the position of the line changes.

  In FIG. 6, the original position of the line is indicated by a dotted line, and the changed position of the line is indicated by a solid line. In order to decrease the brightness of the region, the line is moved slightly closer to the center of the grid point, and in order to increase the brightness of the region, the line is moved slightly away. Therefore, in region A, the line moves toward the center grid point, and in region D, the line moves away.

  Note that the line at the left edge of region A does not return to its normal (dotted) position when exiting the region. This is because the area to the left of area A should also have reduced brightness. If possible, it is generally preferable not to return the lines to their normal position, but instead leave the shifted lines shifted as they enter the adjacent area. Can do. In this way, a larger net line movement within the region is possible and the embedded signal level can be increased.

  Also in this case, the line shift in FIG. 6 is somewhat exaggerated. A more typical line shift is on the order of 3-50 μm.

  One way to consider the embodiment of FIG. 6 is to use magnetic analogy. The grid point at the center of each region can be regarded as a magnet. This will attract or repel the line. For example, a -3 tweak value can correspond to a strong attractive force, a +2 tweak value can correspond to a moderate repulsive force, and so on. In FIG. 6, the grid points in the region A indicate an attractive force (that is, a negative tweak value), and the grid points in the region D indicate a repulsive force (for example, a positive tweak value).

  Magnetic analogy is effective because the magnetic action applied to the line depends on the distance between the line and the grid point. Thus, a line passing near the grid point is shifted in position more than a line near the periphery of the region.

  (In practice, magnetic analogy can help more than conceptual tools. Instead, magnetic effects can be modeled in a computer program to help analyze the desired placement of lines with respect to grid points. .)

  Each of the modifications described above in conjunction with FIG. 5 can be similarly applied to FIG.

  The combination of the embodiments of FIGS. 5 and 6 can of course be used, resulting in increased watermark energy, a better signal-to-noise ratio, and in many cases less noticeable changes. it can.

  In yet another embodiment, the brightness in each region is changed without changing the line. This can be done by interspersing small dots of ink in otherwise empty portions of the area. In high quality printing of the type used for banknotes, droplets on the order of 3 μm in diameter can be placed. (The larger droplets still exceed the perceptual threshold of most of the viewers.) This is in the region (regular array or randomly or according to the desired profile such as normal distribution). By applying a small droplet, a change of about 1% in brightness can be easily given. (Normally, dark dots are added to the area to reduce the brightness. Increasing the brightness creates a bright white space in the line art that lies within the area by scattering the bright ink or otherwise. (In practice, many such microdots are not printed in real manufacturing, but often are printed to some degree statistically.)

  In a variation of this interspersed technique, very thin mesh lines can be inserted into the line drawing to slightly change the brightness of one or more areas (so-called background coloring process).

  In order to decode the watermark data, the encoded line drawing image must be converted into an electronic form for analysis. This conversion is usually performed by a scanner.

  Scanners are well known and will not be described in detail here. To illustrate, scanners traditionally use lines consisting of a plurality of closely spaced photodetector cells that contain light reflected from successive strips of the image. Generate a signal related to the quantity. Most inexpensive consumer scanners have a resolution of 300 dots per inch (dpi) and the center-to-center spacing of the constituent photodetectors is about 84 μm. Higher quality scanners of the type found in most professional image processing devices and photocopies have a resolution of 600 dpi (42 μm), 1200 dpi (21 μm) or higher.

  Taking a 300 dpi scanner (84 μm photodetector spacing) as an example, each 250 μm region 12 in the banknote corresponds roughly to a 3 × 3 array of photodetector specimens. Of course, only in rare cases, an area is physically recorded by the scanner so that the nine photodetector samples acquire the brightness of this area and not the other. More generally, the line art is oblique to the scanner's photodetector or misaligned in the longitudinal direction (ie, some photodetectors image a subsection of two adjacent regions. However, since the scanner oversamples the area, the brightness of each area can be clearly determined.

  In one embodiment, scan data from a line drawing is collected in a two-dimensional array and processed according to one of the applicant's previous patents and the techniques disclosed in the application to obtain data statistics (the applicant's previous Analyze (using techniques disclosed in the document) and extract the embedded bits of data.

  (To reiterate, the applicant's reference to previous watermark decoding techniques is by way of example only. Once the scan is started and the data is available in pixel format, any other watermark decoding technique can be used. Thus, it is easy to extract the correspondingly encoded watermark.)

  In different embodiments, scan data is not collected in a complete array prior to processing. Instead, scan data is processed in real time as it is generated in order to detect embedded watermark data without delay. (Depending on the scanner parameters, it may be necessary to scan a line drawing image on the order of half an inch before the statistics of the obtained data clearly indicate the presence of a watermark.)

  According to another aspect of the present invention, various hardware devices have the ability to recognize and appropriately respond to data embedded in any line drawing image processed by these devices.

  An example is a color photocopier. Such an apparatus uses a color scanner to generate sampling (pixel) data corresponding to an input medium (eg, dollar bill). If watermark data associated with a bill is detected, the photocopier can perform one or more steps.

  One option is simply to abort the duplication and display a message reminding the operator that duplicating money is illegal.

  Another option is to connect to a remote service or site and report attempted banknote duplication. Photocopiers with telephone dialing capabilities are known in the art (eg, US Pat. No. 5,305,199) and are readily adapted for this purpose. Similarly, a personal computer with a modem or network communication capability can send a message to a remote location (eg, on the Internet) to alert about an attempted copy. The remote service may be an independent service or a government agency.

  Yet another option is to allow replication, but insert legal tracking data into the generated replication. This tracking data can take various forms. Steganographically encoded binary data is an example, and the data embedding shown in US Pat. No. 5,568,268 can also be used. The tracking data can record the serial number of the machine that made the replica and / or the date and time that the replica was made. In order to address privacy issues, such tracking data is usually not inserted into the photocopied output, but only when it is detected that the photocopied is a banknote. (Such an apparatus is shown in FIG. 7.)

  Preferably, the scan data is analyzed line by line to detect illegal photocopy processing with the shortest delay. When banknotes are being scanned, one or more lines of scanner output may be provided to the photocopier copying unit before a banknote detection decision is made. In this case, the photocopy has two areas: a first area where no tracking mark is given and a second small area where tracking data is inserted.

  Another hardware device that can use the principles described above is a stand-alone scanner. A programmed processor (or dedicated hardware) inside the scanner analyzes the data generated by the device and responds accordingly.

  Yet another hardware device that can use the principles described above is a printer. The processor inside the device analyzes the graphical data to be printed and searches for the watermark associated with the banknote.

  For both scanner and printer devices, the response method may disable operation or insert tracking information. (Such devices typically do not have a network connection for dialing functions, but if necessary, such functions of traditional computers can be invoked by scanners or printer devices using known techniques. Is possible.)

  Again, it is desirable to process the scanner or printer data as it becomes available so that any banknote processing can be detected with the shortest delay. Also in this case, there is a certain delay time before the detection decision is made. Thus, the scanner or printer output consists of two parts: a part without tracking data and another part with tracking data.

  In another embodiment (FIG. 10), a non-perceptible watermark with a universal ID (UID, unique ID) is printed on a document printed by a printer, scanned by a scanner, or copied by a photocopier. Insert all of the. The UID is associated with a particular printer / photocopier / scanner in a registration database maintained by the product manufacturer. The manufacturer can also enter in this database the name of the seller who originally shipped the product. In addition, the owner's name and address can be added to the database when the product is registered for warranty service. While not preventing the use of such machines in counterfeiting, the embedded UID makes it easy to identify the machine that generated the counterfeit banknote. (This is an application where the secret watermark is optimally used.)

  Another useful application of digital watermarking is in reliable banknote authentication, for example, for automated cash machines that both receive and pay cash. Referring to FIG. 8, such a machine (11) comprises a known optical scanner (13) that generates digital data corresponding to the surface of the banknote (16). Next, the image set (14) is analyzed to extract embedded watermark data. In watermarking techniques that require knowledge of the code signal (20) for decryption (eg, noise modulated signal, secret key, spread signal, etc.), the banknote is watermarked according to some such code. May be. Some of these codes openly allow them to be read by traditional machines. Other codes are secret and are reserved for use by government agencies and the like. (See public code and secret code in patents issued to the applicant.)

  As stated, banknotes currently have some visible structure, i.e. markings (e.g., the seals shown in the patents cited above), which are visible (visual inspection or machine detection). Can be used as an aid to bill authentication. Desirably, the banknote is inspected by the integrated detection system (24) and the presence of both the visual structure (22) and the digital watermark data is verified before determining that the banknote is authentic.

  Visible banknote structures can be detected using known pattern recognition techniques. Examples of such techniques include U.S. Pat. Nos. 5,321,773, 5,390,259, 5,533,144, 5,539,841, 5,583,614, 5,633,952, 4,723,149 and 5,424,807, and European Foreign Application Publications. No. 766449 is disclosed.

  Referring to FIG. 9, for example, the photocopier (30) senses the presence of either the visible structure (32) or the embedded banknote watermark data (34) and if any is present (36). , Replication can be disabled. Scanners and printers can be equipped with similar functionality to analyze scanned or to-be-printed data with respect to any of these bill verification stamps. If either is detected, the software (or hardware) disables further operations.

  Verification with banknote watermark data provides an important advantage that is not otherwise available. As stated, it is possible to tamper with the original bill (e.g., by whiteout, scissors, or rescue techniques) to remove visible structures. Such documents can then be freely replicated either in a visible structure sensitive photocopier or scanner / printer device. The removed visible structure can then be added by a second print / photocopy process. If the printer is not equipped with a bill disable feature, use an image editing tool to insert the visible structure back into the image data set scanned from such tampered bills, Can be printed freely. By including additional watermark data for embedding in the banknote and sensing it, such a strategy will not be successful.

  (A similar strategy is to scan banknote images with a banknote-insensitive scanner. The resulting data set can then be edited with traditional image editing tools to remove visible structures. In addition, such a data set can also be printed in a printer / photocopier that checks such data for the presence of visible structures, in which case the missing visible structures are subsequently It can be inserted by printing / photocopying.

  Preferably, the visible structure detector and the watermark detector are integrated as a single hardware and / or software tool. This device can control various economics, for example, interface processing with scanners, manipulation of pixel data sets for pattern recognition and watermark extraction, electronic rearrangement of images to facilitate pattern recognition / watermark detection, control A signal (eg, a disable signal) is generated in the photocopier / scanner, etc.

  While the applications described above have made possible counterfeit operations impossible in response to detecting either visible structure or watermarked data, in other applications both criteria make the banknotes authentic. It must be met before it can be recognized. Such applications typically include, for example, the receipt or acceptance of banknotes by ATM as described above and shown in FIG.

  (In order to provide a comprehensive disclosure without unduly lengthening the following specification, Applicants incorporate by reference to the patent documents cited above.)

  From the foregoing, it will be appreciated that embodiments of the present invention function as line drawing images as subliminal carriers for binary data. In addition, existing deterrent measures against banknote counterfeiting have been extended to prevent normal detours and provide other benefits.

  While the principles of the invention have been described and illustrated with reference to some exemplary embodiments, it should be recognized that these embodiments are illustrative only and should not be taken as limiting the scope of the invention. is there. Obviously, with the techniques described above, other watermarking, decoding and counterfeiting techniques can be substituted for and / or combined with the detailed elements to produce similar effects.

  For example, although the present invention has been described with reference to an embodiment that uses a regular rectangular array with grid points, one skilled in the art could alternatively use other arrays of points that are neither rectangular nor regular. Will recognize.

  Similarly, although the present invention has been specifically described with reference to banknotes, these principles are equally applicable to other security documents.

  Although the present invention has been specifically described with reference to embodiments that scale the embedded energy according to local image characteristics, in other embodiments, manually formed energy profiles can be implemented. That is, it is possible to manually create a mask that defines the size of a signal to be embedded in a different part of an image, and use this to adjust the change in luminance in each region.

  Similarly, although one encoding technique has been specifically described in detail, the present invention is not limited to this. Any technique for hiding multi-bit binary data in a security document can be used as well so that it can be detected from the sampled optical data corresponding to the security document. Not all such techniques with slight changes to the ink (eg color, density, distribution, etc.). For example, another approach is to digitally watermark the underlying media (paper, polymer, etc.). This can be done by changing the fine structure of the medium (small braille) and representing the pattern of the encoded data. Such texture processing can be optically detected and decoded using an algorithm corresponding to the encoding algorithm that generated the patterned data. Another option is to use a laminate material on or in the banknote, in which case the laminate material has a watermark displayed on / in it. The laminate can be textured (as described above), or its optical transparency can be changed according to the watermarked noise-like pattern, or the chemical properties can be changed as well. Is possible. Such an approach is described in detail, for example, in the Applicant's International Publication Pamphlet cited above.

  Although the present invention has been particularly described with reference to several hardware devices (eg, scanners, photocopiers and printers) and several applications (anti-counterfeiting and banknote authentication), the technology is not limited thereto. For example, many different devices may be equipped with the banknote recognition function detailed herein. An example is a cash acceptance payment machine that inspects banknotes for authentication. Another example is a personal computer that examines the data processed thereby to determine if it is of the same origin with respect to the banknote and interferes appropriately.

  Still other applications can be handled by this technique as well. One is banknote processing, for example, an apparatus used by a bank or issuing institution that counts, sorts, reads, and selects counterfeit banknotes that have passed a specific number of years from circulation. Such an apparatus can include an input unit (for example, a hopper) that receives a plurality of banknotes and a feed machine that transports banknotes from the input unit through an optical detector. The optical detector generates digital image data corresponding to the bills transported. The image data is then processed by a processor, which detects multi-bit digital watermark data that is steganographically encoded into the artwork in at least some portion of the banknote. The control unit responds in a manner depending on the detected watermark data. For example, the control unit operates the apparatus to classify banknotes by face value or age based on the decrypted data, or to thin out counterfeit goods.

  Further, the method of the present invention can be performed by hardware, software, or a combination thereof (eg, some image processing operations are performed by dedicated hardware and other operations are performed in software). Will be recognized.

  In still other embodiments, at least a portion of the watermark can be printed using luminescent ink. This allows, for example, a merchant who is shown a banknote to quickly recognize the presence of a certain watermark on the banknote (for example by inspecting under a black light) without resorting to scanner and computer analysis. Is possible. Such a luminescent ink can also print a human-readable mark on a banknote, such as the face value of a banknote. (Inkjet printers and many other common printing technologies use cyan / magenta / yellow / black to form colors, so they can produce only a limited spectrum of colors. Out of these capabilities, it is possible to similarly use luminescent colors such as yellow, pink and green dyes used for highlight markers, with the advantage that they appear without black light.

  An improvement over the encoding techniques detailed above and other encoding techniques is to add an iterative assessment of the robustness of the digital watermark and corresponding adjustments in the re-watermarking operation. In particular, when encoding multi-bit watermarks, some bits may be more robustly (eg, stronger) encoded than others due to the nature of the underlying content (eg, artwork). . In the exemplary technique used for this improvement, the watermark is first encoded. Next, a trial decoding operation is performed. Next, a reliability measure (eg, signal to noise ratio) for each bit decoded in the decoding operation is evaluated. Bits that appear to be weakly encoded are identified and corresponding changes are made to the watermarking parameters to improve the relative strength of these bits. Next, this object is watermarked using the newly changed parameters. This process may be repeated as necessary until all bits comprising the encoded data are approximately equally detectable from the encoded object, or until some predetermined signal to noise ratio threshold is met. it can.

  Banknotes, as well as most other media and objects, generally benefit from the use of multiple watermarks. For example, a banknote can be marked once in the spatial domain and second time in the spatial frequency domain. (It should be recognized that any change in one area has an effect in another area, where we refer to the area where the change directly affects.)

  Another option is to mark the bill (or other physical or digital object) with two different levels of robustness (robustness) or strength watermarks. A more robust watermark can withstand various forms of tampering and can be detected on an object even after multiple intervening distortions have occurred. A watermark that is less robust can be made so weak that it is compromised by a single distortion of the object. For banknotes, for example, a less robust watermark serves as an authentication mark. Any scan and reprint operation makes it unreadable. Both a robust watermark and a weak watermark should be present in a real banknote and are present when the former watermark is counterfeit.

  Yet another option is to encode two different watermarks according to two different code signals. One watermark can identify (eg, by serial number) the device used in the attempted counterfeit operation. The other watermark can be used for other purposes.

  The above considerations have taken various technical measures against various issues and have been described with particular reference to banknotes. An exemplary solution has been detailed. Others will be apparent to those skilled in the art by extrapolating from the solution given above using conventional knowledge.

  For example, the techniques and solutions disclosed herein use elements and techniques from the cited references. Other elements and techniques from the cited references can be simply combined to obtain other embodiments within the scope of the present invention. Thus, for example, holograms with or without watermark data can be used in banknotes for additional security, single-bit watermarking can be a common alternative to multi-bit watermarking, The techniques described using non-perceptible watermarks can sometimes be performed using visible watermarks (eg emoji), with local scaling of the watermark energy without increasing human perceptibility The signal to noise ratio of the watermark can be increased, different filtering operations can be used for different functions, and the encoding process can be performed with a granularity of one pixel (or DCT coefficients) Or a group of adjacent pixels (or DCT coefficients) may be processed in the same way, and the encoding is a predicted form of content tampering It may be optimized to withstand. Etc. etc. etc. (All cited variations are generally detailed in the Applicant's International Publication Pamphlet.) As such, the exemplary embodiment is a solution available by combining the teachings referenced above. Is just a selected sample. Other solutions, although not necessarily exhaustively described herein, are well within the understanding of those skilled in the art given the above disclosure and familiar with the cited art.

<Various miscellaneous things>
(Most of the following description was originally given in a US priority application prior to the above discussion on banknotes. The claims are clarified as they emphasize the use of banknotes. For this reason, the following material has been moved to the end of the specification: Although not specifically described with reference to banknote applications, this material provides useful content that allows a more complete understanding of banknote applications. give.)

  An improvement over existing watermark coding techniques is to add iterative evaluation of mark robustness and corresponding adjustments in re-watermarking operations. In particular, when encoding a multi-bit watermark, some bits can be more robustly (eg, stronger) encoded than others due to the characteristics of the underlying content. In the exemplary technique used for this improvement, the watermark is first embedded in the object. Next, a test decoding operation is performed. A reliability measurement (eg, signal to noise ratio) for each bit decoded in the decoding operation is then evaluated. Identify bits that appear to be weakly encoded and make corresponding changes to the watermarking parameters to increase the relative strength of these bits. Next, this object is watermarked with the newly changed parameters. This process may be repeated as necessary until all bits comprising the encoded data are approximately equally detectable from the encoded object, or until some predetermined signal to noise ratio threshold is met. it can.

  While the above analysis evaluated reliability on a bit-by-bit basis, the iterative procedure involved can evaluate reliability on a part-by-part basis. That is, the encoded object is considered in parts and each part is analyzed with respect to the robustness (robustness) of the data encoded thereby. In the part indicating “weak” encoding, the encoding parameters can be adjusted to enhance the encoding in one or more subsequent re-encoding operations.

  These parts can take different forms, such as rectangular patches in still or moving images, short excerpts in audio or video, DCT / Fourier / Frequency-based representations of objects in the transformed area. Weblet coefficients (or groups of adjacent coefficients) can be taken.

  This technique increases the certainty that even if the encoded object is extracted or filtered (e.g., spectrally) spatially or temporally, the watermark energy remaining after such processing allows for accurate decoding.

  In the illustrated embodiment, this iterative process is highly automated and essentially transparent to the user. The user simply instructs the computer control system to watermark the object, which performs test watermark processing, performs decoding, makes subsequent adjustments, and final encoded object that satisfies the desired watermark quality condition Responds by repeating as necessary until is generated.

  Watermarking can be used for very many types of information. These include images expressed in either digital form (eg, images of pixels, digital video, MP3 / MP4 audio, etc.) or analog representations (eg, unsampled music, printed images, banknotes, etc.) (Including video) and audio. Watermarking can be applied to digital content (eg, images, audio) before or after compression. Watermarking is also used in various “description” or “composite” language representations of content, such as Structured Audio, Csound, NetSound, SNHC Audio, etc. (see http://sound.media.mit.edu/mpeg4/). Can also be used by specifying a compositing command that generates watermark data and the intended audio signal. The watermark process can be applied to a normal medium regardless of whether the medium conveys information. Examples include paper, plastic, laminate, paper / film photosensitive emulsion, and the like. A watermark can also embed a single information bit or any number of bits.

  The physical representation of watermarked information is most commonly in the form of changed signal values, such as slightly changed pixel values, image brightness, image color, DCT coefficients, instantaneous audio amplitude, etc. Take it. However, watermarks can be applied to other features such as changes in the surface microstructure of the media, local chemical changes (eg in photographic emulsions), local changes in optical density, local changes in emission, etc. It can also be shown in the method. The watermark can be arbitrarily realized in a hologram and a conventional paper watermark.

  One improvement to existing technology is to use watermarked content (on the web) in addition to its normal data collection / indexing using established web crawler services (eg, AltaVista, Excite or Inktomi). Internet newsgroups, BBS systems, online systems, etc.). Crawlers such as these can download files that may have embedded watermarks (eg, * .JPG, * .WAV, etc.) for later analysis. These files can be processed in real time, as will be described later. More generally, such files are queued and processed by a computer different from the crawler computer. Instead of performing a watermark read operation on each such file, screening techniques can be used to identify what is most likely to carry the watermark data. One such technique is to perform DCT processing on the image and look for spectral coefficients for a particular watermarking technique (eg, coefficients for subliminal grids that are tilted and embedded). In order to decode the watermark based on the spread spectrum, the analysis computer needs access to the noise signal that was used to spread the data signal. In one embodiment, the party provides a noise / key signal to the crawler service so that the content marked by the party can be identified. The crawler service kept this information secret and various noise information was watermarked to decode the image (where the image was used as a short term for image, video and audio) Use until data is found (if any). This makes it possible to use a web crawler to find content locations that have a privately encoded watermark, as in the present case, rather than just a publicly encoded watermark. By queuing content data for analysis, you have an opportunity for a computational shortcut. For example, images of equal size (eg, 256 × 256 pixels) are tiled into a larger image and examined as a unit for the presence of watermark data. When the decoding technique (or any prescreening technique) uses a DCT transformation or the like, the block size of the transformation can be tiled to correspond to the tile size (or an integer fraction thereof). A complete read process is then performed on the blocks indicated as possibly having a watermark. If the queued data is classified by file name, file size or checksum, duplicate files can be identified. Once such an overlap is confirmed, the analysis computer need only consider one instance of the file. If watermark data is detected from such a file, the content provider can be informed of each URL where a copy of the file was detected.

  Some commentators have realized that web crawler searches for watermarked images can be broken down by breaking the watermarked image into sub-blocks (tiles). HTML instructions, etc. cause sub-blocks to be produced in a tiled fashion and recreate a complete image. However, watermark reading cannot be reliably achieved due to the small size of the constituent sub-blocks.

  This attack is overcome by instructing the web crawler to collect display instructions (e.g., HTML) to place the image file for display on the web page in addition to the image fail itself. Prior to inspecting the files collected from the web page for watermarks, these files can be aligned to the location specified by the display instructions. With this arrangement, the tiles can be reassembled and the watermark data can be easily reproduced.

  Other such possible attacks on the detection of image watermarks by web crawlers scramble the images (and thus the watermarks) in the file and unscramble them before viewing them using a Java applet or the like. That is. Existing web crawlers inspect the file when the web crawler finds the file, and consequently do not detect the watermark. However, the same applet is used in the web crawler as well so that the Java descrambled applet can be invoked if the user wants access to the file, overcoming the tricks thus attempted for watermark detection. can do.

  The location of “content” can be found and indexed by various web crawlers, but the content of “content” is unknown. For example *. A JPG file may contain pornography, sunset photos, and the like.

  Using watermarks, metadata can be irreversibly associated with content (as stored in a data structure that forms other parts of the object, as is customarily done with metadata) Is different). The watermark can include text such as “sunset”. Alternatively, a more compact information representation can be employed (eg, encoded reference information). In addition, the watermark can include (or consist entirely of) a unique ID (UID), which is an index (to a database that is a networked remote database that contains metadata descriptors). Key). The remote data can include metadata described by an extensible markup language (XML) tag set. With such a configuration, web crawlers and the like can extract and index metadata descriptor tags, and the search is not simply based on the file name, but a semantic description of the file content. It becomes possible to perform based on.

  Existing watermarks typically embed information that helps convey copyright information. Some systems embed text that indicates the copyright owner. Other systems embed a UID that is used as an index into a database that stores the name of the copyright user and related information.

  Considering the future, watermarks should be more useful than static copyright warnings. One option is to embed “intelligence” into the content using watermarks. One form of intelligence is knowing its “home”. “Home” can be the URL of the site to which the content relates. For example, a photograph of a car can be watermarked with data identifying the website of the car dealer who issued the image. Wherever the image goes, the watermark serves as a link back to the original distributor. The same technology can be used for company logos. Wherever they are copied on the Internet, a properly equipped browser or the like can decrypt the data and link back to the company home page. (Decoding can be performed by placing the cursor on the logo and pressing the right mouse button. This mouse operation opens an option window, part of which is "watermark decoding".)

  In order to reduce the data load of the watermark, the intelligence need not be fully encoded in the content watermark. Alternatively, the watermark can again provide a UID, where the UID identifies a remote database record from which the car dealer's URL or the like can be retrieved. In this way, images and the like become marketing agents and link customers with vendors (with some visual sales technology in place). Unlike the example of copyright, where image distribution is an evil that needs to be tracked and stopped, image distribution can be treated here as a sales opportunity. The watermarked image serves as an entrance to commercial transactions.

  (Using an intermediate database between the watermarked content file and its final home (ie, indirectly linking) provides an important advantage, i.e., by which the distributor By updating a record in the database, it is possible to easily change the “home.” Thus, for example, when one company is acquired by another company, a smart image of the former company is acquired. Can be pointed to the new company's home web page by updating a record in the database, unlike the old company's home URL, which is hard-coded (eg, watermarked) in the object. In this sense, this may refer to a URL that will eventually be abandoned. Functions as a switch to couple the file to its current home.

  The above technique is not limited to digital content files. The same approach is equally applicable to printed images and the like. For example, a printed catalog can include a picture showing a jacket. Embed watermark data in this photo. This data can be extracted by a simple hand scanner / decoder device using simple manipulation and decoding techniques (eg known to those skilled in the art). In a watermark reading application using a hand scanner or the like, it is important to make the watermark decoder robust against image rotation. This is because catalog photos are often scanned off axis. One option is to encode a subliminal graticule (eg, a visual synchronization code) in the catalog photo so that the set of image data can be preprocessed to be restored to the proper placement before decoding. is there.

  The scanner / printer device can be coupled to a computer equipped with a modem, a telephone, or any other communication device. In the former example, the device provides URL data to the computer's web browser that links the browser to the catalog vendor's order page. (The device does not need to have its own watermark decoder and this task can be done by a computer.) The vendor's order page is the size and color of the jacket, as is customarily done by online merchants. Options, availability of inventory can be detailed and ordering instructions (credit card number, delivery selector, etc.) can be requested. Such a device connected to the phone can dial the catalog vendor's free auto-order phone number (eg, known from the data encoded in the watermark) and check the jacket with the order center. Voice prompts can then request customer selection of size, color, and delivery options, which can be made by touchtone commands or pronounced words (using known voice recognition software at vendor equipment) Input).

  In such applications, the watermark can be conceptualized as an invisible barcode used in purchase transactions. Here, as is the case in other cases, the watermark can function as a seamless interface that bridges the printing world and the digital world.

  Another way to add intelligence to content is to use a watermark to provide Java or ActiveX code. The code can be embedded in the content or stored remotely and linked to the content. This code can be executed (automatically or at the user's discretion) when the watermarked object is activated. This code can perform virtually any function. One is to “call home”, which starts a browser and links to the home of the object. An object can relay any data to its home. This data can specify certain attributes of the data and their use. The code can also prevent access to the underlying content until permission is granted. An example is a digital movie that automatically executes a watermarked Java applet that links to the movie distributor via a browser when double-clicked. The user is then prompted to enter a credit card number. After the number is verified and billed, the applet passes the file contents to a computer viewer for watching the movie. Support for these operations is preferably provided via a computer operating system or plug-in software.

  If the operating system knows the content format (eg, in an object-oriented file system, or in a “media-aware” file system), a similar function is provided with the operating system to specialize watermarked content. Process can be triggered. One example application is in the acquisition of content from external audio / image / video devices.

  Using devices such as these, it is possible to collect user-provided demographic information when smart image content is accessed by consumers of the content. This demographic information can be written to a remote database and can be used for market research, customization or personalization of information about content provided to consumers, sales opportunities, promotions, etc.

  In audio, video, etc., watermarks are useful for conveying relevant information, such as links to WWW fan sites, actors' biography, and advertising of tying products (T-shirts, CDs, concert tickets). In such applications, it is desirable (not essential) to display a small logo on the user interface (eg, screen) to inform the presence of additional information. When a consumer selects a logo via a selection device (mouse, remote control button, etc.), the information is disclosed to the consumer, and then the consumer can contact that information. .

  Much has been written about the management of asset rights (patented). Sample patent documents include US Pat. Nos. 5,715,403, 5,638,443, 5,634,012, and 5,629,980. Again, much of the technical research is recorded in academic journal articles. These articles can be identified by searching for relevant company names and trademarks such as IBM's Cryptolope system, Portland Software's ZipLock system, Softbank Net Solutions 'Rights Exchange service, and Internet Trust Technologies' DigiBox system. is there.

  The exemplary asset management system makes content available in encrypted form (eg, from a web server or on a hard disk of a new computer). The encrypted content is associated with data for specifying the content (for example, preview) and data for specifying various rights related to the content. If the user desires more complete use of the content, the user grants the charging authority (credit card) to the distributor and then the distributor gives the decryption key to allow access to the content. (Such systems are often implemented using object-based technology. In such systems, content is generally said to be distributed in “safe containers”.)

  Other asset management systems distribute the content “in plain text”, ie, not encrypted and not in a secure container. Such devices are common, for example, in enterprise asset management systems (eg, as opposed to electronic commerce applications where container-based approaches are prevalent).

  Preferably, the content should be marked (personalized / serialized) so that any illegal use of the content can be tracked. This marking can be done by watermarking, which ensures that the mark moves together wherever the content goes in any form. Watermarking can be performed by the distributor, such as by encoding the UID associated with a particular container in the database prior to distribution of the (encrypted) object. A record in the database if the content can be accessed (for example, if access rights are given to a secure container or if unencrypted content is available to the user) Can be updated to reflect the buyer / user, purchase date, rights granted, etc. Alternatively, a watermark encoder may be included in the software tool used to access the content (eg, a decryption engine for secure container-based systems). Such an encoder can embed watermark data into the content before the content is provided to the user (eg, before the content is retrieved from the container in a secure container system). The embedded data can include a UID as described above. This UID can be assigned by the distributor before distributing the content / container. Alternatively, the UID can be a data string that is unknown or not formed until access (or access rights) is granted. In addition to the UID, the watermark may include other data unknown to the distributor, for example, information specifying the time and method of accessing the content.

  In still other non-container-based systems, access rights can be implemented with watermarks in this case as well. For example, full resolution images can be freely made available on the web. When a user wants to incorporate an image into a web page or magazine, the user can query the image regarding the duration and status of its use. This may result in a link (directly or via an intermediate database) to the website specified by the embedded watermark, which may specify the desired information. The user can then arrange the necessary payments and use an image that he knows that the necessary rights are protected.

  Tagging at the time of distribution to an end user (eg, from a managed content store) to an image / video / audio asset need not consist solely of UIDs. Other data, such as serial numbers, recipient identity, distribution date, etc., can also be watermarked into the content. Further, the watermarked data can function to establish a permanent link to the metadata contained in the associated asset management database.

  As indicated, digital watermarking can also be implemented using traditional (eg, paper) watermarking techniques. Known techniques for watermarking media (eg, paper, plastic, polymers) are disclosed in US Pat. Nos. 5,536,468, 5,275,870, 4,760,239, 4,256,652, and 4370200, It is possible to adapt to the display of a visual watermark rather than a logo or the like. Note that some forms of traditional watermarks designed to be visible by transmitted light also appear as low-level signals in reflected light as commonly used in scanners. With a transmitted illumination detection system, such watermarks can also be detected using traditional optoelectronic watermark detection techniques known in the art.

  Similarly, a digital watermark can be implemented as part of an optical hologram. Known techniques for generating and securely incorporating holograms are disclosed in US Pat. Nos. 5,319,475, 5,694,229, 5,492,370, 5,483,363, 5,658,411 and 5,310,222. In order to watermark a hologram, the watermark can be represented by an image or data model that produces a hologram diffraction grating. In one embodiment, the hologram is generated conventionally and the object or symbol is displayed. The watermark marking appears in the background of the image, and as a result, the watermark marking can be detected from all viewing angles. In this case, it is not essential that the watermark representation is essentially undetectable to the viewer. If necessary, a clearly visible noise pattern can be used without compromising the application of the hologram.

  Digital watermarks can also be used with labels and tags. In addition to the traditional label / tag printing process, techniques tailored for security can also be used. Known techniques useful for generating security labels / tags are disclosed in US Pat. Nos. 5,665,194, 5,732,979, 5,561,615 and 4,268,983. The insensitivity of watermarked data and the ease of machine decoding are some of the benefits associated with watermarked tags / labels. In addition, its cost is significantly lower than many related technologies (eg, holograms). Using the watermark in this application, the originality of the product label can be assured to either the relevant product merchant or customer using a simple scanner device, thereby the ratio of counterfeit product sales Can be reduced.

  Consider the steganographic setup, ignoring the inevitable social ramification of the next device for the time being. This allows a given computing system based on a “master clock” drive function to be electrically identified with minor changes when the computing system is in digital communication with the second computing system. Signals in operation can be added. The slight change mentioned is such that it does not affect the basic operation of the first computing system in any way, nor does it affect the communication between the two systems.

  The social divergence mentioned refers to the idea that the concept of pure anonymity still exists on the Internet and the idea related to the right to privacy. The “good” of this concept seems to be a source of non-anonymity, and the “bad” of this concept seems to be a fantastic escape from responsibility for one's own actions. In any case, there appears to be a basis for a socially neutral concept of simply identifying a given physical object relative to its neighbors, so the following approach is A unique signal is provided so that a second system communicating with the first system can identify the first system.

  Steganographically encoded data using spread spectrum principles (eg, modulating a “noise-like” carrier signal with an information signal to produce a data signal) and modulating the instantaneous phase of the computer clock signal To communicate. In one embodiment, the phase of the clock frequency is instantaneously changed according to the data signal. (In some embodiments, the data signal need not reflect modulation by a noise-like carrier signal and can send an unchanged information signal.) In an exemplary system, the period of the clock signal is 3 nanoseconds. Yes, the instantaneous phase shift is in the order of +/− 10 picoseconds. (These numbers will of course depend on the particular microprocessor used and its immunity to phase noise while still running within the specification.) If a binary “1” is to be encoded, the phase is Advance 10 picoseconds. If binary “0” is to be encoded, the phase is delayed by 10 picoseconds. The successive clock edges are continuously advanced or delayed until the end of data is reached, according to the corresponding bit of the data signal. The data signal is then preferably repeated to provide redundant coding.

  Such computer clock system modulation is performed through operations including communication with its connected devices. Thus, for example, a modem or network connection carries data affected by the phase modulation of the clock signal. In particular, signal transmission in any such communication (eg, RS-232C, TCP / IP, wireless modem communication, etc.) does not occur at evenly spaced clock edges, but instead has its exact time characteristics. Occurs at a clock edge that varies slightly according to the data signal modulated by the clock signal.

  A remote computer receiving information sent from such a computer can inspect the timing of the signal (eg, transitions of its edges) and thereby identify the encoded data signal.

  If the channel to the remote computer provides near-perfect temporal fidelity (ie, essentially does not introduce variable delay into the propagation of data), the data is from its expected nominal value for each edge transition. Can be extracted by simply indicating the instantaneous offset of the data signal, and the corresponding element of the data signal is thereby identified. If the data signal is modulated by a noise-like signal, demodulation with the same noise-like signal provides the original information signal. (Such demodulation by a noise signal is not necessary if the original information signal serves as a basis for clock phase shift.)

  More typically, the channel to the remote computer introduces time distortion. One example is the Internet, where different packets follow different paths between the originating computer and the receiving computer. In such an environment, a large number of data must be extracted and processed in order to extract a data signal. For example, to determine whether an edge is leading or lagging, thousands of edge transitions corresponding to a given bit are averaged or otherwise processed so that the bits are highly statistically reliable. May have to be decrypted. Thus, in an environment with very variable transmission delays (eg, the cited Internet example), a 100-bit message is collected and processed on edge changes of the order of one million to ensure reliable message recovery. May be required. However, if the data rate on the Internet is accelerated, this may require only a few seconds of data.

  In the exemplary embodiment, the computer master clock is configured as a phase modulator and is controlled by a phase locked loop (PLL) circuit driven by a data signal. Decoding is performed by another PLL configured as a phase demodulator. (The two PLLs need not share the same time reference because decoding can proceed by examining the deviation of the instantaneous edge transitions from the average clock period.) The design of such PLL phase modulators and demodulators is within the normal capabilities of those skilled in the art.

  Given the many possible embodiments to which the principles of the present invention can be applied, the detailed embodiments are merely illustrative and should not be construed as limiting the scope of the invention. Should be understood. Rather, all embodiments are claimed as falling within the scope and spirit of the claims and their equivalents.

FIG. 2 is a diagram showing a prior art using a line drawing to achieve a gray scale effect. FIG. 2 is a diagram showing a prior art using a line drawing to achieve a gray scale effect. FIG. 4 illustrates a virtual array of grid points that can be overlaid on an image, according to one embodiment of the invention. FIG. 3 shows a virtual array of regions that can be overlaid on an image according to the embodiment of FIG. It is a figure which shows the extract of FIG. 3 which has the line from the line drawing image to pass. FIG. 4 is a diagram illustrating watermark encoding by changing the line width of FIG. 3 according to an exemplary embodiment of the present invention. FIG. 4 is a diagram illustrating watermark encoding by changing the position of the line in FIG. 3 according to another embodiment of the present invention. It is a block diagram of the photocopier according to another embodiment of the present invention. It is a figure which shows a part of automatic teller machine using the principle of this invention. FIG. 2 illustrates a portion of an apparatus (eg, photocopier, scanner, or printer) that uses the principles of the present invention. FIG. 2 illustrates a portion of an apparatus (eg, photocopier, scanner, or printer) that uses the principles of the present invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 11 ... Automatic cash accounting machine, 13 ... Optical scanner, 14 ... Image set, 16 ... Banknote, 20 ... Code signal, 22 ... Visual structure, 24 ... Integrated detection system, 30 ... Photocopier, 32 ... Visual structure, 34 ... banknote watermark data.

Claims (3)

A catalog of paper having a plurality of printed product images, wherein at least one of the plurality of images is slightly modified to steganographically encode multi-bit digital watermark data. Catalog and
A portable device for obtaining image data from product images in the paper catalog;
A digital watermark detector for decoding the multi-bit digital watermark data in the image data;
With
The decrypted multi-bit digital watermark data is used to initiate a commercial transaction for the product without having to mark the catalog with an unsightly visible machine-readable barcode-like marking. be able to,
system. A method for indexing content in a computer network,
Using an automatic web crawler to locate digital content in a computer network;
Detecting, for at least a portion of the content, a multi-bit digital watermark data encoded so as to be hidden in the content, wherein the encoding of the digital watermark is performed on the digital data including the content. The change is very small for human recognition but can be decoded by computer analysis, and the detected digital watermark data includes an identifier, and ,
Accessing a network accessible database using the detected digital watermark identifier as a key;
Obtaining at least one metadata descriptor associated with the content from the network accessible database;
Using the metadata descriptor obtained from the database to point to the content;
Including methods. The method of claim 2, wherein the metadata descriptor obtained from the network accessible database is in XML format.

Images