EP1548642A2 - Bildverarbeitungsvorrichtung - Google Patents

Bildverarbeitungsvorrichtung Download PDF

Info

Publication number
EP1548642A2
EP1548642A2 EP04030177A EP04030177A EP1548642A2 EP 1548642 A2 EP1548642 A2 EP 1548642A2 EP 04030177 A EP04030177 A EP 04030177A EP 04030177 A EP04030177 A EP 04030177A EP 1548642 A2 EP1548642 A2 EP 1548642A2
Authority
EP
European Patent Office
Prior art keywords
color
image information
information
image
difference
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
EP04030177A
Other languages
English (en)
French (fr)
Other versions
EP1548642B1 (de
EP1548642A3 (de
Inventor
Takashi c/o Intell. Property Division Yamaguchi
Shinya c/o Intell. Property Division Tokuda
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Toshiba Corp
Original Assignee
Toshiba Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Toshiba Corp filed Critical Toshiba Corp
Publication of EP1548642A2 publication Critical patent/EP1548642A2/de
Publication of EP1548642A3 publication Critical patent/EP1548642A3/de
Application granted granted Critical
Publication of EP1548642B1 publication Critical patent/EP1548642B1/de
Not-in-force legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N1/00—Scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission; Details thereof
    • H04N1/387—Composing, repositioning or otherwise geometrically modifying originals
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T1/00—General purpose image data processing
    • G06T1/0021—Image watermarking
    • G06T1/0028—Adaptive watermarking, e.g. Human Visual System [HVS]-based watermarking
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2201/00—General purpose image data processing
    • G06T2201/005—Image watermarking
    • G06T2201/0051—Embedding of the watermark in the spatial domain

Definitions

  • This invention relates to an image processing apparatus and image processing method which form synthesized image information by embedding and synthesizing additional sub-information (such as security information) set in an invisible state into main image information (such as a face image of a human being) set in a visible state.
  • additional sub-information such as security information
  • main image information such as a face image of a human being
  • the electronic watermark technique is a technique for embedding additional sub-information (sub-image information) in an invisible state into main image information.
  • the electronic watermark technique is utilized for a personal authentication medium or work on which personal information such as an IC card is recorded.
  • the electronic watermark technique it is possible to take a countermeasure against illicit copying, forgery or falsification of a personal authentication medium or work and protect personal information of the personal authentication medium and copyright of the'work.
  • the electronic watermark technique a technique for embedding sub-image information in main image information by use of the characteristic of a color-difference component or high spatial frequency component which is difficult to be sensed by a human being.
  • an apparatus which records synthesized image information on a recording medium for example, a recording apparatus of sublimation type thermal transfer recording system or melting type thermal transfer recording system is used.
  • a material which can be dyed with a sublimation material is limited. Therefore, in the sublimation type thermal transfer recording system, recording media which can be applied are limited and the degree of freedom of selection of recording media on which an image is recorded is low. Therefore, in the sublimation type thermal transfer recording system, recording media which can be used are limited and the degree of security thereof is lowered in many cases. Further, a sublimation dye is generally low in image durability such as solvent resistance and light resistance.
  • the melting type thermal transfer recording system a coloring material which is generally said to have high light resistance can be selected. Further, in the melting type thermal transfer recording system, the degree of freedom of selection of a recording medium is high. Further, in the melting type thermal transfer recording system, since a recording medium having a highly special property can be used, the degree of security can be enhanced. However, in the melting type thermal transfer recording system, a dot area gradation method which records gradation by changing the sizes of transferred dots is used. Therefore, the melting type thermal transfer recording system has a problem that it is difficult to attain the same gradation performance as that of the sublimation type thermal transfer recording system.
  • the electronic watermark technique basically deals with digital data. Therefore, in an image recording apparatus such as a color printer which prints synthesized image information on a recording medium, sub-information embedded in the synthesized image information can be prevented from being destroyed or altered.
  • Jpn. Pat. Appln. KOKAI Publication No. H6-59739 a recording method of melting type thermal transfer recording system of an image recording apparatus which enhances the gradation recording performance is disclosed.
  • the recording method disclosed in Jpn. Pat. Appln. KOKAI Publication No. H6-59739 data of zigzag form is thinned out from recording image data and information of a corresponding portion is lost. Therefore, if synthesized image information having sub-information embedded in main image information is recorded on a recording medium by using the technique disclosed in Jpn. Pat. Appln. KOKAI Publication No. H6-59739, there occurs a problem that the sub-information embedded in the synthesized image information will be destroyed.
  • Jpn. Pat. Appln. KOKAI Publication No. H9-248935 a technique for embedding sub-information in image data by use of"the characteristic of a color-difference component or high spatial frequency component which is difficult to be sensed by a human being is disclosed.
  • Jpn. Pat. Appln. KOKAI Publication No. H9-248935 only contour information of the sub-information is held in the main image information. Therefore, in the method disclosed in Jpn. Pat. Appln. KOKAI Publication No. H9-248935, only contour information can be held and concentration information cannot be held.
  • the method disclosed in Jpn. Pat. Appln. KOKAI Publication No. H9-248935 is not suitable for a restoration process or a process for embedding two-dimensional plane information such as a two-dimensional bar code.
  • an object of this invention is to provide an image processing apparatus which can form synthesized image information whose image quality is difficult to be lowered by sub-information embedded in main image information and in which the sub-information embedded in the main image information cannot be easily discovered.
  • FIG. 1 is a diagram showing an example of the configuration of an image processing system 1 according to the first embodiment.
  • the image processing system 1 is an apparatus which issues an ID card as a recording material P on which a face image for person authentication or the like is recorded.
  • the image processing system 1 includes an information input device 11, image processing device 12 and recording device 13.
  • the information input device 11 inputs information such as main image information, sub (image) information and N (a plurality of) key image information items (key information).
  • the image processing device 12 forms synthesized information based on various information items input by the information input device 11.
  • the image processing device 12 is configured by a computer (PC) or the like, for example.
  • Various processes performed by the image processing device 12 are functions realized by an application program installed in the computer.
  • the recording device 13 records a synthesized image formed by the image processing device 12 on a recording medium.
  • the recording device 13 is configured by a color printer which prints a synthesized image on a recording medium.
  • the image processing device 12 includes a memory 20, information input section 21, electronic watermark embedding section 22 and image output section 23.
  • the memory 20 stores input information and various information items obtained in the course of processing. For example, main image information, a plurality of key information items and sub image information as input information are stored in the memory 20.
  • the information input section 21 is an input interface which inputs information input by the information input device 11.
  • the electronic watermark embedding section 22 performs a process for embedding sub-information in main image information by use of the main image information, a plurality of key information items and sub-information input by the information input section 21.
  • the image output section 23 is an output interface which outputs information to the recording device 13.
  • the image output section 23 outputs synthesized image information formed by the electronic watermark embedding section 22 to the recording device 13 as recording image data.
  • the electronic watermark embedding section 22 includes a plurality of (N) color-difference amount correcting sections 31 (31a, 31b, ..., 31n), a plurality of (N) color-difference modulating sections 32 (32a, 32b, ..., 32n), synthesis processing section 33 and superposition processing section 34.
  • the image processing apparatus 12 is configured by a PC
  • synthesis processing section 33 and superposition processing section 34 are functions realized by an application program.
  • the color-difference amount correcting sections 31 perform color-difference amount correction processes for main image information by use of preset color-difference amounts ⁇ Cd.
  • the color-difference modulating sections 32 are respectively provided for the color-difference amount correcting sections 31.
  • the color-difference modulating sections 32 perform color-difference modulation processes for image information subjected to color-difference correction process by the color-difference amount correcting sections 31 according to corresponding key information items among a plurality of key information items.
  • the synthesis processing section 33 performs a selecting process and synthesizing process for a plurality of image information items subjected to color-difference modulation by the color-difference modulating sections 32 based on sub-information 102.
  • the superposition processing section 34 performs a superposition process for superposing image information as the processing result of the synthesis processing section 33 with respect to main image information as input information.
  • the electronic watermark embedding section 22 performs a process for embedding sub-information in main image information.
  • FIG. 2 is a diagram for illustrating the flow of the process of the image processing system 1 for forming the recording material P on which synthesized image information having sub-information embedded in main image information is recorded.
  • the image processing system 1 performs an information input process for fetching various information items input by the information input device 11 and used to form a recording material into the image processing device 12 by use of the information input section 21 (step S10).
  • input information is recorded in the memory 20.
  • information such as main image information 101, sub (image) information 102 and a plurality of (N) key image information (key information) items 103 (103a, 103b, ..., 103n) is input as input information.
  • the key information items 103a, 103b, ..., 103n are different types of key information items corresponding to the color-difference amount correcting sections 31a, 31b, ..., 31n.
  • main image information such as face image information is a color image having an R (red), G (green) and B (blue) components.
  • the key information items 103a, 103b, ..., 103n are represented by binary images.
  • the electronic watermark embedding section 22 performs an electronic watermark embedding process for embedding the sub-information 102 in the main image information 101 input in the information input process by use of a plurality of key information items 103a, 103b, ..., 103n (step S11).
  • the color-difference amount correcting process for the main image information 101 input by the information input section 21 is first performed by use of the color-difference amount correcting sections 31a, 31b, ..., 31n (steps S12a, S12b, ..., S12n).
  • a process for'correcting a color-difference amount with respect to the main image information 101 is performed by each of the color-difference amount correcting sections 31 by using a preset color-difference amount ⁇ Cd.
  • the main image information in which the color-difference amount is corrected by each of the color-difference amount correcting sections 31 is subjected to the color-difference modulation process by the color-difference amount modulating sections 32 (32a, 32b, ..., 32n) corresponding to the above color-difference amount correcting sections 31 (step S13a, S13b, ..., S13n).
  • the color-difference modulation process color differences of the main image information items whose color-difference amounts are corrected by the color-difference amount correcting sections 31 are modulated by use of the key information items 103a, 103b, ..., 103n corresponding to the color-difference amount modulating sections 32.
  • a plurality of image information items whose color differences are modulated by the color-difference modulation process are subjected to the selecting and synthesizing processes by the synthesis processing section 33 (step S14).
  • desired information is selected from a plurality of image information items whose color differences are modulated by the color-difference modulation processing sections 32 and synthesized information items are synthesized.
  • a process for superposing the information on the main image information 101 is performed by the superposition processing section 34 (step S15). That is, in the superposition process, synthesized image information 111 having image information as the result of the process of the synthesis processing section 33 superposed on the main image information 101 as input information is formed. The thus formed synthesized image information 111 is output to the recording device 13 by the image output section 23 as information to be recorded on a recording medium (step; S16).
  • step S17 a recording process for recording the synthesized image information output from the image processing device 12 on a recording medium is performed (step S17).
  • the main image information obtained by superposition in the superposing process is visible and sub-information is invisible. Therefore, since the sub-information 102 is invisible to the eyes of a human being, only the main image information 101 of the recording material (printed matter) P on which the synthesized image information is recorded by the recording process can be visually recognized.
  • FIG. 3 is a diagram for illustrating the flow of an electronic watermark restoring process for restoring or reproducing sub-information from the synthesized image information recorded on the recording material P.
  • the above process is performed by an electronic watermark restoring apparatus.
  • the electronic watermark restoring apparatus is realized by a computer to which an image reading device and display device are connected.
  • processes which will be described below are functions realized by causing the computer to execute various types of processing programs.
  • the electronic watermark restoring apparatus first performs an image input process for inputting synthesized image information recorded on the recording material P (step S21).
  • the image input process is realized by causing the electronic watermark restoring apparatus to optically read the synthesized image information recorded on the recording material P by use of the image reading device, convert the read information into digital form and input the thus formed digital image information.
  • the electronic watermark restoring apparatus performs a restoring process for restoring sub-information as an electronic watermark in the input image information (step S22).
  • a restoring process for restoring sub-information as an electronic watermark in the input image information.
  • the sub-information restoring process for example, a frequency detecting process and re-structuring process are performed.
  • the frequency detecting process is a process for detecting a spatial frequency component of particular (one) key information among a plurality of key information items from the image information input by the image input process (step S23). For example, when the spatial frequency component of the key information 103a is detected, the spatial frequency component of the key information 103a is detected from the image information input by the image input process in the frequency detecting process.
  • the re-structuring process is a process for re-structuring sub-information based on the spatial frequency component detected by the frequency detecting process (sep S24).
  • the result of the re-structuring process sub-information restored from the printed matter P is obtained.
  • the spatial frequency component of the key'information 103a is detected by the frequency detecting process
  • sub-information is re-structured based on the spatial frequency component of the key information 103a detected by the frequency detecting process in the re-structuring process.
  • the electronic watermark restoring apparatus outputs the restored sub-information (step S25). For example, in the electronic watermark restoring apparatus, the restored sub-information is displayed on the display section.
  • the truth or not of the recording material P can be determined based on the output result of the restored sub-information (for example, sub-information displayed on the display section).
  • the electronic watermark restoring apparatus it is also possible for the electronic watermark restoring apparatus to determine the truth of not of the recording material P according to whether or not the restored sub-information coincides with desired sub-information and output the result of determination.
  • FIG. 4 is a diagram for illustrating examples of main image information, sub-information and a plurality .of key information items.
  • the main image information 101 is a personal face image as personal authentication information printed on an ID card used as a recording material, for example. Further, the main image information 101 is not limited to the personal authentication face image and may be an image such as a pattern of securities. In addition, the main image information 101 may be a black-white gradation image or a full-color image.
  • the sub (image) information 102 is information to enhance the degree of security of the main image information 101, information used for truth determination, or information used for copyright management (which is a numeral "174" in an example of FIG. 4).
  • information used for truth determination which is a numeral "174" in an example of FIG. 4.
  • information used for copyright management which is a numeral "174" in an example of FIG. 4.
  • information obtained by converting a binary bit string into a light-shade image information obtained by coding a name or date of birth and converting the coded data into an image, a pattern of a logo mark of a company or organization or the like can be used.
  • the key information items 103a, 103b, ..., 103n are information items used in the electronic watermark embedding process and sub-information restoring process.
  • As the key information items 103a, 103b, ..., 103n information obtained by converting a binary bit string into a binary (black-white) image, binary image configured by a geometrical pattern, or information obtained by converting a (pseudo) random number pattern formed based on a preset seed into a binary image can be used.
  • the image processing system 1 performs the electronic watermark embedding process by use of the above information items.
  • the synthesized image information (electronic watermark embedded image information) 111 obtained by the electronic watermark embedding process, the sub-information 102 is set in an invisible state and the main image information 101 is set in a visible state.
  • the image processing system 1 performs a printing process for printing the synthesized image information 111 on a recording medium (step S17).
  • a recording material P on which the main image information 101 having the sub-information 102 embedded therein in a invisible state by use of a plurality of key information items 103a, 103b, ..., 103n is printed is completed.
  • FIG. 5 shows an IC card 121 as one example of the recording material formed by the image processing system 1.
  • a face image 122 of a possessor of the ID card and personal management information 123 of the possessor of the ID card are recorded.
  • the face image 122 is synthesized information subjected to the electronic watermark process explained with reference to FIG. 2 or 4. That is, in the face image 122, sub-information is embedded in an invisible state in a face image as main image information set in a visible state.
  • the personal management information 123 is personal information containing an identification number (ID number), name, date of birth, period of validity or the like. Further, part of the personal management information 123 can be used as sub-information. In this case, in the ID card 121, the face image 122 and personal management information 123 can be related to each other. By setting the face image 122 and personal management information 123 in association with each other, it becomes difficult to forge or falsify part of recording material 121. As a result, the degree of security of the ID card 121 can be enhanced.
  • FIGS. 6A to 6E are diagrams for illustrating the color-difference modulation process of each color-difference modulating section 32.
  • the color-difference modulation process is performed by use of the key information 103 and corrected color-difference amount obtained in the color-difference amount correction process by a corresponding one of the color-difference amount correcting sections 31.
  • FIG. 6A shows part of the binary key information 103 (eight pixels in a horizontal direction ⁇ one pixel in a vertical direction).
  • a white portion indicates a white pixel (W)
  • a slant-line portion indicates a black pixel (K).
  • the color-difference amount ⁇ CD is divided into three components of R, G and B components. That is, the color-difference amount ⁇ CD includes a color-difference amount ⁇ CD -R of the R component, a color-difference amount ⁇ CD -G of the G component and a color-difference amount ⁇ CD -B of the B component. If the color-difference amount is expressed by eight bits, it is assumed that 255 ⁇ ⁇ CD -R , ⁇ CD -G , ⁇ CD -B ⁇ 0. In this example, the value of ⁇ CD is also the intensity of the electronic water mark.
  • ⁇ CD As the value of ⁇ CD becomes larger, it becomes easier to restore sub-information from the synthesized image information in the electronic watermark restoring process. If the value of ⁇ CD is set excessively large, sub-information in the synthesized image information can be easily discovered.
  • the color-difference amount ⁇ CD is derived based on the following equations (A-1) to (A-6) for each component of the R component ⁇ CD -R , the G component ⁇ CD -G and the B component ⁇ CD -B .
  • RSLT(i,j) K
  • RSLT(i,j) -R - ⁇ CD -R
  • RSLT(i,j) -G + ⁇ CD -G
  • RSLT(i,j) -B + ⁇ CD -B
  • FIG. 6B shows the result RSLT -R of the color-difference modulation process for the color-difference amount ⁇ CD -R of the R component by the key information shown in FIG. 6A.
  • FIG. 6C shows the result RSLT -G of the color-difference modulation process for the color-difference amount ⁇ CD -G of the G component by the key information shown in FIG. 6A.
  • FIG. 6D shows the result RSLT -B of the color-difference modulation process for the color-difference amount ⁇ CD -B of the B component by the key information shown in FIG. 6A.
  • FIG. 6E shows image information.
  • FIG. 6E shows the processing result of color-difference modulation obtained by synthesizing RSLT -R shown in FIG. 6B, RSLT -G shown in FIG. 6C and RSLT -B shown in FIG. 6D.
  • the pitch of pixels in the image information is set to a high resolution (approximately 300 dpi or more) which exceeds the sensible range of the naked eyes of a human being, red and cyan in the image information cannot be separately identified and are visually recognized as an achromatic color (gray). That is, by applying the color-difference modulation process to convert a key information pattern into a color-difference information pattern, the key information pattern can be apparently replaced by achromatic color information.
  • color-difference modulation process as an example of a complementary color, the color-difference modulation process using red and cyan is explained.
  • a color-difference modulation process which is the same as the above color-difference modulation process can be realized by using other combinations of complementary colors such as green and magenta, blue and yellow and the like.
  • red is allocated to the white pixels of key information shown in FIG. 6A and cyan is allocated to the black pixels.
  • colors allocated in the color-difference modulation process are set in a relative relation. That is, even if the complementary colors respectively allocated to binary key information items are allocated in reverse, there occurs no problem in principle.
  • key information in which white and black pixels are changed at regular intervals in a two-pixel unit is provided.
  • key information may contain black and white pixels in substantially the same ratio in a particular area (a macro area visually recognized by the naked eyes of a human being) in principle.
  • key information may be information which cannot be identified by the naked eyes of a human being. Therefore, binary pixels in the key information are not always required to appear at regular intervals.
  • a random number (pseudo) pattern generated based on a certain seed can be used as the key information.
  • FIG. 7 is a diagram for illustrating the flow of the whole color-difference amount correction process.
  • the step S70 shown in FIG. 7 is the color-difference amount correction process by the color-difference amount correcting section 31 (corresponding to the steps 12a, 12b, ..., 12n shown in FIG. 2, for example) and the step S71 shown in FIG. 7 is the color-difference modulating process by the color-difference modulating section 32 (corresponding to the steps 13a, 13b, ..., 13n shown in FIG. 2, for example).
  • each color-difference amount correcting section 31 sets a target pixel in the main image information 101 (step S73). If the target pixel in the main image information 101 is set, each color-difference amount correcting section 31 sets a color-difference amount used as a reference (step S74).
  • the color-difference amount used as the reference is the color-difference amount ⁇ Cd shown in FIG. 2 and is a previously set value.
  • each color-difference correcting section 31 performs a correction amount calculating process (color-difference correction process in each pixel unit) for a color-difference amount in the pixel unit based on each of pixel information items which configure the main image information 101 (step S75).
  • the color-difference correction process of each pixel unit is a process which prevents occurrence of overflow or underflow of image information at the time of a superposition process which will be described later.
  • each color-difference correcting section 31 sets a minimum color-difference amount (step S76).
  • the minimum color-difference amount is previously set as a parameter. The minimum color-difference amount is used to prevent that the color-difference amount becomes "0" and the electronic watermark embedding process cannot be performed (sub-information cannot be embedded).
  • each color-difference amount correcting section 31 performs a correction amount calculating process (color-difference correction process of each block unit) for a color-difference amount of each block unit.
  • the color-difference correction process of each block unit is a process which establishes a coordinated relation between the pixels (step S77).
  • the coordinated relation between the pixels cannot be established only by the color-difference correction process of each pixel unit in the step S74. That is, when only the color-difference correction process of each pixel unit in the step S74 is used, the color-difference correction amounts may be variously changed in each pixel unit. Therefore, in each color-difference amount correcting section 31, the correction process is performed again for the block area in a preset range by performing the color-difference correction process in the block unit as described above.
  • each block unit it is preferable to set a range corresponding to the inherent interval of the key information items used in the color-difference modulation process as the block area. This is because the balance between the color differences of the image information can be easily attained.
  • the result of a series of processes is the result of the color-difference correction process. Therefore, the color-difference amount obtained in the color-difference correction process of each block unit in the step S77 is supplied as the result of the color-difference correction process to each color-difference modulating section 32. Thus, in each color-difference modulating section 32, the color-difference modulating process is performed for the corrected color-difference amount.
  • the color-difference modulating process is performed for a plurality of corrected color-difference amounts obtained by the color-difference correction process of each color-difference amount correcting section 31 in each color-difference modulating section 32. Therefore, when a plurality of key information items are used, a plurality of results of color-difference modulating processes can be obtained.
  • step S14 the selecting/synthesizing process (step S14) by the synthesis processing section 33 is explained in detail.
  • a pixel arranged adjacent to a pixel of a particular color and having the same color (or a color of the same series) as the particular color is defined as a connecting pixel. Therefore, in a binary image configured by black and white pixels, adjacent white pixels or adjacent black pixels are called connecting pixels.
  • the number of pixels which configure an area (connecting component) including a pixel of a particular color and a pixel or pixels arranged adjacent thereto and having a color of the same series is defined as the number of connecting pixels. Therefore, in the binary image, the number of pixels configuring an area containing adjacent white pixels (the number of white pixels used for connection) or the number of pixels configuring an area containing adjacent black pixels (the number of black pixels used for connection) is called the number of connecting pixels.
  • FIG. 8 is a view showing an example of a black-white image as an example of a binary image.
  • the binary image shown in FIG. 8 is configured by a plurality of pixels including eight pixels in the horizontal direction ⁇ six pixels in the vertical direction.
  • a pattern P1 is configured by black pixels indicated by slant lines in FIG. 8 on the background of white pixels in an area of eight pixels in the horizontal direction ⁇ six pixels in the vertical direction.
  • the number of connecting pixels of black pixels in an area K-A is two
  • the number of connecting pixels of black pixels in an area K-B is four
  • the number of connecting pixels of black pixels in an area K-C is two
  • the number of connecting pixels of black pixels in an area K-D is four
  • the number of connecting pixels of white pixels in an area W-A is twenty-eight.
  • FIG. 9 shows an example of key information 131 as a binary image.
  • FIG. 10 shows an example of key information 132 as a binary image.
  • a white portion indicates a white pixel (W) and a slant-line portion indicates a black pixel (K).
  • FIGS. 11 and 12 show examples of image information items 141, 142 obtained as the processing results of the selecting/synthesizing process.
  • the image information 141 shown in FIG. 11 and the image information 142 shown in FIG. 12 are examples obtained by first subjecting the key information 131 shown in FIG. 9 and the key information 132 shown in FIG. 10 to the color-difference modulating process and then subjecting them to the selecting/synthesizing process based on sub-information.
  • a white portion indicates a pixel which is rich in a red component (R-rich) and a slant-line portion indicates a pixel which is rich in a cyan component (C-rich).
  • R-rich red component
  • C-rich cyan component
  • a central portion (four pixels in the horizontal direction ⁇ four pixels in the vertical direction) in the image information of eight pixels in the horizontal direction ⁇ eight pixels in the vertical direction obtained by synthesizing four image information items each having four pixels in the horizontal direction ⁇ four pixels in the vertical direction.
  • the number of connecting pixels which are cyan component rich (C-rich) pixels is eight and the number of connecting pixels which are red component rich (R-rich) pixels is two.
  • the image information 141 shown in FIG. 11 has the maximum number of eight connecting pixels (C-rich) and the minimum number of two connecting pixels (R-rich).
  • pixels which are cyan component rich (C-rich) are concentrated with the central portion set as a center (particular area). Therefore, in the image information 141 shown in FIG. 11, it can be said that the color balance of red (R)-cyan (C) is bad. In such image information, there is a strong possibility that the image quality attained after the electronic watermark embedding process is deteriorated and sub-information is discovered.
  • the number of connecting pixels which are cyan component rich (C-rich) pixels is four and the number of connecting pixels which are red component rich (R-rich) pixels is four.
  • the image information 142 shown in FIG. 12 has the maximum number of four connecting pixels (C-rich) and the minimum number of four connecting pixels (R-rich).
  • neither pixels which are cyan component rich nor pixels which are red component rich are concentrated with the central portion set as a center (particular area). Therefore, in the image information 142 shown in FIG. 12, it can be said that the color balance of red (R)-cyan (C) is good. In such image information, the image quality attained after the electronic watermark embedding process is difficult to be deteriorated and sub-information is difficult to be discovered.
  • the image processing operation is performed so as to reduce the number of connecting pixels in the image information subjected to color-difference modulation.
  • the rotation process or the like is performed for the image information obtained as the result of the color-difference modulation process so as to reduce the number of connecting pixels in a preset area.
  • FIG. 13 is a view showing an example of image information 151 obtained by synthesizing a plurality of image information items subjected to color-difference modulation by use of a plurality of key information items based on sub-information.
  • FIG. 14 is a view showing an example of four image information items 152a to 152d obtained by dividing the image information 151 of FIG. 13.
  • FIG. 15 is a view showing image information 153 as the processing result of a selecting/synthesizing process.
  • a white portion indicates a red component rich (R-rich) pixel and a slant-line portion indicates a cyan component rich (C-rich) pixel.
  • the image information 151 shown in FIG. 13 is divided into a plurality of image information items according to a plurality of key information items used in the color-difference modulation process.
  • the image information 151 shown in FIG. 13 is divided into four image information items 152a, 152b, 152c, 152d shown in FIG. 14.
  • the image information items 152a and 152d shown in FIG. 14 are image information subjected to color-difference modulation by using the key information 131 shown in FIG. 9.
  • the image information items 152b and 152c shown in FIG. 14 are image information subjected to color-difference modulation by using the key information 132 shown in FIG. 10.
  • the maximum number of connecting pixels is eight and the minimum number of connecting pixels is two.
  • the connecting pixels of the maximum number are configured by cyan component rich (C-rich) pixels and exist in an area R151 shown in FIG. 13. Therefore, in the image information 151 shown in FIG. 13, the color balance of the red (R)-cyan (C) is bad. If the color balance in the whole image is bad, the image quality is deteriorated and sub-information tends to be discovered. For example, in the image information 151, since the number of connecting pixels which are cyan component rich (C-rich) pixels is large, a cyan color may be easily visually recognized.
  • the image information 151 shown in FIG. 13 can be divided into the four image information items 152a to 152d shown in FIG. 14, the image information items 152b and 152c are laterally inverted by the synthesis processing section 33. Then, the image information 151 shown in FIG. 13 is converted to image information 153 shown in FIG. 15.
  • the maximum number of connecting pixels is four and the minimum number of connecting pixels is two.
  • the number of connecting pixels which are red component rich (R-rich) pixels indicated by white portions and cyan component rich (C-rich) pixels indicated by slant-line portions are set to four.
  • the color balance of red (R)-cyan (C) is improved in comparison with the image information 151 shown in FIG. 13.
  • the image information 153 shown in FIG. 15 if the color balance in the whole image is good, the image quality is difficult to be deteriorated and sub-information is difficult to be discovered.
  • image information obtained by converting a binary bit string into a light-shade image image information obtained by coding a name, date of birth or the like and converting the thus coded data into an image or image information of a pattern such as a logo mark of a company or organization can be used.
  • FIG. 16 is a view showing an example of sub (image) information 162 obtained by converting a code 161 into a light-shade image.
  • the code 161 is a binary bit string represented by "1" and "0".
  • the sub image information 162 is an image obtained by converting the code 161 into a binary light-shade image.
  • the sub image information 162 is image information obtained by setting "0" of the code 161 to correspond to a black pixel (K) and setting "1" of the code 161 to correspond to a white pixel (W) and thus converting the code 161 into two-dimensional light-shade information.
  • the sub-information 102 is embedded in the main image information 101 by the superposition process (step S15 in FIG. 2) of the superposition processing section 34. Therefore, the image size of the sub-information 102 must be equal to or smaller than the size of the main image information 101. If the image size of the sub-information 102 is larger than the image size of the main image information 101, it is necessary to adjust the image size of the sub-information 102.
  • the image size of the sub-information 102 obtained by converting a certain code into a light-shade image is larger than the size of the main image information 101, it is possible to adjust the pixel bits of the sub-information 102 in the depth direction (increase the number of bits for each pixel). However, it is not necessary to set the resolution and the number of pixels of the sub-information 102 converted into the light-shade image equal to those of the main image information 101.
  • a symbol 163 indicates a case wherein the sub-information 162 is converted into a 2-bit light-shade image.
  • a symbol 164 indicates a case wherein the sub-information 162 is converted into a 3-bit light-shade image.
  • a symbol 165 indicates a case wherein the sub-information 162 is converted into an 8-bit light-shade image.
  • the first selection/synthesis processing method is a method for dividing an area corresponding to one pixel of the sub-information 102 and allocating a plurality of key information items 103 (103a, ...) to each divided area.
  • a plurality of key information items 103 103a, ...)
  • FIG. 17 is a diagram showing an example of the correspondence relation between each bit plane corresponding to the number of bits (gradation levels) of the light-shade image as sub-information and a plurality of key information items.
  • FIG. 18 is a view showing nine divided areas 171 to 179 obtained by dividing an area 170 corresponding to one pixel of the sub-information.
  • the first and second key information items are allocated to the seventh bit plane (MSB: Most Significant Bit)
  • the third and fourth key information items are allocated to the sixth bit plane
  • ... the thirteenth and fourteenth key information items are allocated to the first bit plane
  • the fifteenth and sixteenth key information items are allocated to the 0 th bit plane (LSB: Least Significant Bit).
  • the seventh bit of the sub-information is "0"
  • the first key information is allocated to the seventh bit plane and if the seventh bit of the sub-information is "1”, the second key information is allocated to the seventh bit plane.
  • key information items are allocated to the respective bit planes according to the values of the respective bits (seventh to 0 th bits) of the sub-information items.
  • the result of the selecting/synthesizing process for one pixel of the sub-information can be obtained as shown in FIG. 18, for example, in the first selection/synthesis processing method.
  • an area (a square area of an outer thick frame) 170 corresponds to one pixel of the sub-information. Further, in FIG. 18, it is assumed that areas 171, 172, 173, 174, 175, 176, 177 and 178 sequentially indicated from the upper left portion respectively correspond to seventh, sixth, fifth, fourth, third, second, first and 0 th bit planes.
  • first or second key information third or fourth key information, ..., thirteenth or fourteenth key information, fifteenth or sixteenth key information
  • seventh (sixth, ..., first, 0 th ) bit plane is allocated to the area 171 (172, ..., 177, 178) shown in FIG. 18.
  • a bit plane corresponding to an area 179 in the central portion is not present. Therefore, desired information can be allocated to the area 179. For example, dummy key information can be used in the area 179. In the example of FIG. 18, the first or second key information corresponding to the seventh bit plane is allocated to the area 179 again.
  • the area 170 of one pixel of the sub-information having 8-bit light-shade information is divided into the plurality of areas 171 to 179 in the first selection/synthesis processing method.
  • a plurality of key information items corresponding to the bit values of the eight bits indicating the value of the pixel are respectively allocated to the thus divided areas 171, ..., 178 based on a preset correspondence relation.
  • image information 170 as shown in FIG. 18 can be formed as the result of the first processing method for one pixel of the sub-information.
  • N color-difference amount correcting sections 31 and N color-difference modulating sections 32 are necessary to use N color-difference amount correcting sections 31 and N color-difference modulating sections 32.
  • the second selection/synthesis processing method is a method for allocating key information items of the same number as the bit number corresponding to one pixel of sub-information.
  • FIG. 19 is a diagram showing an example of the correspondence relation between the number of bits (gradation levels) representing one pixel of sub-information and key information items of the number of bits.
  • FIG. 20 is a view showing an area 180 of one pixel of sub-information as the processing result by the second selection/synthesis processing method.
  • FIG. 19 is a diagram showing an example of the correspondence relation of 256 key information items with respect to one pixel of sub-information represented by 256 bits.
  • 256 key information items corresponding to the respective bits (0 th bit to 255 th bit) of the 256 bits representing one pixel are shown.
  • the area of one pixel of sub-information is divided into a plurality of areas and key information is allocated for each area.
  • the second selection/synthesis processing method one key information corresponding to the representative value of one pixel of sub-information is selected based on the correspondence relation as shown in FIG. 19 and the thus selected key information is allocated to the pixel.
  • image information in which one key information corresponding to the pixel value of one pixel is allocated to the pixel can be obtained as shown in FIG. 20.
  • N color-difference amount correcting sections 31 and N color-difference modulating sections 32 it becomes necessary to use N color-difference amount correcting sections 31 and N color-difference modulating sections 32.
  • the third selection/synthesis processing method is to divide sub-information for each bit plane, convert the thus divided information into a light-shade image and allocate key information.
  • the symbol 165 in FIG. 16 an example of the third selection/synthesis processing method for the sub (image) information converted into an 8-bit light-shade image is explained.
  • the sub (image) information converted into an 8-bit light-shade image is divided into a 0 th bit plane to a seventh bit plane and the seventh bit plane to 0 th bit plane are repeatedly arranged in order. Further, when the third selection/synthesis processing method is applied, it is necessary to store the arrangement of the seventh bit plane to 0 th bit plane repeatedly arranged in order so as to use the arrangement in the later restoring process.
  • Each bit plane is a 1-bit ("0" or "1") light-shade image. Therefore, the light-shade image in which the seventh bit plane to 0 th bit plane are repeatedly arranged in order finally becomes a 1-bit ("0" or "1") light-shade image.
  • the 8-bit light-shade image is converted into a 1-bit light-shade image.
  • Such a 1-bit light-shade image can be subjected to the selecting/synthesizing process by use of only two key information items. For example, if only the first and second key information items shown in FIG. 17 are used, the first key information is allocated when the light-shade image is "0" and the second key information is allocated when the light-shade image is "1".
  • the sub-information formed by the third selecting/synthesizing process is restored by using the order in which the seventh bit plane to 0 th bit plane are repeatedly arranged. That is, the sub-information formed by the third selecting/synthesizing process is restored by performing the procedures of the third selecting/synthesizing process in a reverse direction. Therefore, in the third selecting/synthesizing process, it is necessary to store the order in which the seventh bit plane to 0 th bit plane are repeatedly arranged so as to restore the sub-information.
  • N color-difference amount correcting sections 31 and N color-difference modulating sections 32 it becomes necessary to use N color-difference amount correcting sections 31 and N color-difference modulating sections 32.
  • the superposing process for superposing image information obtained as the processing result by the synthesis processing section 33 on the main image information 101 is performed.
  • the image information obtained as the processing result of the superposing process by the superposition processing section 34 becomes synthesized image information 111. Therefore, the synthesized image information 111 obtained as the processing result of the superposing process by the superposition processing section 34 is set in a state in which the sub-information 102 is embedded in an invisible state in the main image information 101.
  • calculating processes are separately performed for three components (R, G, B) configuring the image information by use of the following equations (E-1) to (E-3).
  • DES(i,j) -R SRC(i,j) -R + RSLT2(i,j) -R
  • DES(i,j) -G SRC(i,j) -G + RSLT2(i,j) -G
  • DES(i,j) -B SRC(i,j) -B + RSLT2(i,j) -B
  • DES(i,j) indicates synthesized image information
  • SRC(i,j) indicates main image information
  • RSLT2(i,j) indicates image information obtained as the processing result of the selecting/synthesizing process.
  • pattern information of key information is converted into a color-difference information pattern by use of the complementary color relation and replaced by apparently achromatic color information.
  • sub-information is related to a plurality of key information items.
  • the sub-information is image information obtained as the processing result of the selecting/synthesizing process and corresponds to RSLT2(i,j) in the equations (E-1) to (E-3).
  • RSLT2(i,j) is information whose color difference cannot be identified in a macro fashion to the naked eyes of a human being and which can be observed as an achromatic color. Therefore, as indicated by the following equations (F-1) and (F-2), synthesized image information DES(i,j) looks like main image information SRC(i,j). RSLT2 ⁇ 0 DES ⁇ SRC
  • the electronic watermark embedding process in the image processing system 1 mainly has the following features (1) to (3).
  • the synthesized image information (image information with the electronic watermark embedded therein) formed by the electronic watermark embedding process does not depend on the stored image format. Therefore, the synthesized image information can cope with not only the image format such as BMP, TIFF or JPEG which is now available but also a new image format which will be provided in the future.
  • step S22 the restoring process (step S22) in FIG. 3 is explained.
  • the sub-information restoring process is a process for extracting a particular spatial frequency component from synthesized image information based on key information used in the electronic watermark embedding process and re-structuring sub-information based on the extracted frequency component.
  • a plurality of key information items are used.
  • one pixel of sub-information represented by a plurality of bits is divided into a plurality of bit planes and key information items which are related to the respective bit planes are allocated to the bit planes.
  • key information items which are related to the respective bit planes are allocated to the bit planes.
  • FIG. 17 when a plurality of key information items are allocated to each bit plane, sub-information can be restored by use of the first key information in the seventh bit plane.
  • sub-information can be restored by use of the fifteenth key information in the 0 th bit plane.
  • key information corresponding to a bit plane to be processed is simply referred to as key information.
  • the frequency detecting process (step S23) is a process for extracting a particular spatial frequency component based on the key information.
  • a spatial frequency filter As a method for extracting a particular spatial frequency component based on the key information, a spatial frequency filter can be used.
  • the coefficient of the spatial frequency filter corresponding to the key information is calculated by the following procedures (1) to (4). It is possible to previously perform the calculation for the coefficient and store the result or calculate the coefficient each time or before the extracting process of the particular spatial frequency component is performed.
  • the process (1) is explained with reference to FIGS. 21 and 22.
  • FIG. 21 is a view showing an example of key information.
  • a white portion indicates a white pixel (W) and a slant-line portion 192 indicates a black pixel (K).
  • FIG. 22 is a view showing an example of synthesized image information input by the image input process.
  • a white circle 193 indicates a dot which is red component rich and a black circle 194 indicates a dot which is cyan component rich.
  • a waveform 195 indicates a fundamental frequency waveform in the main scanning direction and a waveform 196 indicates a fundamental frequency waveform in the sub-scanning direction.
  • synthesized image information input in the image input process is expressed as shown in FIG. 22.
  • key information embedded in the synthesized image information is converted into a form 197 as shown in FIG. 22.
  • the fundamental frequency of the form 197 is set equivalent to the frequency obtained when the size of key information is expanded by the ratio of the reading resolution to the printing resolution. Therefore, in order to calculate a filter coefficient, variations in the resolution (printing resolution) in the recording process into the recording medium and the resolution (reading resolution from the recording medium) of an input image in the image input process are previously stored.
  • a frequency filter used to extract the spatial frequency component of key information from synthesized image information is designed.
  • the key information is originally binary image information. Therefore, the synthesized image information has a feature that the inclination of an edge (a boundary between the black and white pixels which contact each other) is steep. As the edge is steeper in the spatial region, more harmonics are contained in the frequency region. Therefore, if a frequency filter coefficient calculated by use of synthesized image information having a large number of steep edges is used, noise lying on the harmonic side passes as it is, the S/N ratio is lowered and there occurs a problem in restoring sub-information.
  • the adjusting operation by the process (3) becomes necessary.
  • the contents of the process (3) depend on individual key information items and the operation environment of the system. Generally, in order to suppress noises, occurrence of harmonics is inhibited and only the frequency component lying near the fundamental frequency is permitted to pass. Further, in the environment containing less noise, it is also possible to positively utilize the complexity of key information by passing the harmonics and enhance the degree of security.
  • the method for extracting the specified spatial frequency component is not limited to the above method using the spatial frequency filter.
  • a method for extracting the specified spatial frequency component by mapping synthesized image information into a space and processing the same by use of a known Fourier transform and wavelet transform and then mapping the thus processed information in reverse can ,be used.
  • the re-structuring process (step S24) is a process for re-structuring sub-information based on a spatial frequency component extracted by the frequency detecting process.
  • sub-information is restored as black and white binary image information by subjecting the extraction result by the frequency detecting process to the binary coding process by use of a preset threshold value.Th. That is, in the re-structuring process, sub-information is restored for each bit plane by use of key information items related to the respective bit planes (for example, the correspondence relation shown in FIG. 17) and sub-information items restored based on the respective bit planes are synthesized. Thus, the sub-information embedded is completely restored by use of a plurality of bit planes.
  • the restoring mask sheet can be formed by using black pixels of key information used in the bit planes of sub-information which is desired to be restored as recording information, using white pixels thereof as non-recording information, setting the resolution equal to the recording resolution of synthesized image information and recording the information items on a transparent recording medium.
  • the restoring mask sheet is formed such that the first color of the complementary colors is recorded as black pixels on the transparent recording medium and the second color is set in a non-recording state. Therefore, in the restoring mask sheet, the black pixels corresponding to the first color shield the underlying image and the non-recording pixels corresponding to the second color are transparent and permit the underlying image to be observed.
  • the synthesized image information on which the restoring mask sheet is superposed can be seen in portions corresponding to one of the first and second colors set in the complementary color relation and is shielded and cannot be observed in the remaining portions.
  • the color balance of the color difference of the synthesized image information on which the restoring mask sheet is superposed is no more maintained and sub-information is deviated from the achromatic color state. Therefore, the sub-information embedded in the synthesized image information can be visually observed.
  • the color-difference amount of the main image information is corrected by a plurality of color-difference correcting processes and a plurality of color-difference modulating processes are performed based on a plurality of key information items and a plurality of color-difference correction amounts attained by the plurality of color-difference correcting processes. Then, the processing result obtained by the plurality of color-difference modulating processes is subjected to the selecting/synthesizing process based on the sub-information and synthesized image information having the sub-information embedded in an invisible state in the main image information is formed by superposing the processing result of the selecting/synthesizing process on the main image information.
  • the synthesized image information having the additional sub-information with plural gradation levels embedded in an invisible state in the main image information can be formed with respect to analog data to be output to a recording medium.
  • the synthesized image information can be maintained.
  • a recording material with the high degree of security can be formed.
  • FIG. 23 is a diagram showing an example of the configuration of an image processing system 201 according to the second embodiment.
  • the image processing system 201 is an apparatus which issues an ID card as a recording material P having a face image for person authentication recorded thereon, for example.
  • portions which are the same as those of the image processing system 1 shown in FIG. 1 are denoted by the same reference symbols and the explanation thereof is omitted.
  • the image processing system 201 includes an information input device 11 and image processing device 212. Further, as shown in FIG. 23, the image processing device 212 of the image processing system 201 includes a memory 20, information input section 21, first pre-processing section 224, second pre-processing section 225, electronic watermark embedding section 22, post-processing section 226 and image output section 23.
  • the first pre-processing section 224 performs a first preset pre-process for main image information input from the information input section 21.
  • the first pre-process is explained later in detail.
  • the second pre-processing section 225 performs a second preset pre-process for main image information which is processed by the first pre-processing section 224.
  • the second pre-process is explained later in detail.
  • the post-processing section 226 performs a preset post-process for synthesized image information formed by the electronic watermark embedding section 22.
  • the post-process is explained later in detail.
  • FIG. 24 is a diagram for illustrating the flow of a process of the image processing system 201 for forming a recording material P on which synthesized image information having sub-information embedded in main image information is recorded.
  • portions which are the same as those in the process of FIG. 2 are denoted by the same reference symbols and the detail explanation thereof is omitted.
  • the image processing system 201 performs an information input process for inputting information such as main image information 101, sub-information 102 and a plurality of key information items 103 (103a, 103b, ..., 103n) used to form an ID card by use of the information input device 11 and information input section 21 (step S10).
  • the first pre-processing section 224 When information is input in the information input process, the first pre-processing section 224 performs a first pre-process for the main image information 101 (step S31).
  • the first pre-processing section 224 performs a first pre-process corresponding to an image forming process of the recording process which will be described later for the main image information 101 input in the information input process.
  • a thinning-out (invalidating) process is performed for the main image information 101.
  • the second pre-processing section 225 performs a second pre-process for main image information subjected to the first pre-process by the first pre-processing section 224 (step S32).
  • the second pre-process is to perform a geometrical conversion process for the main image information formed by the first pre-process and subjected to the first pre-process.
  • the second pre-process is to perform a rotating process for the main image information subjected to the first pre-process and remove the pixel portions thinned out in the first pre-process so as to compress the size of the effective pixels, for example.
  • the main image information subjected to the second pre-process is used as embedding-use image information 101' in which the sub-information 102 is embedded.
  • the electronic watermark embedding section 22 performs the electronic watermark embedding process for embedding the sub-information 102 in the embedding-use image information (main image information subjected to the second pre-process) 101' by use of a plurality of key information items 103a to 103n (step S11).
  • the electronic watermark embedding process the same process as that explained in the first embodiment is performed (steps S12a to S15).
  • the post-processing section 226 performs a post-process for the synthesized image information formed by the electronic watermark embedding section 22 (step S23).
  • the post-process is a process for converting the synthesized image information into information (recording image information) 231 to be recorded on a recording medium.
  • the post-process performs an inverse rotating process for the synthesized image information and then adds the pixel portion which has been removed in the second pre-process to expand the effective pixel size.
  • the image output section 23 performs an output process for outputting recording image information to the recording device 13 (step S34).
  • the recording device 13 performs a recording process for recording the recording image information on a recording medium used as a recording material P (step S35).
  • recording image information output from the image output section 23 to the recording device 13 is color information in which each pixel is expressed by R (red), G (green) or B (blue). If the recording image information is received, the recording device 13 converts R (red), G (green) and B (blue) of each pixel in the image information into C (cyan), M (magenta) and Y (yellow).
  • the color conversion is performed according to the characteristic of the recording device. For example, the color conversion is performed based on a preset set parameter (3 ⁇ 3 color conversion matrix, 3 ⁇ 9 color conversion matrix or look-up table).
  • the recording device 13 forms a control signal to control the recording device.
  • the recording device 13 is a printer of melting type thermal transfer recording system
  • the recording device 13 generates a drive pulse signal or drive voltage control signal for a thermal head used as the recording device as a control signal. Further, the recording device 13 performs the heat control operation for the thermal head.
  • the recording device 13 alternately forms even-numbered pixels and odd-numbered pixels for each recording line in the main scanning direction of the thermal head based on the image information converted into C, M, Y for a recording medium.
  • the recording device 13 records synthesized image information converted into recording image information by the above post-process on a recording medium.
  • FIG. 25 is a view showing an example of the arrangement of dots formed on the recording medium by the recording device 13.
  • the dots on the A-A' line in FIG. 25 are linearly arranged in a direction of 45° with a pitch d (1/ ⁇ 2 of the pitch of heating elements of the thermal head) instead of being arranged on every other dot.
  • FIG. 26 is a flowchart for illustrating the flow of the process for restoring sub-information.
  • the process shown in FIG. 26 is performed by the electronic watermark restoring apparatus.
  • the electronic watermark restoring apparatus is realized by a computer to which an image reading device and display device are connected.
  • the processes which will be described later are performed by causing the computer to execute various types of processing programs.
  • the electronic watermark restoring apparatus first performs an image input process for inputting synthesized image information recorded on the recording medium P (step S41).
  • the image input process is performed by, for example, permitting the electronic watermark restoring apparatus to optically read synthesized image information recorded on the recording material P by use of an image reading device, convert the thus read information into a digital form and input the digital image information.
  • the restoring apparatus sets key information used to restore sub-information from the input synthesized image information (step S42).
  • key information key information used in the electronic watermark restoring apparatus is set. Therefore, in the restoring process, it is necessary to specify key information. For example, when a plurality of key information items are separately and selectively used, one of the key information items which is used is determined based on information previously related to the recording material P. Further, when the key information is single key information, information of a specified frequency of the key information may be held in a table in the restoring apparatus and information in the table may be set as the key information setting process.
  • the electronic watermark restoring apparatus performs a restoring process for restoring sub-information as electronic watermark information in the input image information (step S43).
  • the sub-information restoring process may be attained by performing the frequency detecting process and re-structuring process explained in the first embodiment. For example, for frequency filtering by use of a specified frequency of the set key information, the FFT operation or digital frequency filter can be used.
  • the electronic watermark restoring apparatus When sub-information is restored by the sub-information restoring process, the electronic watermark restoring apparatus outputs the restored sub-information (step S44). For example, in the electronic watermark restoring apparatus, the restored sub-information is displayed on the display section. As a result, the truth or not of the recording material P can be determined based on the output result of the restored sub-information (for example, sub-information, displayed on the display section). The electronic watermark restoring apparatus may determine the truth or not of the recording material P according to whether or not the restored sub-information coincides with desired sub-information and output the result of determination.
  • the melting type thermal transfer recording system an image is formed according to the presence or not of dots.
  • the apparent concentration is controlled by performing an area modulating process for changing the areas of dots. Therefore, in the melting type thermal transfer recording system, it is required to precisely modulate the dot size. Thus, in the melting type thermal transfer recording system, it is preferable to apply the alternate driving/recording system.
  • FIG. 27 is a diagram showing an example of recording image information items arranged in a grid form.
  • FIG. 28 is a diagram showing an image when the image information shown in FIG. 27 is actually recorded by the alternate driving/recording system.
  • the alternate driving/recording system alternately drives odd-numbered heating elements of odd-numbered lines of the recording head (line type thermal head) and even-numbered heating elements of even-numbered lines for each recording line.
  • image information items arranged in a grid form as shown in FIG. 27 are actually arranged in a zigzag form as shown in FIG. 28 and formed as an image on a recording medium. Therefore, in the alternate driving/recording system, even-numbered information items of odd-numbered lines of image information recorded in practice and odd-numbered information items of even-numbered lines thereof are lost.
  • the sub-information (electronic watermark information) embedded in the synthesized image information is destroyed or modified. That is, in the recording device of the alternate driving system, even if sub-information is simply embedded in' an invisible state in image information to be recorded by use of the electronic watermark process, only one half of the area of the original image information becomes effective and the other information is lost. Generally, if the electronic watermark information is destroyed as described above, it becomes extremely difficult to restore the sub-information from the synthesized image information. Therefore, in the recording medium on which the synthesized image information is recorded, the high degree of security cannot be maintained.
  • the main image information 101 is subjected to the first pre-process by the first pre-processing section 224 and the second pre-process by the second pre-processing section 225 to form an embedding-use image 101' before the electronic watermark embedding process by the electronic watermark embedding section 22 is performed. Further, in the image processing section 201, a post-process by the post-processing section 226 is performed after the electronic watermark embedding process is performed by the electronic watermark embedding section 22. Thus, in the image processing system 201, the electronic watermark is prevented from being destroyed at the alternate driving/recording time.
  • FIG. 29 is a diagram showing an example of the arrangement of whole image information items to be recorded.
  • each black portion 251 corresponds to a pixel to be recorded (information which is not thinned out) and each white portion 252 corresponds to a pixel not to be recorded (information which is thinned out).
  • FIG. 30 is a diagram showing a state in which the image information shown in FIG. 29 is rotated by 45°. As shown in FIG. 30, if the image information shown in FIG. 29 is rotated by 45°, the black portions 251 (information items which are not thinned out) are linearly arranged. In the second pre-process, the white portions 252 (information items which are thinned out) shown in FIG. 30 are removed and re-arranged. Thus, embedding-use image information 101' including only image information that is not influenced by the alternate driving/recording system is formed.
  • FIG. 31 is a diagram showing an example of the arrangement of image information such as main image information to be recorded.
  • FIG. 32 is a diagram showing a state in which the image information of FIG. 31 is subjected to the first pre-process.
  • FIGS. 33 and 34 are diagrams each showing a state of image information in a corresponding step of the second pre-process.
  • FIG. 33 is a diagram showing image information set in a state in which the image information set in the state of FIG. 32 is rotated by 45°.
  • FIG. 34 is a diagram showing image information attained by re-arranging pixels of effective image information in the image information set in the state of FIG. 33.
  • the first pre-processing section 224 removes pixels having an x mark as shown in FIG. 32 (even-numbered pixels of odd-numbered lines and odd-numbered pixels of even-numbered lines).
  • the second pre-processing section 225 first, a rotating process is performed for image information subjected to the first pre-process by the first pre-processing section 224. For example, if image information shown in FIG. 32 is given as the result of the first pre-process, the second preprocessing section 225 subjects the image information to the rotating process of 45° as shown in FIG. 33. Further, the second pre-processing section 225 rearranges the effective pixels in the image information subjected to the rotating process to form image information having the same size as that of the original image information.
  • the second pre-processing section 225 rearranges pixels of the effective image information other than portions having the x mark shown in FIG. 33 as shown in FIG. 34. Then, it stores pixel values (in this case, "0") as information indicating no recording in arrangement pixels of vacant spaces.
  • an area of the effective pixels is subjected to the electronic watermark embedding process by the electronic watermark embedding section 22.
  • the electronic watermark embedding section 22 performs the electronic watermark embedding process for the pixels in the area surrounded by the thick line as shown in FIG. 34.
  • the post-processing section 226 After the electronic watermark embedding process, the post-processing section 226 performs the post-process for image information in which the electronic watermark is embedded.
  • the post-process by the post-processing section 226 performs a process which is set in a reverse direction with respect to the second pre-process.
  • the post-processing section 226 subjects the image information to a conversion process so as to set the arrangement of the effective pixels (pixels in the area surrounded by the thick line in FIG. 34) of the image information to the arrangement shown in FIG. 33. Further, it rotates the image information subjected to the conversion process by 45° in a direction opposite to that in the second pre-process.
  • Image information obtained by the post-process is output as recording image information to the recording device 13 from the image output section 23.
  • the image processing operation is not limited to the image processing system having a recording apparatus of the melting type thermal transfer recording system, but can be applied to an image processing system having a recording apparatus of a recording system for making gradation representation by modulation of dot areas of recording pixels.
  • FIG. 35 is a diagram for illustrating an example of the whole processing procedure of the image processing system 201.
  • main image information 101 is face image information for person authentication.
  • sub-information 102 is information (in this example, a numeral "174") which is used to enhance the degree of security of the main image information 101.
  • the sub-information 102 an image obtained by coding a name and date of birth or a pattern of a logo mark of a company can be used.
  • a plurality of key information items 103a to 103n are used to embed the sub-information 102 in the main image information 101. Further, the key information items 103a to 103n are also used as information items for restoring the sub-information embedded in an invisible state from the synthesized image information recorded on a recording medium.
  • the main image information 101, sub-information 102 and a plurality of key information items 103a to 103n are input by means of the information input section 21.
  • the first pre-processing section 224 and second pre-processing section 225 perform the first and second pre-processes for the main image information 101 to form embedding-use image information 101'.
  • the electronic watermark embedding section 22 performs the electronic watermark embedding process by use of the embedding-use image information 101', sub-information 102 and a plurality of key information items 103a to 103n.
  • the post-processing section 226 performs the post-process as the inverse transform process with respect to the second pre-process for the image information 231'.
  • recording image information 231 as the synthesized image information having the sub-information 102 embedded therein in an invisible state is formed.
  • the recording image information 231 is output to the recording device 13 from the information output section.
  • the recording device 13 performs the recording process to record the recording image information 231 as the synthesized image information having the sub-information 102 embedded therein in the invisible state on a recording medium by the alternate driving system.
  • a recording material P is completed.
  • sub-information is embedded in main image information subjected to the pre-process by use of a plurality of key information items and the thus obtained image information is post-processed and recorded on a recording medium.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Databases & Information Systems (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Data Mining & Analysis (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Image Processing (AREA)
  • Editing Of Facsimile Originals (AREA)
  • Facsimile Image Signal Circuits (AREA)
  • Record Information Processing For Printing (AREA)
  • Color Image Communication Systems (AREA)
  • Electrical Discharge Machining, Electrochemical Machining, And Combined Machining (AREA)
  • Ultra Sonic Daignosis Equipment (AREA)
  • Polarising Elements (AREA)
EP04030177A 2003-12-22 2004-12-20 Bildverarbeitungsvorrichtung Not-in-force EP1548642B1 (de)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2003424329A JP4167590B2 (ja) 2003-12-22 2003-12-22 画像処理方法
JP2003424329 2003-12-22

Publications (3)

Publication Number Publication Date
EP1548642A2 true EP1548642A2 (de) 2005-06-29
EP1548642A3 EP1548642A3 (de) 2007-03-28
EP1548642B1 EP1548642B1 (de) 2011-03-23

Family

ID=34544920

Family Applications (1)

Application Number Title Priority Date Filing Date
EP04030177A Not-in-force EP1548642B1 (de) 2003-12-22 2004-12-20 Bildverarbeitungsvorrichtung

Country Status (7)

Country Link
US (1) US7577271B2 (de)
EP (1) EP1548642B1 (de)
JP (1) JP4167590B2 (de)
KR (1) KR100701907B1 (de)
AT (1) ATE503235T1 (de)
DE (1) DE602004031922D1 (de)
TW (1) TWI281617B (de)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2007032758A1 (en) * 2005-09-09 2007-03-22 Thomson Licensing Video watermarking
WO2007032760A1 (en) * 2005-09-09 2007-03-22 Thomson Licensing Coefficient modification for video watermarking
US20090136083A1 (en) * 2005-09-09 2009-05-28 Justin Picard Coefficient Selection for Video Watermarking
US20090252370A1 (en) * 2005-09-09 2009-10-08 Justin Picard Video watermark detection

Families Citing this family (18)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7573491B2 (en) * 2004-04-02 2009-08-11 David Hartkop Method for formatting images for angle-specific viewing in a scanning aperture display device
JP2006224665A (ja) * 2005-01-21 2006-08-31 Seiko Epson Corp 印刷装置
JP4660212B2 (ja) * 2005-01-24 2011-03-30 株式会社東芝 画像処理装置および画像処理方法
JP2006287868A (ja) * 2005-04-05 2006-10-19 Sharp Corp 画像形成装置
JP4093262B2 (ja) * 2005-10-06 2008-06-04 コニカミノルタビジネステクノロジーズ株式会社 画像形成装置
JP4872317B2 (ja) * 2005-11-10 2012-02-08 富士ゼロックス株式会社 情報処理システム
JP4550749B2 (ja) 2006-03-07 2010-09-22 株式会社東芝 画像処理方法および画像処理装置
JP2008085695A (ja) * 2006-09-28 2008-04-10 Fujitsu Ltd 電子透かし埋め込み装置および検出装置
JP4922205B2 (ja) * 2007-08-17 2012-04-25 株式会社東芝 画像処理方法および画像処理装置
JP2009088614A (ja) 2007-09-27 2009-04-23 Toshiba Corp 画像処理方法および画像処理装置
JP4901675B2 (ja) * 2007-09-28 2012-03-21 株式会社東芝 画像処理方法および画像処理装置
US20090323125A1 (en) * 2008-06-27 2009-12-31 Kabushiki Kaisha Toshiba Image generating apparatus
JP2010136206A (ja) * 2008-12-05 2010-06-17 Toshiba Corp 画像処理方法及び画像処理装置
RU2495494C2 (ru) * 2011-10-12 2013-10-10 Федеральное государственное автономное образовательное учреждение высшего профессионального образования "Национальный исследовательский университет "Высшая школа экономики" Способ внесения дополнительной информации в цифровые графические изображения (варианты)
JP5791082B2 (ja) * 2012-07-30 2015-10-07 国立大学法人横浜国立大学 画像合成装置、画像合成システム、画像合成方法およびプログラム
TWI625715B (zh) * 2016-05-31 2018-06-01 瑞鼎科技股份有限公司 顯示驅動裝置及其運作方法
JP7155737B2 (ja) * 2018-08-10 2022-10-19 株式会社リコー 画像処理装置、画像処理システム、プログラムおよび画像処理方法
US12093405B2 (en) 2022-02-14 2024-09-17 Hiwin Technologies Corp. Method and system for automatically loading parameters, and client-end server thereof

Family Cites Families (23)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH0659739A (ja) 1992-08-07 1994-03-04 Sumitomo Heavy Ind Ltd 二重テーブル旋回装置
JP3547892B2 (ja) 1996-03-14 2004-07-28 株式会社東芝 画像記録装置及び画像記録方法
US7130442B2 (en) * 1996-10-28 2006-10-31 International Business Machines Corporation Protecting images with an image watermark
DE69840308D1 (de) * 1997-02-24 2009-01-22 Nec Corp System zur Kodierung digitaler Daten
DE69822950T2 (de) * 1997-09-03 2009-09-24 Hitachi, Ltd. Verfahren und System zum Einbetten von Information in Daten
EP0921675B1 (de) * 1997-12-03 2006-07-05 Kabushiki Kaisha Toshiba Verfahren zur Bildinformationsverarbeitung und Verfahren zur Verhinderung von Fälschungen von Zertifikaten und dergleichen
US5946414A (en) * 1998-08-28 1999-08-31 Xerox Corporation Encoding data in color images using patterned color modulated image regions
US6665417B1 (en) * 1998-12-02 2003-12-16 Hitachi, Ltd. Method of judging digital watermark information
US7142689B2 (en) * 2000-01-31 2006-11-28 Canon Kabushiki Kaisha Image processing apparatus for determining specific images
JP4495824B2 (ja) * 2000-03-21 2010-07-07 株式会社東芝 情報処理方法
JP3666570B2 (ja) * 2000-03-30 2005-06-29 松下電器産業株式会社 コピー防止方法、及びデジタル放送受信装置
JP4218920B2 (ja) * 2000-04-07 2009-02-04 キヤノン株式会社 画像処理装置及び画像処理方法並びに記憶媒体
JP2003025728A (ja) 2001-07-18 2003-01-29 Toshiba Corp 熱転写記録方法および熱転写インクリボン
US7298865B2 (en) * 2001-07-30 2007-11-20 Sarnoff Corporation Secure robust high-fidelity watermarking
JP2003115990A (ja) 2001-10-05 2003-04-18 Mitsubishi Electric Corp 電子透かし埋込み方法、電子透かし検出方法
US20030210803A1 (en) * 2002-03-29 2003-11-13 Canon Kabushiki Kaisha Image processing apparatus and method
JP4143441B2 (ja) * 2002-04-24 2008-09-03 キヤノン株式会社 情報処理方法及び装置、並びにコンピュータプログラム及びコンピュータ可読記憶媒体
AUPS206802A0 (en) * 2002-05-01 2002-06-06 Canon Kabushiki Kaisha Steganographic image encoding
CA2435659A1 (en) * 2002-07-23 2004-01-23 Kabushiki Kaisha Toshiba Image processing method
JP4227048B2 (ja) * 2003-04-25 2009-02-18 株式会社東芝 画像処理システム
KR20030055189A (ko) * 2003-04-29 2003-07-02 김은수 홀로그램 워터마크 삽입 및 검출 방법과 그 장치
US7197161B2 (en) * 2003-06-20 2007-03-27 Xerox Corporation Embedding information in images using two-layer conjugate screening
US7245740B2 (en) * 2003-07-01 2007-07-17 Oki Electric Industry Co., Ltd. Electronic watermark embedding device, electronic watermark detection device, electronic watermark embedding method, and electronic watermark detection method

Non-Patent Citations (6)

* Cited by examiner, † Cited by third party
Title
FLEET D J ET AL.: "EMBEDDING INVISIBLE INFORMATION IN COLOR IMAGES", PROCEEDINGS OF THE INTERNATIONAL CONFERENCE ON IMAGE PROCESSING, 1 October 1997 (1997-10-01)
HEE-SOO KIM ET AL.: "Digital watermarking based on color differences", PROCEEDINGS OF THE SPIE - THE INTERNATIONAL SOCIETY FOR OPTICAL ENGINEERING SPIE-INT. SOC. OPT. ENG USA, vol. 4314, 2001
LI X. ET AL.: "ROBUST SPATIAL DATA HIDING FOR COLOR IMAGES", INT. CONF. ON COMMUNICATIONS AND MULTIMEDIA SECURITY, 2 October 2003 (2003-10-02), pages 205 - 213
RUDAZ N. ET AL.: "PROTECTING IDENTITY DOCUMENTS WITH A JUST NOTICABLE MICROSTRUCTURE", PROCEEDINGS OF THE SPIE, vol. 4677, 2002
WAN ADNAN W A ET AL.: "A view of image watermarking", STUDENT CONFERENCE ON RESEARCH AND DEVELOPMENT SCORED, 25 August 2003 (2003-08-25), pages 381 - 384
XIAOQIANG LI ET AL.: "Multi-channel data hiding scheme for color images", INFORMATION TECHNOLOGY: CODING AND COMPUTING, PROCEEDINGS. ITCC 2003. INTERNATIONAL CONFERENCE, 28 April 2003 (2003-04-28), pages 569 - 573

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2007032758A1 (en) * 2005-09-09 2007-03-22 Thomson Licensing Video watermarking
WO2007032760A1 (en) * 2005-09-09 2007-03-22 Thomson Licensing Coefficient modification for video watermarking
US20090136083A1 (en) * 2005-09-09 2009-05-28 Justin Picard Coefficient Selection for Video Watermarking
US20090252370A1 (en) * 2005-09-09 2009-10-08 Justin Picard Video watermark detection

Also Published As

Publication number Publication date
DE602004031922D1 (de) 2011-05-05
US20050134622A1 (en) 2005-06-23
US7577271B2 (en) 2009-08-18
JP4167590B2 (ja) 2008-10-15
KR20050063746A (ko) 2005-06-28
TWI281617B (en) 2007-05-21
EP1548642B1 (de) 2011-03-23
TW200535632A (en) 2005-11-01
JP2005184603A (ja) 2005-07-07
EP1548642A3 (de) 2007-03-28
ATE503235T1 (de) 2011-04-15
KR100701907B1 (ko) 2007-04-02

Similar Documents

Publication Publication Date Title
EP1548642B1 (de) Bildverarbeitungsvorrichtung
JP4227048B2 (ja) 画像処理システム
US7489800B2 (en) Image processing method
KR101023616B1 (ko) 화상 처리 방법 및 화상 처리 장치
KR100864997B1 (ko) 이미지 처리 방법 및 디바이스
JP2006222788A (ja) 画像検証装置，画像埋め込み装置，画像検出装置，画像埋め込み方法，画像検出方法，コンピュータプログラム，および印刷物製造方法
JP2005159438A (ja) 画像処理方法
JP4686578B2 (ja) 画像処理方法
JP4746663B2 (ja) 画像処理装置および画像処理方法
JP4088191B2 (ja) 画像処理方法および画像記録装置
JP2008252786A (ja) 画像処理装置、画像処理方法および記録物
JP2009065364A (ja) 画像処理方法、画像処理装置および記録物
CA2545472A1 (en) Image processing method
Malviya et al. Survey on Transform Domain Secure Watermarking Algorithm for Color Images
JP2010011446A (ja) 画像処理装置

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

17P Request for examination filed

Effective date: 20041220

AK Designated contracting states

Kind code of ref document: A2

Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IS IT LI LT LU MC NL PL PT RO SE SI SK TR

AX Request for extension of the european patent

Extension state: AL BA HR LV MK YU

PUAL Search report despatched

Free format text: ORIGINAL CODE: 0009013

AK Designated contracting states

Kind code of ref document: A3

Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IS IT LI LT LU MC NL PL PT RO SE SI SK TR

AX Request for extension of the european patent

Extension state: AL BA HR LV MK YU

17Q First examination report despatched

Effective date: 20070627

AKX Designation fees paid

Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IS IT LI LT LU MC NL PL PT RO SE SI SK TR

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IS IT LI LT LU MC NL PL PT RO SE SI SK TR

REG Reference to a national code

Ref country code: GB

Ref legal event code: FG4D

REG Reference to a national code

Ref country code: CH

Ref legal event code: EP

REG Reference to a national code

Ref country code: IE

Ref legal event code: FG4D

REF Corresponds to:

Ref document number: 602004031922

Country of ref document: DE

Date of ref document: 20110505

Kind code of ref document: P

REG Reference to a national code

Ref country code: DE

Ref legal event code: R096

Ref document number: 602004031922

Country of ref document: DE

Effective date: 20110505

REG Reference to a national code

Ref country code: NL

Ref legal event code: VDEP

Effective date: 20110323

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20110323

Ref country code: GR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20110624

Ref country code: LT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20110323

LTIE Lt: invalidation of european patent or patent extension

Effective date: 20110323

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: CY

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20110323

Ref country code: BG

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20110623

Ref country code: FI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20110323

Ref country code: AT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20110323

Ref country code: SI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20110323

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: BE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20110323

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: PT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20110725

Ref country code: EE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20110323

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: CZ

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20110323

Ref country code: RO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20110323

Ref country code: SK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20110323

Ref country code: IS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20110723

Ref country code: ES

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20110704

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: NL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20110323

PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT

26N No opposition filed

Effective date: 20111227

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: PL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20110323

Ref country code: DK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20110323

REG Reference to a national code

Ref country code: DE

Ref legal event code: R097

Ref document number: 602004031922

Country of ref document: DE

Effective date: 20111227

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20110323

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MC

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20111231

REG Reference to a national code

Ref country code: CH

Ref legal event code: PL

GBPC Gb: european patent ceased through non-payment of renewal fee

Effective date: 20111220

REG Reference to a national code

Ref country code: IE

Ref legal event code: MM4A

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LI

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20111231

Ref country code: GB

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20111220

Ref country code: CH

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20111231

Ref country code: IE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20111220

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LU

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20111220

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: TR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20110323

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: HU

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20110323

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 12

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 13

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 14

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: FR

Payment date: 20211115

Year of fee payment: 18

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: DE

Payment date: 20220622

Year of fee payment: 19

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: FR

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20221231

REG Reference to a national code

Ref country code: DE

Ref legal event code: R119

Ref document number: 602004031922

Country of ref document: DE

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: DE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20240702