WO1998015114A1 - A method for color correction in color facsimiles - Google Patents

A method for color correction in color facsimiles

Info

Publication number
WO1998015114A1
WO1998015114A1 PCT/US1997/017306 US9717306W WO1998015114A1 WO 1998015114 A1 WO1998015114 A1 WO 1998015114A1 US 9717306 W US9717306 W US 9717306W WO 1998015114 A1 WO1998015114 A1 WO 1998015114A1
Authority
WO
Grant status
Application
Patent type
Prior art keywords
color
device
data
image
sending
Prior art date
Application number
PCT/US1997/017306
Other languages
French (fr)
Inventor
Glenn Walter CREPPS
Robert Del Widergren
John Lewis Douglas
Original Assignee
Compressent Corporation
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date

Links

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N1/00Scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission; Details thereof
    • H04N1/46Colour picture communication systems
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N1/00Scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission; Details thereof
    • H04N1/46Colour picture communication systems
    • H04N1/64Systems for the transmission or the storage of the colour picture signal; Details therefor, e.g. coding or decoding means therefor

Abstract

Color image information (123) is transmitted with color translation information (130S) in a data stream (101) from a sending device (100) to a remote receiving device (150). The color translation information contains information including the color space (121) of the sending device (100), information about nonlinearities in the color space (124) of the sending device (100), and optionally the gamut (122) of the sending device (100). The remote receiving device (150) uses this information (130SR) to accurately reproduce the color image on the receiving device (150). Alternatively, color image data (123) is transformed into color image data corresponding to an intermediate color space at the sending device (100). The intermediate color image data is transmitted to a remote receiving device (150). The remote receiving device (150) uses information about its own color space to transform the intermediate color image data space into corrected color image data in its own color space. The corrected color image can then be printed or displayed on the remote receiving device (150).

Description

A METHOD FOR COLOR CORRECTION IN COLOR FACSIMILES

FIELD OF INVENTION

This invention relates to color facsimile transmission systems, and in particular to color- correction of color facsimiles.

BACKGROUND OF THE INVENTION

Accurate replication of colors observed in the physical realm is recognized as difficult, but essential to many areas of technology. A color viewed in natural daylight usually appears to be a slightly different shade when viewed indoors under fluorescent light.

Considerable effort has been devoted to achieving color consistency between reproduced colors and colors observed in the physical world. Nevertheless, accurate reproduction of color is an ongoing problem. This is particularly true in desktop publishing, for example, where there is an inconsistency in color reproduction across devices, and sometimes even across devices in the same manufacturer product line.

Prior to desktop publishing, color image reproduction systems were proprietary, and though color images varied between devices, the manufacturer could ensure color consistency. A single vendor developed and serviced the entire color image reproduction system that typically included a high-end scanner, an imagesetter, and associated software.

Unfortunately, most desktop color reproduction systems are made using equipment and software from a variety of sources. Different devices use different technologies in color reproduction.

One fundamental problem in color reproduction is that color monitors and other sources of transmitted light use a Red-Green-Blue (RGB) color space while devices that depend on reflected light use a Cyan- Magenta-Yellow (CMY) color space. The gamut, i.e., the colors that are reproducible in a particular process, of CMY-based process printing is much smaller than the gamut of a RGB monitor, for example. Each device, scanner, monitor, color printer, etc. has its own color gamut. Since the gamut of different devices do not match (indeed, the color space and gamut varies across serial numbers for the same manufacturer part number) , the colors of an image or document do not match when the image or document is sent from a scanning device having one color space and one gamut to an output device having a different color space and gamut. Color management systems have been developed for devices at a single site to resolve nonlinearities in the color space and color gamut differences for limited applications. A color management system compares the color spaces for two known devices . The color management system records the differences between the two color gamuts and uses the information to transform the colors from one color space to the other. Consequently, the colors are represented as closely as the two devices allow. However, typically, the software that controls these color management systems knows what colors are available and what transformations are required to render an accurate output color image .

While color management systems have been applied to monitors, scanners, and printers on a desktop, i.e., to devices at a single site, color facsimile transmission to remote sites presents additional problems . For example, at a single site, devices could be calibrated so that a color image on one device could be reproduced accurately on a second device, limited only by the gamut of the second device. This calibration becomes impractical and generally impossible for communication with remote devices whose color space and gamut are unknown and thus cannot be taken into account by the sending device so that the image is accurately reproduced on a receiving device at the remote site. Accordingly, conventional desk top color management systems are not practicable for color facsimile transmissions to remote-site systems.

Nevertheless, some standards have been developed to support the transmission of color images between machines at different sites. For example, the method used to transmit data between two color facsimile devices is fixed so that a color facsimile machine of one vendor can communicate with a color facsimile machine of another vendor. Since color facsimile transmission is still frame, color facsimile machines encode a digital bit stream representing a scanned color image (i.e., raw data) using the JPEG standard, and transmit the data according to the format specified by JPEG. JPEG is defined in International Standards

Organization (ISO) Document ISO/IEC 18918-1 and in The International Telecommunication Union (ITU) , in Recommendation ITU-T/T81.

JPEG allows data images and other information to be transferred between sending and receiving sites, and later reproduced on different applications and platforms. JPEG accomplishes this by requiring that specific, identifiable information be included with the data, including Huffman or discrete cosine transform (DCT) tables, though JPEG does not specify how to implement the DCT coder. Under the JPEG standard, data is transmitted in a "frame" or a sequence of frames. Each frame is constructed from one or more scans through the image, and is defined by the encoded color image coefficients, as discussed below, that represent the color image. Using JPEG, each frame or group of frames are compressed into packets of data using well-known techniques . The present invention is not limited to the JPEG standard, and supports data standards that perform other operations on the color image data, such as performing a unitary operation, before transmitting the color image data.

The data is transmitted in a data bit stream, which includes data segments and marker segments. Data segments contain, for example, compressed color image coefficients . Marker segments contain information used to decode the compressed color image coefficients, as explained in more detail below.

The various segments in the data stream are defined by markers. Marker segments, which contain data used by the receiving device and applications programs, are identified by a unique two-byte marker. The first byte, which is xFF, identifies the two bytes as a marker segment. The second byte identifies the information contained in the appended segment, e.g., a comment or a Huffman table. For example, xFFC4 identifies an appended Huffman table; xFFFE identifies an appended comment field; xFFDB identifies an appended quantization table; and xFFEO through xFFEF identify appended segments containing information reserved for applications use. A list of these two-byte markers is presented in W. Pennebaker and J. Mitchell, JPEG, Still Image Data Compression Standard ( "JPEG" ) at page 107 (1993 Van Nostrand Reinhold) . The marker segments may precede a frame header or a scan header. The headers for each frame or scan contain information used to decode the color image data, and are recognized by the decoder without being decoded themselves. A frame header contains, for example, a marker identifying the header as a frame header, a header length, the number of lines in the frame, as well as an identifier for the particular Huffman code table needed to decode the frame data. Different Huffman tables may be required for different frames since the occurrences of symbols used to represent a sequence of color image coefficients which uniquely define the color image usually differ from frame to frame .

Annex I of ITU-T T.4 specifies a JPEG interchange format for Group 3 facsimile. Annex B of ITU-T T.503 specifies a different JPEG interchange format for Group 4 facsimile. Both interchange formats are part of the world-wide communication standards for color facsimiles, which are established by the International Telecommunications Union (ITU) in the form of "Recommendations" for color facsimile transmissions over the world-wide telephone system. The following ITU-T Recommendations set the de facto standards for all switched telephone network facsimile communications, commonly known as Group 3 Facsimile (for analog lines) and Group 4 Facsimile (for digital lines) :

Recommendation Information in Recommendation

T-4 Standardization of Group 3 Facsimile Apparatus for Document

Transmission

T-4 Amend 2 Amendment to Above T-23 Standardized Colour Test Chart for

Document Facsimile

T-30 Procedures for Document Facsimile

Transmission in the General Switched Telephone Network T-30 Amend 1 Amendment to Above (Annex E . - Procedure for the G3 document facsimile transmission of continuous-tone colour images)

T-30 Amend 2 Amendment to Above T-35 CCITT Defined Codes for Non- Standard Facilities

T-42 Continuous-Tone Colour Representation Method for Facsimile (CIELAB)

T-81 Color Image Compression (Commonly known as JPEG)

T-434 Binary File Transfer Format

T-434 Amend 1 Amendment to Above

T-503 Document Application file for the Interchange of Group 4 Facsimile

T-503 Annex B Extension for Continuous-Tone Colour and Grey-Scale Image Documents T-503 Amend 2 Amendment to Above

The color extensions to the ITU specification for Group 3 and Group 4 facsimile recognized the problems associated with transmitting a color image between two sites and obtaining an accurate reproduction of the color image at the receiving site. The color extensions define some flexibility in attempting to provide color correction capability.

Specifically, the CIELAB color space components L*, a* and b* are used to represent each pixel in the color image . Component L* signifies "lightness" while components a* and b* relate to the standard CIE stimulus values X and Y. Color data in this space are acquired under a particular illuminant, and determined from spectral or colormetric data using a particular white point . The basic illuminant is CIE standard illuminant D50. The white point is the perfectly diffuse reflector associated with the standard illuminant D50.

According to the color extensions, a receiving color facsimile machine provides an indication of the capabilities of the receiving device to the sending color facsimile machine, e.g., JPEG, custom illuminant, and custom gamut. For devices with the custom illuminant and custom gamut capability, data for scaling the gamut range data and illuminant data are transmitted to the receiving color facsimile machine. This data allows the receiving color facsimile machine to linearly scale the gamut range of color data for the particular receiving color facsimile machine. However, many color effects are non-linear, and so the resulting image is an accurate reproduction of the original image only if the non-linear color effects are of no significance .

Unfortunately, non-linear color effects are often of significance, and when these non-linear color effects are not taken into account, the displayed color image is an inaccurate reproduction of the original color image .

Real devices have non-linear color characteristics . An ideal device, e.g., an ideal color scanner, would have a color space whose output color coefficients, i.e., the normalized coefficients which determine the colors of the image output by the scanner, vary linearly with the input coefficients, i.e., the normalized coefficients which determine the colors of the image input into the scanner. In real devices, however, the output color coefficients do not vary linearly with the input color coefficients.

Because color facsimile machines do not correct inherent nonlinearities in the color coefficients received from other color facsimile machines, color images reproduced using the color coefficients are inaccurate. Consequently, a method is needed to overcome the color reproduction limitations inherent in JPEG and the problems associated with the numerous different devices that are used for color facsimile transmission.

SUMMARY OF THE INVENTION

According to the principles of this invention, in one embodiment, digital color images are transmitted from a first device to a second device utilizing the JFIF standard, and the second device accurately recreates the digital color image sent by the first device. The accurate reproduction is obtained even though the first device has no prior knowledge of the color characteristics of the second device, and conversely. All of the information needed by the second device to accurately recreate the color image sent by the first device is included within the data bit stream provided to the second device. In this embodiment of the invention, the sending device processes, e.g., compresses, the color image data and transmits the processed color image data along with a characterization of the color performance of the sending device that includes any non-linear effects. The characterization of the color performance of the sending device includes non-linear color correction data, e.g., a transformation that removes non-linear effects from the color image data to produce linearized color image data, and a linear color space transformation that converts the linearized color image data from a native color space of the sending device to an intermediate color space .

Thus, the transmitted data bit stream includes compressed color image data, and the characterization of the color performance of the sending device that includes any non-linear effects. The receiving device decompresses the color image data, and stores that data and the characterization of the color performance of the sending device . The receiving device uses the decompressed color image data, characterization of the color performance of the sending device, and a characterization of the color performance of the receiving device to produce a color corrected image at the receiving device. The color corrected image can then be printed or displayed on the receiving device. In a second embodiment of this invention, the sending device uses sending device color performance characterization information to remove nonlinear color effects of the sending device from the color image data and to translate the resulting linear color image data in the native color space of the sending device to color image data in an intermediate color space. The sending device then compresses the intermediate color image data and transmits a data stream, which includes the color image data, to the receiving device. The receiving device decompresses the color image data, and uses color performance characterization information to translate the intermediate color image data to color image data in the color space of the receiving device, and then to correct the resulting color image data to account for nonlinear color effects of the receiving device. The corrected color image can then be printed or displayed on the receiving device.

In both of these embodiments the sending and receiving device-dependent color performance characterizations includes both linear and non-linear transformations. Consequently, the unrealistic requirement of relying upon a particular color specification and linearity of color reproduction across a variety of devices has been eliminated. The information included in the sending and receiving device-dependent color performance characterizations allows true color reproduction by the receiving device. The information is sufficient to allow the receiving device to accurately characterize both the linear and non-linear color performance of the sending device. Consequently, the receiving device uses this information to transform the received color image into a color image in the color space for the receiving device, and to compensate for the non- linear characteristics of both devices. In accordance with one embodiment of the present invention, at the sending color facsimile machine, the color image is scanned and transformed into discrete cosine transform (DCT) color coefficients in the sending facsimile machine's color space. The DCT color coefficients are quantized and then Huffman encoded. Other information which determines the colors of the image produced on the sending facsimile machine, i.e, sending device linear color space transformation, and sending device non- linear transformation are stored in the sending facsimile machine's memory. The sending machine transmits a bit stream that includes the encoded image, the sending device linear color space transformation, and the sending device non-linear transformation . At the remote location, the receiving color facsimile machine first parses the incoming bit stream and stores the encoded color image, the sending device linear color space transformation, and the sending device non- linear transformation in the memory of the receiving color facsimile machine. The color image is decoded and operated on by sending device linear and non-linear transformations. The resulting color image is operated on by receiving device linear and nonlinear transformations to obtain a color-corrected image reproducible in the native color space of the receiving device. While four separate operations are described here, this is for clarity only. Typically, the transformation used in the four operations are combined and a single operation is performed on the decoded color image data that has the same affect as the four separate operations.

The resulting color-corrected color image in the native color space of the receiving device is an accurate high-quality reproduction of the original color image that was scanned by the sending device . The color-corrected color image is printed on the receiving color facsimile machine or displayed on another receiving device.

Hence, a method for providing information to a receiving device to improve the accuracy of color reproduction by the receiving device includes transmitting a sending device non-linear transformation in a bit stream to a remote receiving device. A sending device linear color space transformation is also transmitted in the bit stream to the remote receiving device. The sending device non-linear transformation and the sending device linear color space transformation are used by the remote receiving device in generating a color image from information in the bit stream. The sending and receiving devices can be color facsimile machines, or alternatively two independent devices that transmit and reproduce color image data. In one embodiment, the color image data in the bit stream has a structure defined by a JPEG file interchange format. Alternatively, the bit stream structure can have any desired format so long as both the sending and receiving devices recognize the format.

In one embodiment, a first field marker is transmitted in the bit stream immediately preceding the sending device non-linear transformation. Similarly, a second field marker is transmitted in the bit stream immediately preceding the sending device linear color space transformation.

In one embodiment, transmitting a sending device linear color space transformation further comprises transmitting a linear color space transformation from a native color space of the sending device to an intermediate color space. The intermediate color space is a CIE color space.

In this embodiment of the invention, a structure in a buffer memory of a sending device used in transmitting a bit stream including color image data to a remote device includes a color correction field marker; and a color correction field segment data field wherein the color correction field segment data field includes sending device-dependent color performance characterization information. Alternatively, the structure in a buffer memory of a sending device used in transmitting a bit stream including color image data to a remote device includes a sending device non- linear descriptor transformation field marker; and a sending device non-linear transformation immediately following the sending device non- linear color space transformation field marker. This structure, in addition, includes a sending device linear color space transformation field marker; and a sending device linear color space transformation immediately following the sending device linear transformation field marker.

The novel memory and bit stream structures of this invention are utilized in a method for improving remote device color reproduction of a color image generated by a sending device. In the method, color image data is processed in the sending device to generate processed color image data. A bit stream including non- linear color correction data and the processed color image data is transmitted from the sending device. The transmitted non-linear color correction data removes non-linear effects from the color image data to produce linearized color image data. The bit stream is received at the remote receiving device, and a color image is generated using the non-linear color correction data and the processed color image data at the remote receiving device wherein the sending device and the remote device are independent unrelated devices .

BRIEF DESCRIPTION OF THE DRAWINGS Figure 1 is a block diagram of the structures stored in memory of color facsimile machines to implement one embodiment of the present invention.

Figure 2 illustrates a structure of a data bit stream including the novel sending device dependent color performance characterization information of this invention.

Figure 3 illustrates one specific embodiment of structures in a data bit stream including the novel sending device dependent color performance characterization information of this invention that in turn includes a sending device linear color space transformation SM_LINEAR and a sending device nonlinear transformation SM_NONLINEAR.

Figure 4 is a process flow diagram for facsimile color correction according to a first embodiment of this invention.

Figure 5 is a process flow diagram for facsimile color correction according to a second embodiment of this invention.

DETAILED DESCRIPTION OF THE INVENTION

According to the principles of this invention, a sending color facsimile machine 100 (Fig. 1) first scans a color image into a location 123 in memory 120. Facsimile machine 100 uses a particular color space, referred to herein as a native color space, to represent each pixel of scanned color image 123. (Hereinafter, a reference numeral for a memory location is used to identify the information stored in the memory location. Similarly, elements with the same reference numeral are the same element.) For example, each pixel in stored color image 123, in one embodiment, is defined in a red-green-blue (RGB) color space .

According to the principles of this invention, sending device dependent color performance characterization information 130S is stored in memory 120 of sending color facsimile machine 100. As explained more completely below, sending device dependent color performance characterization information 130S includes a transformation to convert color image data in the native color space of machine 100 to color image data in an intermediate color space, i.e., linear color correction data; a native color space non-linear transformation to correct for nonlinear color effects of machine 100, i.e., nonlinear color correction data; and, optionally, a native color space gamut range . Sending device dependent color performance characterization information 130S is known, and is used, as explained more completely below, by receiving color facsimile machine 150 to generate a color image in which the colors of the original image are accurately reproduced at receiving color facsimile machine 150.

Device dependent color performance characterization information is generally contained in a device profile, which is unique to each device. It is not critical to this invention how the device profile is generated. Different manufacturers may generate and use different device profiles. For example, profiles that follow the ICC format are discussed in ICC Profile Format, version 3.01, May 8, 1995, International Color Consortium, which is incorporated by reference in its entirety. This invention is not limited to devices that operate only in accordance with the ICC format, which is discussed here for illustration only.

Sending color facsimile machine's transmit/receive engine 110 processes the scanned color image in accordance with the JPEG standard, e.g., encodes the image, and constructs a bit stream that includes sending device dependent color performance characterization information 13 OS and the encoded data representing the color image. The bit stream is sent over a communication channel 101 to receiving color facsimile machine 150. The present invention supports devices that process and transmit color images in under a variety of image formats. Typically, the processing of color image data and the format for transmitting the processed color image data is defined by a standard. For example, a standard could specify that raw color image data is transmitted in a particular format. In this case, the processing of the color image data is effectively an unitary operation.

The standard used to define the processing and transmission of color image data is not essential to this invention. The important aspect is that the both sending color facsimile machine 100 and receiving color facsimile machine 150 utilize the same standard so that color image data can be accurately identified and color corrected according to the principles of this invention. Thus, in general, raw color image data is processed, as defined by a standard, or perhaps, a de facto standard, and the processed color image data is transmitted by the sending device along with sending device dependent color performance characterization information 130S. In the embodiment described above, each pixel in the raw color image data was processed and transmitted. When the receiving color facsimile machine's transmit/receive engine 160 receives the bit stream, engine 160 parses the bit stream and stores (1) encoded data 123E that represents the color image and (2) sending device dependent color performance characterization information 130SR in memory 170. In Figure 1, information 13 OS and 130SR are the same information. The different reference numerals are used to indicate that the information is initially in memory 120, and is not in memory 170 until after the bit stream is received from machine 100, parsed, and the data stored by engine 160. Also stored in memory 170 is receiving device dependent color performance characterization information 18 OR. As explained more completely below, receiving device dependent color performance characterization information 180R includes a transformation that converts color image data in the intermediate color space to color image data in the native color space of machine 150; a native color space non- linear transformation that corrects for nonlinear color effects of machine 150; and, optionally, a native color space gamut range. Receiving device dependent color performance characterization information 180R is also known.

Also, while in Figure 1 only one set of device- dependent color performance characterization information is shown in the memory of the device, according to the principles of this invention, each device that has both send and receive capabilities includes both sending device dependent color performance characterization information and receiving device dependent color performance characterization information stored in a memory. Transmit/receive engine 160 decodes encoded data 123E representing the color image to obtain pixel data 123D for the color image. Pixel data 123D is in the native color space of machine 100. In this embodiment, receiving color facsimile machine 150 processes pixel data 123D using receiving device dependent color performance characterization information 18OR stored in memory 170 in combination with sending device dependent color performance characterization information 130SR. The processing corrects the color image data for non-linear effects of both machine 100 and machine 150. Thus, unlike prior art attempts at color compensation that used a linear scaling, this invention corrects for non-linear effects and so obtains an improved color reproduction.

In this embodiment, each pixel in the native color space of machine 100 is (1) corrected for non-linear effects and (2) transformed to a pixel in an intermediate color space, both using information in sending device dependent color performance characterization information 130SR. Next, the pixels in the intermediate color space are (3) transformed to pixels in the color space of receiving device 150 and (4) corrected to take into account the nonlinearities in the color space of receiving facsimile machine 150, both using receiving device dependent color performance characterization information 180R. While these operations are discussed separately, in practice the operations are performed in a single combined operation. This process provides an accurate reproduction of the original color image. Further, this process is suitable for use between any two devices capable of transmitting color image data, because all the information needed by the receiving device is included in the bit stream sent by the sending device. Thus, the advantages of this invention are not limited to color facsimile transmission. The color correction process and data structures that include linear and non-linear color correction data can be used in any application that transmits color image data from a first device to a second device where the first device is unaware of the color performance characteristics of the second device. When the first device is unaware of the color performance characteristics of the second device, and conversely, the devices are considered independent. Of course, after the bit stream of this invention is transmitted to the second device, the second device has all the necessary color performance characteristics of the first device, but prior to the transmission the devices are independent.

Figure 2 is a block diagram of a bit stream 200 that is generated by sending color facsimile machine 100 which, in accordance with the principles of this invention, includes sending device dependent color performance characterization information 130S. In this embodiment, a single image is transmitted, and so bit stream 200 is constructed using the definitions for marker and data segments from the JPEG standard.

Typically, transmit receive engine 110 constructs bit stream 200 or portions of bit stream 200 in a buffer memory 140, and subsequently, the information is passed from buffer memory 140 to communication channel 101 as bit stream 200. Consequently, Figure 2 also represents a structure stored in buffer memory 140.

Bit stream 200 for the color image starts with a start of image marker 201 that identifies the appended data structure as a color image. Following start of image marker 201 are tables and other marker segments 202. Tables and other marker segments 202 may include, for example, any combination of a Huffman code table specification, a quantization table, a restart definition, a comment, or application data.

Following tables and other marker segments 202 in the data structure in bit stream 200 is a sending device dependent color performance characterization marker segment 203. As explained more completely below, sending device dependent color performance characterization marker segment 203 contains al]. the information necessary for the receiving device to create a reproduction of the original color image that has true colors. Specifically, marker segment 203 includes sending device dependent color performance characterization information 13OS that includes both linear and non-linear color characterizations of sending device 100.

Following sending device dependent color performance characterization marker segment 203 is a frame marker 204. Frame marker 204 identifies the subsequent appended data as the start of a frame. Specifically, frame marker 204 includes a start of frame marker, a frame length, sample precision, number of lines in the color image, a number of samples per line, a number of color components, color component identifiers, sampling factors, and quantization table selectors.

Following frame marker 204 is a scan one data segment 205. Scan one data segment 205 includes a scan marker and entropy encoded data for the first scan of the color image. The structure and information in scan one data segment 205 is defined by the JPEG standard and so is well-known to those of skill in the art.

Similarly, in bit stream 200 following scan one data segment 205 is a define number of line marker segment 206, followed by one or more additional scan data segments 207, and an end of image marker 208. The structure, content, and organization of these segments are also defined by the JPEG standard and so are well- known to those of skill in the art.

The information included in sending device- dependent color performance characterization marker segment 203, as indicated above, is dependent upon the particular sending device and may differ from information used in other devices supported by this invention. Therefore, the following description is illustrative only of the principles of the invention, and is not intended to limit the invention to the specific embodiment described.

In one embodiment , sending device dependent color performance characterization information 13 OS, that is included in transmitted bit stream 200, includes a sending device linear color space transformation SM_LINEAR and sending device non-linear transformation SM_NONLINEAR, both for sending device 100. In another embodiment, sending device- dependent color performance characterization information 13OS also includes a gamut range for each color component of sending device 100. The gamut range for each color component of sending device 100 is not essential to this invention, and is thus optional. Sending device non-linear transformation SM_NONLINEAR includes non-linear descriptors that remove the non-linear characteristics from each color component of each pixel in the raw scanned color image data. Sending device non-linear transformation descriptors SM_NONLINEAR functions similar to a quantization table.

As is known to those of skill in the art, sending device non-linear transformation SM_NONLINEAR can be either a one-dimensional table or a three-dimensional table. Sending device non-linear transformation SM_NONLINEAR is stored at location 124 in memory 120. As described more completely below, when sending device non- linear transformation SM_NONLINEAR is used to process the raw color image data, the non-linear effects for each color component of a pixel are removed from the raw color image data to obtain linear color image data in the native color space for facsimile machine 100.

Sending device linear color space transformation data SM_LINEAR is stored at location 121 in memory 120. Sending device linear color space transformation data SM_LINEAR is used to convert linear color image data in the native color space of facsimile machine 100 to linear color image data an intermediate standard color space .

In one embodiment, the intermediate color space is the CIE color space defined by color components X, Y and Z. Thus, sending device linear color space transformation SM_LINEAR is a transformation from the native color space of sending device 100 to the X, Y, and Z components of the CIE color space, i.e., a transformation from signals of a first type to signals of a second type.

For example, if native color space of sending device 100 is the NTSC red-green-blue (RGB) color space (named for the "National Television Systems Committee, " which sets color television standards used in the United States and Japan) as discussed in The Reproduction of Colour in Photography, Printing & Television, by R.W.G. Hunt (Fountain Press England 1987), at page 103, linear color components R_o, G_o, and B_o in the RGB (native) color space are mapped to linear color components X, Y, and Z in the CIE (intermediate) color space using the following sending device linear color space transformation SM_LINEAR : R o

--SM LINEAR B 'o

G Ό where

2.3646 -0.5151 0.0052

SM LINEAR -0.8965 1.4264 -0.0144 -0.4681 0.0887 1.0092

Of course, the particular coefficients used in sending device linear color space transformation SM_LINEAR vary depending on the individual scanning device used to capture the image. Therefore, the coefficients in transformation SM_LINEAR are illustrative only and are not intended to limit the invention to the specific transformation given. Thus, in one embodiment of this invention, sending device linear color space transformation SM_LINEAR and sending device non-linear transformation SM_NONLINEAR are transmitted in bit stream 200 by sending color facsimile machine 100. Specifically, bit stream 200A (Figure 3) includes a first sending device dependent color performance characterization marker segment 203A that includes sending device non-linear transformation SM_NONLINEAR, and a second sending device dependent color performance characterization marker segment 203B that includes sending device linear color space transformation SM_LINEAR.

In this embodiment, sending device-dependent color performance characterization marker segment 203A starts with a color correction marker 213A, e.g., a first field marker, that identifies the segment 203A as containing a sending device non- linear transformation SM_NONLINEAR. The particular number used to designate the marker used in this invention is not essential . The important aspect is that unique markers are defined so that the data stream can be unambiguously parsed by any receiving device. Marker 213A is followed in bit stream 200A by a length of data field 301. In this embodiment, field 301 is one byte in length. Sending device nonlinear transformation SM_NONLINEAR is appended in bit stream 200A after length of data field 301. In this embodiment, each coefficient in sending device nonlinear transformation SM_NONLINEAR is one byte, and so field 301 is set to nine to indicate that nine bytes follow in data field 214A.

Similarly, sending device dependent color performance characterization marker segment 203B starts with a color correction marker 213B, e.g, a second field marker, that identifies the segment 203B as containing sending device linear color space transformation SM_LINEAR. Marker 213B is followed in bit stream 200A by a length of data field 302. In this embodiment, field 302 is also one byte in length. Sending device linear color space transformation SM_LINEAR is appended in bit stream 200A after length of data field 302. The particular arrangement of transformations SM__LINEAR and SM_NONLINEAR in bit stream 200A is not essential to this invention. The only requirement is that a particular arrangement must be defined so that each machine can parse and properly interpret the information in bit stream 200A.

Bit stream 200A also includes an optional gamut range marker segment 310 that conforms to the T-30 E Recommendation, and has a gamut marker 311, with a code value xFFEl, followed by a two-byte length field 312 that in turn is followed by gamut range marker segment data 310A. In this embodiment, gamut range marker segment data 310A includes a FAX identifier field 313 and gamut range data 314 for each color component.

FAX identifier field 313 contains a six-byte value that signifies that data stream 200A contains optional gamut range data 314. Gamut range data 314 contains three pairs of two-byte signed integers. Each pair contains a span Q and an offset P of the zero point for one color component . The remainder of the data in bit stream 200A is the same as that described above for bit stream 200.

In this embodiment, as indicated above, bit stream 200 is constructed in a buffer memory 140 and transmitted over a communication channel 101 by sending color facsimile machine's transmit/receive engine 110. Communication channel 101 could be a switched telephone network, a satellite link, the Internet, or any other communication path over which facsimile transmissions are carried.

Thus, in this embodiment, a scan engine of sending facsimile machine 100 scans a color image and converts the color image to pixels in the native color space of machine 100 in scan operation 401 (Figure 4) . The scanned color image, or at least a portion of the scanned color image, is shown in this embodiment as stored image 123 in memory 120. The pixels for the scanned color image are processed in process raw data operation 402. While in this embodiment, data compression is used in process raw data operation 402, data compression is not essential to the features of this invention. The important aspect is that the color image data is processed for transmission according to the particular standard of interest. The color correction features of this invention are independent of any particular standard for color image representation. Thus, the invention could also be used for color images in, for example, a TIFF format.

Specifically, in operation 402 according to the JPEG standard, each pixel block is operated on by a discrete cosine transform process to generate a two- dimensional array of frequency coefficients. The two- dimensional array of frequency coefficients are quantized. The quantized array of frequency coefficients are stored in a one-dimensional array using the zig-zag sequence. The resulting one- dimensional array of quantized frequency coefficients is Huffman coded to generate encoded data for the pixel block.

In construct and transmit bit stream operation 403, transmit and receive engine 110 generates and transmits bit stream 200A. As explained above, bit stream 200A includes sending device dependent color performance characterization information 130S including sending device linear color space transformation SM_LINEAR and sending device nonlinear transformation SM_NONLINEAR, both for sending device 100, as well as the encoded data representing the digital color image .

The particular process used to construct and transmit the bit stream is not a critical aspect of this invention. The important aspect is that the bit stream includes sending device dependent color performance characterization information 130S.

When the receiving color facsimile machine's transmit and receive engine 160 receives bit stream 200A from buffer 190 in receive and store operation 410, engine 160 parses bit stream 200A, as defined by the JPEG standard, and stores compressed color image 123E, sending device linear color space transformation SM_LINEAR 121R, and sending device nonlinear transformation SM_NONLINEAR 124R into memory 170 in receive and store operation 410. Thus, receiving facsimile machine 150 has sending device dependent color performance characterization information 130SR stored in memory 170.

In reconstruct raw data operation 411, receiving color facsimile machine's transmit and receive engine 110 decodes, i.e., decompresses, encoded color coefficients 123E and stores the reconstructed color image 123D in memory 170. The pixels in reconstructed color image 123D are specified in the color components of the native color space of sending device 100. After reconstruct raw data operation 411 is complete, engine 160 transfers processing to non-linear and linear color correction operation 412. In operation 412, the color components for each pixel in reconstructed color image 123D are transformed to color components for a pixel in the native color space of device 150 using a single operation. The operation corrects for non-linear color effects in both sending machine 100, and receiving machine 150.

In one embodiment of this invention, the single operation in operation 412 is equivalent to first correcting the color components for a pixel in reconstructed color image 123D for nonlinearities of sending device 100 using sending device non-linear transformation SM_NONLINEAR 124R. This generates linear color components for each pixel of reconstructed color image 123.

The linear color component for each pixel are transformed from the native linear color space of sending machine 100 to an intermediate color space using sending device linear color space transformation SM_LINEAR 121R. This generates linear color components for each pixel of color image 123 in the intermediate color space, e.g, X, Y, and Z components in the CIE color space.

The linear color components in the intermediate color space are transformed to the native color space of receiving machine 150 using receiving device linear color space transformation RM_LINEAR, that is also stored in memory 170 at memory location 180R. This generates linear color components for each pixel of color image 123 in the native color space of receiving machine 150.

Finally, the linear color components in the native color space of receiving machine 150 are corrected for the non-linear color effects of receiving machine 150 using receiving device nonlinear transformation RM_NONLINEAR stored in memory 170 at memory location 180R. The specific transformation, that is stored in memory 170 at memory location 180R in receiving machine 150, depends upon the apparatus used to display the color image. For example, one set of transformations would be used for a color printer and another set of transformations would be used for a color monitor.

The combination of the four operations into a single operation means that each pixel is processed once rather than the four times described above. This reduces the data handling operations required in the color correction process.

This embodiment of the invention uses the CIE color space as the intermediate color space because, as discussed below, one transformation between the CIE color space and one other color space is easier to store than multiple arrays that can transform one color space to a number of other color spaces. Also, the CIE color space contains a range of colors that exceed the spectrum of colors that the human eye can detect; this range thus includes colors used in a wide variety of applications, such as photographic film and RGB monitors . Because of this wide range of reproducible colors, the CIE color space is independent of any device on which an image is printed. The CIE color space is thus considered an "absolute color space, " and has accordingly been adopted as an international standardized color space.

As discussed above, receiving device 150 needs only color components for each pixel, sending and receiving device linear color space transformations SM__LINEAR and RM_LINEAR, respectively, and sending and receiving device nonlinear transformations SM_NONLINEAR and RM_NONLINEAR, respectively, to transform reconstructed image 123D to an image in the native color space of receiving device 150. Nonlinear and linear color correction operation 412 performs the composite transformation using the four transformations for each pixel to generate an image in the native color space of receiving device 150. As each pixel is transformed, in this embodiment, the new pixel components are stored back in decoded image 123D. Upon completion of nonlinear and linear color correction process 412, processing optionally transfers to scale gamut process 413.

Upon completion of nonlinear and linear color correction operation 412, the image in the native color space of the receiving device has been corrected for the color space and non-linear characteristics of the sending device. Therefore, the resulting image could simply be printed. However, this invention, optionally provides gamut scaling such as that in the Annex E in Recommendation T-30 (T-30 Annex E) . Thus in Figure 4, scale gamut operation 413 is included with a dash box to show that the operation is optional. The operation implemented in scale gamut operation 413 is equivalent to that described in T-30 Annex E and so is known to those of skill in the art. Thus, the color image is printed or displayed in print or display image operation 414, either after non-linear and linear color correction operation 414 or after scale gamut operation 413. In a second embodiment of this invention, carried out in accordance with the T-30 Annex E standard, color image data is sent in accordance with the T-30 Annex E standard and with the novel color correction of this invention. As shown in Figure 5, in the T-30 Annex E standard, color image data is processed at sending color facsimile 100 before being transmitted to receiving color facsimile 150. In Figure 5, color image 123 is first scanned in scan operation 501. In both embodiments of this invention, the initial raw color image data is described as scanned data. However, this is illustrative only and is not intended to limit the invention to scanned data. The principles of this invention are applicable to any digital color image data that is transmitted from a first device over a communication channel to an independent second device.

Upon completion of operation 501, the initial color corrections operations are performed in sending non-linear and linear color transformation operation 502. Sending device linear color space transformation SM_LINEAR and sending device non-linear transformation SM_NONLINEAR are used in a manner similar to that described above. Specifically, transformation SM_NONLINEAR is used to remove nonlinear color effects from the raw scanned data, and transformation SM_LINEAR is used to transform the resulting linear color image data into an intermediate color space which is the CIE color space. Typically, both of these transformations are applied in a single operation to produce color components for each pixel in the CIE color space.

In process color image data operation 503, the color image data in the CIE color space is encoded according to the JPEG standard, as described above. The encoded CIE color image data is formed into a bit stream according to the JPEG standard described above, and transmitted over communication channel 101 in construct and transmit bit stream operation 504. In this embodiment, the data structure in the bit steam, as shown in Figure 3 , does not include transformations SM_NONLINEAR and SM_LINEAR, because these transformations have been used to operate on the color image data in sending facsimile machine 100, as just described.

At receiving color facsimile machine 150, the compressed color image in the CIE color space is received and stored in receive and store operation 510, and decoded in reconstruct color image data operation 511. The operations in operation 511 are similar to those described above for process 411, and that description is incorporated herein by reference. In receiving non-linear and linear color correction operation 512, the decoded color image, in the CIE color space, is translated into a color image in the color space of receiving color facsimile 150 using receiving device linear color space transformation RM_LINEAR. The resulting data is corrected for nonlinearities in the color space of receiving facsimile machine 150 using receiving device nonlinear transformation RM_NONLINEAR. Optionally, the gamut of the decoded color image is scaled in scale gamut operation 513, and the resulting color image is printed or displayed in print or display operation 514.

As used herein a color facsimile machine is a generic device that is capable of only sending, only receiving, or both sending and receiving color facsimile images. The particular modification to the above disclosure for a send only, or a receive only device will be apparent to those of skill in the art in view of this disclosure. Similarly, the sending and receiving device can be a stand-alone color facsimile machine, or perhaps a computer system with color facsimile capability connected to a color scanner. Further, the advantages of this invention are not limited to color facsimile transmission. The color correction process and data structures that include linear and non-linear color correction data can be used in any application that transmits color image data from a first device to a second device where the first device is unaware of the color performance characteristics of the second device.

Although the invention has been described in connection with several embodiments, it is understood that this invention is not limited to the embodiments disclosed, but is capable of various modifications which would be apparent to one of ordinary skill in the art. Thus, the invention is limited only by the following claims.

Claims

CLAIMS We claim:
1. A method for improving remote device color reproduction of a color image generated by a sending device, said method comprising: processing color image data in said sending device to generate processed color image data; transmitting a bit stream including sending device-dependent color performance characterization information and said processed color image data from said sending device receiving said bit stream at a remote receiving device; and generating a reproduction of said color image data using said sending device-dependent color performance characterization information and said processed color image data at said remote receiving device wherein said sending device and said remote device are independent unrelated devices.
2. The method of Claim 1 wherein said transmitting said bit stream including sending device- dependent color performance characterization information further comprises: transmitting sending device non-linear color correction data in said bit stream wherein said sending device non-linear color correction data removes non-linear effects from said color image data to produce linearized color image data.
3. The method of Claims 1 or 2 wherein said transmitting said bit stream including sending device- dependent color performance characterization information further comprises: transmitting a sending device linear color space transformation in said bit stream; wherein said sending device non-linear color correction data and said sending device linear color space transformation are used by said remote receiving device in generating said reproduction of said color image data.
4. The method of Claim 2 wherein said transmitting said bit stream including sending device- dependent color performance characterization information further comprises: transmitting a first field marker in said bit stream immediately proceeding said non-linear color correction data.
5. The method of Claim 3 wherein said transmitting said bit stream including sending device- dependent color performance characterization information further comprises: transmitting a first field marker in said bit stream immediately proceeding said non-linear color correction data.
6. The method of Claim 5 wherein said transmitting said bit stream including sending device- dependent color performance characterization information further comprises: transmitting a second field marker in said bit stream immediately proceeding said sending device linear color space transformation.
7. The method of Claim 1 wherein said sending device is a color facsimile machine.
8. The method of Claim 1 wherein said remote receiving device is a color facsimile machine.
9. The method of Claim 1 wherein processed color image data in said bit stream has a structure defined by a JPEG file interchange format .
10. The method of Claim 3 wherein said transmitting a sending device linear color space transformation further comprises transmitting a linear color space transformation from a native color space of said sending device to an intermediate color space.
11. The method of Claim 10 wherein said intermediate color space is a CIE color space.
12. The method of Claim 10 wherein said generating said reproduction of said color image data further comprises using (a) a linear color space transformation from said intermediate color space to a native color space of said remote receiving device, (b) said sending device non-linear color correction data, and (c) said linear color space transformation from a native color space of said sending device to an intermediate color space to obtain linear color data in said native color space of said remote receiving device.
13. The method of Claim 12 wherein said generating said reproduction of said color image data further comprises using a non-linear color data transformation of said remote receiving device to correct said linear color data in said native color space of said remote receiving device.
14. The method of Claim 1 wherein said processing color image data comprises encoding said color image data in accordance with a JPEG standard.
15. The method of Claim 1 wherein said processing color image data comprises encoding said color image data to generate encoded color image data.
16. The method of Claim 15 wherein said generating a reproduction of said color image data further comprises decoding said encoded color image data to obtain said color image data.
17. The method of Claim 1 wherein said transmitting said bit stream including sending device- dependent color performance characterization information further comprises: including a color correction field marker in said bit stream; and including a color correction field segment data field in said bit stream following said color correction field marker.
18. The method of Claim 17 wherein said color correction field marker comprises a sending device non- linear transformation field marker and said color correction field segment data field includes a sending device non-linear transformation immediately following said sending device non-linear transformation field marker .
19. The method of Claim 18 wherein said color correction field marker further comprises a sending device linear transformation field marker and said color correction field segment data field further includes a sending device linear transformation immediately following said sending device linear transformation field marker.
20. The method of Claim 17 wherein said color correction field marker comprises a sending device linear transformation field marker and said color correction field segment data field includes a sending device linear transformation immediately following said sending device linear transformation field marker.
21. The method of Claim 17 wherein said color correction field segment data field includes a linear color space transformation from a native color space of said sending device to an intermediate color space.
22. The method of Claim 21 wherein said color correction field segment data field includes a non- linear color space transformation from raw pixel data in a native color space of said sending device to linear pixel data in said native color space.
23. A method for improving remote receiving device color reproduction of a color image generated by a sending device, said method comprising: processing color image data in said sending device to compensate for non-linear effects and produce linearized color data in a native color space of said sending device; transforming said linearized color data in said native color space to data in an intermediate color space; transmitting said data in said intermediate color space; receiving said data in said intermediate color space at said remote receiving device; transforming said data in said intermediate color space to a native color space of said remote receiving device to produce linearized color data in said native color space of said remote receiving device; and correcting said linearized color data in said native color space for non-linear effects to generate said improved remote receiving device color reproduction of said color image.
PCT/US1997/017306 1996-10-03 1997-10-01 A method for color correction in color facsimiles WO1998015114A1 (en)

Priority Applications (4)

Application Number Priority Date Filing Date Title
US72601996 true 1996-10-03 1996-10-03
US72243096 true 1996-10-03 1996-10-03
US08/722,430 1996-10-03
US08/726,019 1996-10-03

Publications (1)

Publication Number Publication Date
WO1998015114A1 true true WO1998015114A1 (en) 1998-04-09

Family

ID=27110595

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US1997/017306 WO1998015114A1 (en) 1996-10-03 1997-10-01 A method for color correction in color facsimiles

Country Status (1)

Country Link
WO (1) WO1998015114A1 (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
GB2326553A (en) * 1997-06-07 1998-12-23 Roland Man Druckmasch Correction of geometric faults in the transfer of information to a printing material

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5067027A (en) * 1988-08-18 1991-11-19 Ricoh Company, Ltd. Image data transmission method and apparatus
EP0487329A2 (en) * 1990-11-21 1992-05-27 Canon Kabushiki Kaisha Color image communication apparatus
JPH066563A (en) * 1992-06-18 1994-01-14 Canon Inc Picture transmitter
EP0581590A2 (en) * 1992-07-31 1994-02-02 Canon Kabushiki Kaisha Color processing method
EP0680202A2 (en) * 1994-04-28 1995-11-02 Canon Kabushiki Kaisha Image processing apparatus and method
JPH07303190A (en) * 1994-05-06 1995-11-14 Ricoh Co Ltd Image processor

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5067027A (en) * 1988-08-18 1991-11-19 Ricoh Company, Ltd. Image data transmission method and apparatus
EP0487329A2 (en) * 1990-11-21 1992-05-27 Canon Kabushiki Kaisha Color image communication apparatus
JPH066563A (en) * 1992-06-18 1994-01-14 Canon Inc Picture transmitter
EP0581590A2 (en) * 1992-07-31 1994-02-02 Canon Kabushiki Kaisha Color processing method
EP0680202A2 (en) * 1994-04-28 1995-11-02 Canon Kabushiki Kaisha Image processing apparatus and method
JPH07303190A (en) * 1994-05-06 1995-11-14 Ricoh Co Ltd Image processor

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
PATENT ABSTRACTS OF JAPAN vol. 18, no. 205 (E - 1536) 12 April 1994 (1994-04-12) *
PATENT ABSTRACTS OF JAPAN vol. 96, no. 3 29 March 1996 (1996-03-29) *

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
GB2326553A (en) * 1997-06-07 1998-12-23 Roland Man Druckmasch Correction of geometric faults in the transfer of information to a printing material
GB2326553B (en) * 1997-06-07 2002-02-20 Roland Man Druckmasch Method for the correction of geometric faults in the transfer of information to a printing material

Similar Documents

Publication Publication Date Title
US5323249A (en) Method for reproducing color images having one color gamut with a device having a different color gamut
US6268939B1 (en) Method and apparatus for correcting luminance and chrominance data in digital color images
US5719956A (en) Image processing apparatus with separate color conversion for CMY signals and K signal
US5488483A (en) Facsimile with user selective compression methods
Anderson et al. Proposal for a standard default color space for the internet—srgb
US5677736A (en) Data processor
US5402253A (en) Color conversion method and apparatus with a variable gray component replacement ratio
US5565931A (en) Method and apparatus for applying gamma predistortion to a color image signal
US6404511B1 (en) Self-calibration of network copier system
US20040114172A1 (en) Method of and apparatus for processing image data, and computer product
US5481380A (en) Method and apparatus for calibration of color values
EP0482790B1 (en) Method and apparatus for generating representation of an image
US5754746A (en) Multi bit per pixel compression/decompression using parallel encoded streams
US4887151A (en) Encoding apparatus for color image data with block-by-block individual quantizing or individual encoding of luminosity, structure, and color information
US6009192A (en) Color correction of a compressed image
US7289663B2 (en) Producing an extended color gamut luminance-chrominance digital image from a captured image
US5787193A (en) System for converting color image signals from RGB to CMY color spaces through look-up tables
US6719392B2 (en) Optimized color ranges in gamut mapping
US20020029715A1 (en) Color conversion coefficient preparation apparatus, color conversion coefficient preparation method, storage medium, and color conversion system
US6560285B1 (en) Region-based information compaction as for digital images
US6285784B1 (en) Method of applying manipulations to an extended color gamut digital image
US6396422B1 (en) Methods for quantizing and compressing digital image data
US7227666B1 (en) Dynamic selection of rendering intent for color proofing transforms
US5835244A (en) Method and apparatus for the conversion of color values
US6198842B1 (en) Multi-spectral image compression with bounded loss

Legal Events

Date Code Title Description
AK Designated states

Kind code of ref document: A1

Designated state(s): AL AM AT AU AZ BA BB BG BR BY CA CH CN CU CZ DE DK EE ES FI GB GE GH HU IL IS JP KE KG KP KR KZ LC LK LR LS LT LU LV MD MG MK MN MW MX NO NZ PL PT RO RU SD SE SG SI SK SL TJ TM TR TT UA UG UZ VN YU ZW AM AZ BY KG KZ MD RU TJ TM

AL Designated countries for regional patents

Kind code of ref document: A1

Designated state(s): GH KE LS MW SD SZ UG ZW AT BE CH DE DK ES FI FR GB GR IE IT LU MC NL PT

121 Ep: the epo has been informed by wipo that ep was designated in this application
REG Reference to national code

Ref country code: DE

Ref legal event code: 8642

NENP Non-entry into the national phase in:

Ref country code: JP

Ref document number: 1998516682

Format of ref document f/p: F

122 Ep: pct application non-entry in european phase