JP2008048419A - Image correction system and method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a system and method for correction the quality of image data. <P>SOLUTION: This system receives image data, calculates its spectral frequency data, and obtains a cumulative probability distribution. In addition, this system calculates a spatial parameter from the image data and determines whether an image represented by that image data has been back lit. When it has been back lit, a mask for a backlight area is created for hue or brightness correction in the area. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to an image correction system and method, and more particularly to an image correction system and method that includes a backlit main image region.

  Nowadays, electronically encoded images exist everywhere. Currently, electronically encoded images are obtained directly from devices such as digital still cameras or digital video recorders, image scanned on other media such as photographs, or live television broadcasts Or from previously obtained images retrieved from a storage device, such as from a numerically encoded image archive. Many such images are acquired under less than ideal conditions, or images that result in suboptimal results due to variations in lighting or other properties of various aspects of the acquired image Was either recorded by the rendering device. An example is an image taken in a backlight setting. Such a situation can occur when a clear sky, direct sunlight or other relatively strong background illumination source is behind an object of interest, such as a building, person or landscape. The background illumination in such a situation is strong enough to degrade the detail or resolution of the foreground image or object, the backlit image portion, or both. Previous approaches to address such concerns are algorithmically, electrically, or through signal processing, or mechanically through filtration, F-stops, aperture diameters, etc. in acquiring images, It was done. However, previous systems have focused on acquisition or processing of the entire image, so attempts to address concerns for a portion of the image can adversely affect other aspects of the image. There was a point.

  Captured or stored images are typically stored in an encoding format, such as digitally, and the encoding is often associated with component values in the primary color space. Such color components are essentially additive, such as red, green, blue (RGB), or subtractive, such as cyan, magenta, yellow (CMY), etc. The latter is often black Combined with (K), called CMYK or CMY (K). The additive primary color space description generally relates to an image displayed on a device that generates light, such as a monitor or projector. On the other hand, the subtractive three-primary color space description generally relates to an image generated on a medium or device that does not generate light, such as paper printing. In order to move an image from a display to a fixed medium such as paper, it is necessary to convert between the color spaces associated with the electronic encoding of the document.

  The concepts disclosed herein can be better recognized by understanding the numerical models used for image representation and image colorization in image processing or rendering applications. One of the first mathematically defined color spaces is the CIE / XYZ color space, created by CIE in 1931, also known as the CIE 1931 color space. The human eye has receptors for short (S), medium (M), and long (L) wavelengths, also known as blue, green and red receptors. All that is necessary is to generate three parameters that describe the perception of the color. A specific method for associating three numerical values, ie, tristimulus values with each color, is called a color space, and the CIE / XYZ color space is one of many color spaces. The CIE / XYZ color space is directly based on human eye measurements, and many other color spaces are defined from the CIE / XYZ color space and are the common basis.

  In the CIE / XYZ color space, tristimulus values are not S, M, and L stimuli in the human eye, but a set of tristimulus values called X, Y, and Z. These are also generally red, green, and blue, respectively. It is possible to create two light sources consisting of different mixtures of various colors and having the same color (conditional color matching (metamerism)). When two light sources have the same apparent color, they will have the same tristimulus value regardless of the light mixture used to make them.

CIE / L ^* a ^* b ^* (also expressed as CIE / LAB or Lab) is often considered as one of the most complete color models. Traditionally, the CIE / LAB model is used to describe all colors visible to the human eye. The CIE / LAB model was developed by the International Commission on Lighting (Commission Internationale d'Eclairage; often abbreviated as CIE) for the purpose of describing all colors visible to the human eye. Of the three parameters (L, a, b) of this model, L represents luminance (brightness) and a, b represents a color region. When the luminance L is 0, it is black, and when it is 100, it is white. a indicates a position between red and green. If the value is positive, it represents red, and if the value is negative, it represents the complementary green of red. b indicates a position between yellow and blue. If the value is positive, it represents yellow, and if the value is negative, it represents the complementary blue of yellow.

  The Lab color model was created to serve as a device independent reference model. Therefore, the full gamut of colors in this model, i.e. the visual representation of the full range available, is not completely accurate, but rather is used to conceptualize the color space. Since the Lab model is three-dimensional, it is correctly represented in three-dimensional space. A useful feature of the Lab model is that the first parameter, L, is very intuitive. Changing the value L is similar to changing the brightness setting in a TV receiver. In the Lab model, luminance is represented on the vertical axis, and only a few representations of horizontal “slices” in the model are sufficient for conceptual visualization of the entire color reproduction region.

The Lab model is essentially parameterized correctly. Therefore, a specific color space based on this model is not necessary. The CIE 1976 L ^* a ^* b ^* mode, or Lab mode, is intended to define the perceptibility of color differences. The circle representation in the Lab space corresponds to an ellipse in the XYZ space. The non-linear relationship in L ^* , a ^* and b ^* is related to the cubic root and is intended to reproduce the logarithmic response of vision. Coloring information is referenced to the color of the system white point.

  Electronic documents, such as documents that represent color images, are typically encoded in one or more standard formats. There are many such formats, but recent typical description formats include Microsoft Word® file (* .doc), tagged information file format (TIFF), graphic image format (GIF), Adobe Portable Document Format (PDF), Adobe Systems PostScript (registered trademark), Hyper Text Markup Language (HTML), Extensible Markup Language (XML), Drawing Exchange Files (* .dxf), drawing files (* .dwg), paintbrush files (* .pcx), joint photography expert groups (JPEG), and many other bitmapped or Encoded or compressed format or vector It is included to take file format.

  Accordingly, there is a system and method for easily correcting the image quality of an image that has been arbitrarily encoded as described above, in order to cope with image quality degradation related to a region portion of the image affected by different illumination or lighting characteristics. desired.

  The present invention has been made in view of the above-described conventional problems, and an object thereof is to provide an image correction system and method. It is another object of the present invention to provide a system and method for modifying the image quality of an image in which two or more main image regions of the captured image have distinct brightness, brightness or contrast characteristics. Furthermore, the present invention provides a system and method that allows for easy image quality correction of arbitrarily encoded images to correct the image quality for the main image areas of the image affected by different lighting or lighting characteristics. The purpose is to provide.

  An image correction system according to the present invention includes means for receiving image data, and transition detection for identifying a transition between at least one main image area and at least one background area from the received image data. And adjusting means for adjusting the parameter of the image data relating to at least one of the main image area and the background area in accordance with the transition part specified by the transition part detection means.

  In one embodiment, the adjusting means includes means for adjusting the brightness of at least one of the image data of the main image area and the image data of the background area. In another embodiment, the adjusting means includes means for increasing the brightness of the image data of the main image area and decreasing the brightness of the image data of the background area.

  In a further embodiment, the image correction system according to the present invention further comprises calculation means for calculating spectral frequency data representing a spectral frequency distribution of color data included in the image data, wherein the adjustment means Means for adjusting the brightness of at least one of the image data of the main image region and the image data of the background region in accordance with spectral frequency data; Preferably, the spectrum frequency data includes distribution data representing a cumulative probability distribution of encoded values in the image data.

  In another embodiment, the apparatus further comprises mask data generating means for generating mask data corresponding to the specified transition part, and the adjusting means is provided in the transition part specified according to the mask data. Therefore, it includes means for selectively adjusting a parameter of image data related to at least one of the main image region and the background region. Preferably, the mask data corresponds to at least one portion of the image represented by the image data, and the at least one portion defines a shape having no discontinuities.

  An image correction method according to the present invention includes a step of receiving image data, a step of specifying a transition portion between at least one main image region and at least one background region from the received image data, and specifying in this step Adjusting a parameter of image data related to at least one of the main image area and the background area in accordance with the transition section.

  In accordance with the present invention, an image correction system and method are provided. The present invention also provides a system and method for modifying the image quality of an image in which two or more main image regions of the captured image have distinct brightness, brightness or contrast characteristics. Further, according to the present invention, a system that allows easy image quality correction of an arbitrarily encoded image to correct the image quality for the main image area of the image affected by different illumination or lighting characteristics, and A method is provided.

  The image modification system according to the present invention operates effectively by the analysis and manipulation of numerically encoded image data, such as digitally encoded image data associated with many of the aforementioned formats. In the following description, a digital image encoded in a commonly used RGB color space will be described for purposes of illustration because it is commonly found in image capture devices or digital image processing devices. However, as will be apparent from the following description, it should be appreciated that the present invention applies to any coded image in any color system or gray scale. Furthermore, the present invention can be embodied in any suitable computer platform and will be described in the context of a general purpose digital computing device such as a workstation. However, as described in more detail below, the image modification system according to the present invention can be implemented in a digital image processing device, a controller of a document processing device, or directly embodied in an image capture device such as a digital camera, These devices can incorporate the ability to perform the analysis and calculations described below.

  Hereinafter, embodiments of the present invention will be described as appropriate with reference to the drawings. FIG. 1 is a block diagram showing a hardware configuration example of a computer or workstation in which an embodiment according to the present invention is implemented. The workstation 100 performs data communication with a read-only memory (ROM) 104, a RAM 106, a display interface 108, a storage interface 110, and a network interface 112, which are a nonvolatile read-only memory, a volatile read-only memory, or a combination thereof. A processor 102 is provided. In one embodiment, the interface to the module is through the bus 114. As can be seen from the following description, the functionality of the workstation is typically embodied appropriately by instructions that are read from the storage device and deployed on the RAM 106.

  Read-only memory (ROM) 104 stores BIOS, system functions, configuration data, and firmware such as static data or fixed instructions such as other routines used by processor 102 for operation of workstation 100. .

  The RAM 106 provides a storage area for data and instructions related to applications and data processing executed by the processor 102.

  The display interface 108 receives data or instructions from other modules connected to the bus 114. Those data or instructions are specific to the generation of the display and make the user interface valid. The display interface 108 provides output to a display monitor 128, such as a video display device such as a CRT, LCD, plasma display or other suitable visual output device.

  Storage interface 110 provides a mechanism for non-volatile mass storage or long-term storage of data or instructions within workstation 100. The storage interface 110 is an optical drive such as a disk drive, tape, CD-ROM, DVD-ROM or other relatively large capacity addressable or storage such as a storage device 118 comprised of a serial storage medium. Use the mechanism.

  The network interface 112 communicates with at least one communication unit that interfaces with a network, shown as a network interface card (NIC) 120, for example. The network interface consists of both a physical layer and a protocol layer, such as Ethernet®, token ring or other wide area or local area network communication system via the network interface card 120. It is possible to communicate with other devices via any wired network system 132 or via wireless interface 130, such as WiFi (Wireless Fidelity), WiMax or any other suitable wireless network system.

  An input device 122 such as a keyboard is connected to the input / output interface 116 that performs data communication via the bus 114. The input / output interface 116 also provides data output to the peripheral interface 124, which may be USB, SCSI, IEEE1394 or any other suitable interface. Further, the input / output interface 116 performs data communication with a pointing device interface 126 for connection with a mouse, a light pen, a touch screen, or the like.

  In the example shown in FIG. 1, a network interface such as network interface card 120 enables network interface 112 to be in data communication with wired network system 132. In the figure, a digital image processing device 134 and a document output device 136 including a controller 138 are also in data communication with the wired network system 132. Intelligent output devices such as printers, copiers, facsimile devices, scanner devices or devices that combine these functions as well as the digital image processing device 134 often use intelligent controllers as illustrated in the figures. Such an apparatus can include sufficient capabilities to perform the functions described below, which improve image quality. Alternatively, the image quality enhancement function can be appropriately distributed among a plurality of intelligent devices capable of data communication with each other.

  Next, operations in the embodiment according to the present invention will be described with reference to FIG. FIG. 2 is a flowchart for explaining the image correction operation in the embodiment according to the present invention. Initially, at S202, image data is received through any suitable means known in the art. As described above, the received image data is, for example, an electronic document such as a digitally encoded image from any of the plurality of image data sources described above. In step S204, the input image data is analyzed for frequency information regarding the encoded data, and spectrum frequency data is calculated. In one embodiment, a histogram is generated from an analysis of frequency information about the encoded data, details of which are described below.

  Next, in S206, the cumulative probability distribution function is calculated to form a histogram for the spectral analysis performed in S204, that is, the image content analysis. Next, in S208, a spatial parameter, that is, a characteristic relating to the characteristic region in the image represented by the image data is calculated. By this calculation, regions that are included in the image and can be distinguished are identified, and transition portions between regions having different characteristics are also identified. Thereafter, in S210, a determination based on statistics is performed on the received image data, and it is determined whether the received image data is backlit, that is, backlit. If it is determined that the image is not backlit, the process ends. If it is determined that the image is backlit, in S212, a mask area is generated or specified for one or more backlight image portions, that is, the backlight image portion. In the backlight image of one embodiment, the mask is contiguous and blurred. In this embodiment, a backlight area mask is used, but the generated mask is either a backlight area mask or a frontal image area mask. Appropriate algorithmic adjustments are used that are made according to which mask is selected. Next, in S214, in this embodiment, tone correction is applied to the backlight area to obtain a corrected image output.

  Here, in the embodiment described above, the color tone is corrected for the backlight image portion which is the main image region, but the correction is not necessarily limited to the color tone, and the parameters of the image data are not limited. Obviously, it is possible to adjust the brightness, for example. Further, in the embodiment described above, the color tone is corrected for the backlight image portion which is the main image region, but the correction target is not limited to the backlight image portion, and the background It can also be performed on the image portion.

  Image correction as described above is often performed on metadata associated with an encoded image. However, such correction can also be performed by calculating directly from the image data. Devices such as digital cameras often include encoded images that include metadata. Images from digital capture devices such as digital cameras are particularly problematic for image acquisition unless a backlight situation is unavoidable or considered by an untrained photographer.

  The system described above performs image modification by calculating parameters related to the image, as well as spatially constrained changes made in tone scale rendering. The actual correction is done in one embodiment using cumulative probability distributions and spatial predictors. In addition, if only a portion of the image has a tone scale problem like a clear sky in a backlit photo, i.e. a photo taken in backlight, to allow a significant improvement in overall image quality, You only need to deal with the problem. Complementary image portions are left unchanged or image correction is performed independently in an appropriate manner for each portion. This is a significant difference from conventional systems that typically apply image correction methods or algorithms throughout the image. Conventional algorithms may process or adjust image portions with acceptable image quality if not processed, resulting in degradation of image portions with acceptable image quality.

  With reference now to FIG. 3, a spectral frequency analysis used in connection with the implementation of an embodiment according to the present invention will be described. The cumulative probability distribution of the color values for the image pixels effectively provides an indication of the corresponding electronic image backlighting, ie the degree of backlighting. Compared to a well illuminated image, the cumulative probability distribution for the backlight image rises sharply initially. Further, in the cumulative probability distribution related to the backlight image, there is often a flat portion in the middle range of the scale. As mentioned above, typical encodings are associated with the red, green, blue or RGB color space, and this encoding is represented in the exemplary graphs shown in FIGS. 3, 4 and 8 described below. It is reflected.

  FIG. 3 shows an example of cumulative probability distribution data for two sample images in which backlighting, that is, backlight illumination exists. The two graphs shown in FIG. 3 are merely representative, but well show the characteristics of the backlight image. In this example, 8 bits are used to encode each of the red, green, and blue RGB encodings, and the red, green, and blue components are represented as a solid line, a dotted line, and a dashed line in the graph. This is reflected in each curve (the same applies to FIGS. 4 and 8). With 8-bit encoding, each of the three additive primary colors can have a level of 256 (0-255). The horizontal axis of the graph represents red (R), green (G), and blue (B) values. The vertical axis represents the cumulative probability distribution related to each RGB value. That is, the value on the vertical axis represents a probability that is a function of a coefficient of variation that is less than the corresponding RGB value shown on the horizontal axis.

  The example shown in FIG. 3A is a cumulative probability distribution related to the Hamburg Cathedral image, which will be described later with reference to FIGS. The graph shown in FIG. 3 (a) exhibits a first sudden rise, a flat portion present in the middle range of the scale, and a subsequent resume rise. This indicates that the corresponding image is a backlit image as described above. The example shown in FIG. 3B is a cumulative probability distribution related to a backlit Buddha image. Also in this graph, the first rapid rise, a flat portion existing in the middle range of the scale can be seen. However, it is noted that in this graph, unlike the graph shown in FIG. 3 (a), there is no second rapidly rising region.

  Next, FIG. 4 shows an example of cumulative probability distribution data for two sample images in which front lighting, that is, front lighting is present. As in FIG. 3, the red, green, and blue components are represented by solid lines and dotted lines in the graph. , And are reflected in each curve as an alternate long and short dash line. It is noted that in the two examples shown in FIG. 4, the rapid rise and subsequent flattening trends described in connection with the graph shown in FIG. 3 are not seen in either example. Thus, the cumulative probability distribution provides a mechanism that can easily detect whether an image is front-lighting or back-lighting.

  Another consideration is the region of interest from which a cumulative probability distribution is obtained, and the relative distribution of code values in the various regions. For example, if it is statistically assumed that most people take a picture with the main subject in the center, the central weighted cumulative probability distribution is of interest. In cases such as a backlight situation, the top of the image generally must have a sign value that is significantly higher than the center or bottom region.

  FIG. 5 shows an example of a representative photograph of the Hamburg Cathedral referred to in FIG. The original image shown is in color, but the color image is shown converted to gray scale. In one embodiment, an operation is performed to identify a relatively dark image portion as a continuous blob. Here, a blob is generally defined as a shape without a discontinuity associated with it at the center of a picture or image frame. As described with reference to FIG. 2, in one embodiment, a mask is suitably generated from the blob and the code value in the image portion inside the mask area is changed. If the blob has discontinuities, a direct operation is used appropriately to fill all discontinuities, reaching a continuous blob region for applying image quality correction.

  FIG. 6 shows an example of an appropriate mask area corresponding to the image shown in FIG. 5 and specified in the embodiment according to the present invention. As described above, the code value outside the specified mask area is also appropriately changed, such as by darkening, to improve the view of the background image portion. FIG. 7 illustrates an image that has been subjected to image correction processing by applying the above-described processing for brightening the image, processing for darkening the background or backlight portion of the image, or a combination of these processing. Although the original image illustrated in FIG. 7 is also in color, the color image is shown converted to gray scale. Algorithms that brighten or darken an image or a portion thereof are well known in the art. It will be apparent that the image shown in FIG. 7 is remarkably improved in detail by applying the processing described above compared to the original image shown in FIG.

  FIG. 8 shows cumulative probability distribution data for the image that has been subjected to the image correction processing illustrated in FIG. Also in FIG. 8, as in FIGS. 3 and 4, each component of red, green, and blue is reflected in each curve as a solid line, a dotted line, and an alternate long and short dash line in the graph. The cumulative probability distribution related to the modified image shown in FIG. 7 is greatly different from the shape of the cumulative probability distribution (FIG. 3A) related to the original image (FIG. 5). That is, in FIG. 8, there is no flat portion existing in the middle range of the first rapid rise and scale, as seen in the cumulative probability distribution curve of FIG. The cumulative probability distribution shown in FIG. 8 relating to the modified image illustrated in FIG. 7 is the cumulative probability distribution of a normal front-lit image as shown in FIGS. 4 (a) and (b). It turns out that it is more similar.

  The invention may be in the form of source code, object code, code intermediate source and object code, such as partially compiled form, or any other form of computer computer suitable for use with embodiments of the invention. Applies to the program. A computer program can be a stand-alone application, a software component, a script, or a plug-in to another application. The computer program for carrying out the present invention transmits a computer program such as a storage medium such as ROM and RAM, an optical recording medium such as a CD-ROM, and a magnetic recording medium such as a floppy (registered trademark) disk. It can be embodied on a carrier that is any entity or device capable of. The carrier is any transmissible carrier such as an electrical or optical signal transmitted by electrical or optical cable, or by radio or other means. The computer program can also be downloaded from the server via the Internet. The function of the computer program can also be incorporated in an integrated circuit. Any and all embodiments that contain code that causes a computer or processor to substantially execute the described principles of the invention are within the scope of the invention.

  The foregoing description of the preferred embodiment of the present invention has been presented for purposes of illustration and description. The description is not exhaustive and is not intended to limit the invention to the form disclosed. Obvious modifications or variations are possible in light of the above disclosure. For example, in this specification, the application of the backlit main image region (specimen) to the captured image has been mainly described. However, the technical idea disclosed herein is applicable to any image modification in which two or more main image regions of a captured image have distinct brightness, brightness or contrast characteristics. The embodiments best illustrate the principles of the invention and its practical applications, so that those skilled in the art can use the invention in various modifications in various embodiments suitable for the particular intended use. Was selected and explained. It will be apparent that all such modifications and variations can be made by those skilled in the art within the principles and scope of the invention as set forth in the appended claims. It is within the scope of the defined invention.

In an embodiment by the present invention, it is a block diagram showing an example of hardware constitutions of a workstation where operation of a system is performed. 5 is a flowchart illustrating an example of an image correction operation in the embodiment according to the present invention. It is an example of the cumulative probability distribution data about the image in which backlighting exists. It is an example of the cumulative probability distribution data about the front-lit image. It is an example of the image in which backlighting exists. It is an example of the mask area | region corresponding to the original image shown in FIG. 5, and identified in embodiment by this invention. FIG. 6 is an example of an image obtained by performing image correction processing according to the embodiment of the present invention on the original image shown in FIG. 5. FIG. 6 is cumulative probability distribution data for an image obtained by performing the image correction processing according to the embodiment of the present invention on the original image shown in FIG.

Explanation of symbols

102 processor 104 read only memory, ROM
106 RAM
108 Display Interface 110 Storage Interface 112 Network Interface 114 Bus 116 Input / Output Interface 118 Storage Device, Disk Drive 120 Network Interface Card 122 Input Device, Keyboard 124 Peripheral Interface 126 Pointing Device Interface 128 Display Monitor 130 Wireless Interface 132 Wired Network System 134 Digital Image Processing Device 136 Document Output Device 138 Controller

Claims (14)

Means for receiving image data;
Transition part detection means for identifying a transition part between at least one main image area and at least one background area from the received image data;
An image correction system comprising: adjusting means for adjusting a parameter of image data relating to at least one of a main image area and a background area in accordance with the transition part specified by the transition part detection means.   2. The image correction system according to claim 1, wherein the adjustment unit includes a unit that adjusts brightness of at least one of the image data of the main image region and the image data of the background region.   2. The image correction system according to claim 1, wherein the adjustment unit includes a unit that increases the brightness of the image data of the main image region and decreases the brightness of the image data of the background region. A calculation means for calculating spectral frequency data representing a spectral frequency distribution of the color data included in the image data;
The said adjustment means includes a means to adjust the brightness about at least one of the image data of the said main image area | region and the image data of the said background area | region according to the said spectrum frequency data. Image correction system.   The image correction system according to claim 4, wherein the spectral frequency data includes distribution data representing a cumulative probability distribution of encoded values in the image data. A mask data generating means for generating mask data corresponding to the identified transition portion;
The adjusting means is a means for selectively adjusting a parameter of image data related to at least one of the main image area and the background area according to a transition part specified according to the mask data. The image correction system according to claim 1, further comprising:   7. The image correction system according to claim 6, wherein the mask data corresponds to at least one portion of an image represented by the image data, and the at least one portion defines a shape having no discontinuity. . Receiving image data;
Identifying a transition between at least one main image area and at least one background area from the received image data;
Adjusting the parameter of the image data related to at least one of the main image area and the background area according to the transition part specified in this step.   9. The image correction method according to claim 8, wherein the adjusting step includes adjusting the brightness of at least one of the image data of the main image area and the image data of the background area.   9. The image correction method according to claim 8, wherein the adjusting includes increasing the brightness of the image data of the main image area and decreasing the brightness of the image data of the background area. Calculating spectral frequency data representing a spectral frequency distribution of color data included in the image data;
9. The image according to claim 8, further comprising a step of adjusting brightness of at least one of the image data of the main image region and the image data of the background region according to the spectral frequency data. How to fix.   The image correction method according to claim 11, wherein the spectrum frequency data includes distribution data representing a cumulative probability distribution of encoded values in the image data. Generating mask data corresponding to the identified transition portion;
Selectively adjusting a parameter of image data related to at least one of the main image area and the background area according to the transition part specified according to the mask data. The image correction method according to claim 8.   14. The image correction method according to claim 13, wherein the mask data corresponds to at least one part of an image represented by the image data, and the at least one part defines a shape having no discontinuity. .