JP5583679B2 - Method and apparatus for processing image data for display by a display device - Google Patents

Description

  The present invention relates to a method and apparatus for processing image data for display by a display device.

  Known active matrix liquid crystal display devices include those that can be switched between a public display mode and a private display mode, and those that can guide different images to different observers. In the first mode (public mode), such a display is generally set as a standard display. A single image is displayed by the display device in the widest possible viewing angle range with optimal brightness, image contrast, and resolution for all viewers. In the second mode (private mode), the main image is visible only within a narrow viewing angle range centered on the normal line of the normal display surface. An observer viewing the display from outside this narrowed range of angles will either see the second image (masking image) that makes the main image appear blurry or the main image that has deteriorated so that it cannot be seen. Perceive.

  In GB 2413394 (Sharp), a display device with a switchable privacy protection function is produced by adding one or more liquid crystal layers and a polarizing plate to the display panel. The original viewing angle dependency of these added elements can be changed by electrically switching the liquid crystal by a known method. Examples of display devices that employ this technology include Sh851i and Sh902i type mobile phones manufactured by Sharp Corporation.

  However, these types of display devices can only selectively reduce the brightness of the main image to an observer outside the narrowed field of view, and the reconstructable color video image can be Cannot be displayed to a person. Multi-view display devices based on parallax barrier technology with this function are described in US 2007/0296874 and US Pat. No. 7,154,653.

  Each of the above methods has the disadvantage that extra devices must be added to the display device in order to provide the functionality of electrically switching the range of viewing angles. As a result, the cost and especially the bulk of the display device increase, which is very undesirable particularly in equipment equipped with a portable display device such as a mobile phone or a laptop computer. A display device based on a parallax barrier, even in public mode, can only display half of the display device's pixels for each viewing area, and the actual image resolution is half that of the underlying panel. .

  One example of a display device having a privacy mode function without increasing the hardware scale of the display device is a SH702iS mobile phone manufactured by Sharp Corporation. This mobile phone uses the processing of the image data displayed on the LCD together with the angle data / luminance characteristics inherent in the liquid crystal mode used in the display device to observe the display device from a position off the center. Create a private mode where the displayed information is not visible to the viewer. However, in the private mode, the quality of an image displayed for a regular observer (an observer on the axis) located on the axis is deteriorated to some extent.

  A multi-view display device based solely on image processing technology, which can provide detailed and reconfigurable side images in private mode without the need for a separate optical device for the display panel, is disclosed in GB-A-2428152A1 and It is described in British Patent Application No. 0804022.2. In these display devices, the main image data is processed in a manner dependent on the second image (masking image). As a result, when the corrected image data is displayed on the panel, the masking image is displayed on the axis. Perceived by a non-positioned observer (off-axis observer).

  Due to the high spatial frequency modulation applied to the main image to generate a multi-view, there are image features present in the main image, as described in UK patent application 0804022.2 Unwanted color artifacts become noticeable at the location. According to the patent application, an image processing filter (e.g. UK patent application No. 0701325.3 (publication number GB-A-) is applied to the main image in order to blur fine features in the main image. The appearance of these artifacts is greatly improved by applying the image processing filter described in 2445982). However, standard image blur filters include certain color artifacts associated with the process for multiview processing image data, including those described for use with multiview display devices as disclosed above. It is not designed to solve specifically. As a result, it has been shown that these filters are not effective enough to remove color artifacts for all types of images while minimizing blurring of the main image. In particular, these filters cannot correct color artifacts that occur around black and white text using the same parameter set.

  Therefore, a multi-view display device with almost no change in public mode, that is, relative to the corresponding standard display panel, brightness, resolution, contrast, field of view, etc. Multi-view or privacy based on image processing method, which optimizes the quality of the main image viewed by the main observer by including an image processing step that specifically corrects for the type of color artifacts generated by It would be desirable to provide a multi-view display device that has the ability to implement modes.

  According to a first aspect of the present invention, there is provided a method for processing image data for display by a display panel of a display device, wherein pixel data of an image representing an image is obtained, and in a first processing step, Processing the pixel data to create a multi-view effect for the observer, and in the second processing step, each of the plurality of subsets of pixel data comprises the same number of pixel groups, and each pixel group is at least For each subset with one pixel, for at least one pixel group of the plurality of pixel groups of the subset, depending on the pattern of pixel data in the plurality of pixel groups of the subset, at least one of this A method is provided for deriving new pixel data in such a way that such a pattern can be distinguished from its inverted pattern.

  The first processing step, on the axis, has a tendency to locally balance the observer on the axis located on the axis by spatial averaging, whereby the on-axis observation Introduce non-perceptible brightness variations, and other than on-axis, off-axis observers that are not on-axis are not locally balanced by spatial averaging, which It may be implemented in consideration of the characteristics of the display panel so as to introduce a variation in luminance perceivable by the viewer.

  According to a second aspect of the present invention, there is provided a method for processing image data for display by a display panel of a display device, wherein pixel data of an image representing an image is obtained, and in a first processing step, an axis Above, there is a tendency to balance locally by spatial averaging with respect to the on-axis observer located on the axis, which results in brightness variations that are not perceptible to the on-axis observer. In other than on-axis, off-axis observers that are not located on-axis are not locally balanced by spatial averaging, which results in a perceivable brightness for off-axis observers. The pixel data is processed in consideration of the characteristics of the display panel so that variation can be introduced. In the second processing step, each of the plurality of subsets of the pixel data includes the same number of pixel groups, and each image For each subset in which the group comprises at least one pixel, new to at least one pixel group of the subset of pixel groups, depending on the pattern of pixel data in the plurality of pixel groups of the subset A method is provided for deriving pixel data.

  In the first processing step, the luminance introduced on the axis for one pixel group of a pair of pixel groups perceived by the observer as having a single luminance by spatial averaging. Any increase may substantially coincide with an approximately equivalent decrease in luminance for the other of the pair of pixels.

  In the above method, the luminance obtained as a result of one pixel group of the pair of pixel groups is in the vicinity of the maximum luminance, or the other pixel group of the pair of pixel groups It may be configured such that the resulting luminance is either in the vicinity of the lowest luminance.

  In the derivation of the new pixel data, a filter is selected depending on the pattern of the pixel data, and the selected filter is applied to at least a part of the plurality of pixel groups of the subset to obtain a new one. Pixel data may be derived.

  The method may compare the pattern of the pixel data with a plurality of predetermined patterns and derive new pixel data depending on the result of the comparison step.

  In the method, each of the plurality of predetermined patterns is associated with a corresponding filter, a matching pattern is determined in the comparison step, and the filter associated with the pattern is derived in deriving the new pixel data. You may choose to use it.

  In the comparison step, the relative brightness of the at least one pixel group for which new pixel data is being derived is compared with at least one adjacent pixel group, and a relatively high brightness and a relatively low brightness. A measure for distinguishing between the two may be determined.

  The method assigns each pixel group in the subset to one of a plurality of levels forming a predetermined set according to the pixel data, and assigns a pattern of the assigned level in the comparing step to the plurality of levels. It may be compared with a predetermined pattern.

  The method includes the step of calculating a scale based on pixel data of at least some of the plurality of pixel groups of the subset and assigning the pixel group to one of the predetermined plurality of levels. Depending on the calculated scale, it may be implemented.

  According to a third aspect of the present invention, there is provided a method of processing image data for display by a display panel of a display device, wherein pixel data of an image representing an image is obtained, and in a first processing step, Processing the pixel data to create a multi-view effect for the observer, and in the second processing step, each of the plurality of subsets of pixel data comprises the same number of pixel groups, and each pixel group is at least For each subset having one pixel, a scale is calculated based on the pixel data of at least some of the plurality of pixel groups of the subset, and each pixel group in the subset is used as its pixel data. Therefore, depending on the calculated scale, it is assigned to one of a plurality of levels in a predetermined set, and the pattern of assigned levels is assigned to a plurality of places. Compared with the pattern, depending on the result of the comparison step, to derive the new pixel data for at least one pixel group of the plurality of pixel groups of the subset, a method is provided.

  The scale may be an average value.

  In the method, the pixel group is set to one of a plurality of levels forming a predetermined set based on a comparison between the pixel data and the calculated scale, for example, using the calculated scale as a threshold. It may be assigned.

  The method may convert the pixel data into an apparent luminance value, for example based on a gamma power function, for use in the derivation step.

  The method may use at least one conversion parameter determined depending on a pattern of the pixel group of the subset.

  The subset of pixels may be continuous and may extend substantially in one dimension.

  The subset of pixels may have a continuous two-dimensional configuration.

  Each of the subsets may be separated from the other subsets by one pixel group.

  The above step of deriving new pixel data may use knowledge obtained by the processing performed in the first processing step.

  The new pixel data may be derived so that a pattern of pixel data representing image features can be generated that is likely to cause artifacts as a result of the processing performed in the first processing step.

  In the derivation step, the at least one pixel group from which new pixel data is being derived is a pixel data pattern representing image characteristics that is likely to cause artifacts as a result of the processing performed in the first processing step. It may be determined whether to form a part.

  In the deriving step, a pattern of pixel data representing a bright feature on a relatively dark background may be treated separately from a pattern of pixel data representing a dark feature on a relatively light background.

  In the deriving step, each of the at least one pixel group from which new pixel data is being derived is adjacent to a dark or bright line having a width of one pixel group, and a dark or bright line having a width of one pixel group. At least of the left edge of a dark or bright line having a width of two pixel groups, a checkered pattern consisting of light and dark having a pitch of one or two pixel groups, and an image feature consisting of diagonal lines Whether or not to form a part of one feature may be determined from the pattern of the pixel data.

  The second processing step may be performed before the first processing step.

  The second processing step may be performed after the first processing step.

  The new pixel data may be derived only for one pixel group of the subset of pixel groups.

  Each pixel group may include a composite color pixel group including a plurality of color component pixels, and the above method may be applied to each color component pixel in order.

  The composite color pixel group may include red, green, and blue color component pixels.

  The first processing step is performed to interleave received pixel data representing different images, and to present the different images to a plurality of viewers at different viewing positions. May be.

  According to a fourth aspect of the present invention there is provided an apparatus configured to perform a method according to any one of the first to third aspects of the present invention.

  According to a fifth aspect of the present invention, there is provided a display device comprising the apparatus according to the fourth aspect of the present invention.

  According to a sixth aspect of the present invention, there is provided a program for controlling an apparatus to perform a method according to any one of the first to third aspects of the present invention. This program may be conveyed by a conveyance medium. The carrier medium may be a storage medium or a transmission medium.

  The characteristics of the display panel considered above are that the variation in brightness introduced on the axis tends to balance locally for the observer by spatial averaging and is therefore not perceptible to the observer on the axis. It may be the responsiveness of the on-axis luminance to the signal voltage of the display panel in a state where it is configured to be possible. In this case, the relationship between the off-axis luminance and the on-axis luminance of the panel becomes nonlinear, and as a result, the variation in luminance introduced off-axis is locally balanced for the observer by spatial averaging. Rather, it is perceptible to the on-axis observer.

  One embodiment of the present invention provides image data processing and multiple images to multiple images in order to maintain high image quality viewed on-axis without introducing unwanted image artifacts by the multi-view data processing process. An improved image processing method for realizing these types of display that enables privacy display and multi-view display of the kind that utilizes the inherent viewing angle dependency of the LCD panel that can be viewed by an observer. provide.

  In the first embodiment of the present invention, it is known that at least one of the plurality of image data sets input to the display device causes image artifacts when the multi-view image process is performed. In order to detect the characteristics of the particular image being processed, it is processed using a filter. These particular image features are then processed in a way that prevents the introduction of artifacts while preserving the maximum details of the original image, depending on the type.

  In the second embodiment of the present invention, the localized pixel group in the output image generated by the multi-view image process corresponds to the same region of at least one input image of the plurality of input images. Compare with group. A mismatch is then detected, and the multi-view output image is changed, for example, in a way that corrects the mismatch or diffuses the mismatch over a large area of the image to make it inconspicuous.

FIG. 2 is a schematic diagram of a standard arrangement of control electronics for a liquid crystal display device. It is a figure which shows the flow of the data process in a standard single view LCD. FIG. 5 is a diagram showing a data process flow in a known type of multi-view LCD in a private or multi-view mode. FIG. 6 is a schematic diagram showing output pixel data obtained by a known multi-view display process that operates on uniform main input image data and side input image data regions. FIG. 6 is a schematic diagram showing output pixel data obtained by a known multi-view display process that operates on a shaded area of a main input image and a uniform area of a side input image. FIG. 3 is a schematic diagram of a flow of an image data process in a known multi-view display device using a preprocessing filter added separately for a main input image and a side input image. In the first step (classification step) of the first embodiment, in the 4 × 1 pixel sampling window of the main input image, the pixel is set to “high” (1) or “low” based on its image data value. It is a figure which shows the preferable method classified into (0). It is a table | surface which shows the mutually different calculation implemented in a 3rd step in the case of ten different specified in the 2nd step of 1st Embodiment. It is a figure which shows an example of the implementation aspect by the hardware of 1st Embodiment. It is a figure which shows the calculation method used in the 3rd (calculation) step of the 2nd example of 1st Embodiment. FIG. 11 shows the result of the calculation of FIG. 10 using non-linear pre- and post-transformation of data values using different “gamma” parameters and acting on two input data sets (a) and (b). . It is a figure which shows each example of the six different cases specified in the 2nd step of the 3rd example of 1st Embodiment. It is a figure which shows an example of the calculation implemented with respect to input main image data in a 3rd step in each case specified in embodiment of FIG. It is a figure which shows an example of the implementation aspect by the hardware of 1st Embodiment. It is a figure which shows the flow of the image data process in the LCD display device of 2nd Embodiment. It is a figure which shows the calculation method used in 2nd Embodiment. FIG. 17 is a diagram illustrating the ability of the calculation method of FIG. 16 to generate different results depending on the kernel position in the output multi-view image even when the input main image is uniform.

  In a preferred embodiment, the display device consists of a standard LCD display device with modified control electronics. An LCD display device is generally composed of a plurality of component parts including those listed below.

  1. A backlight unit for supplying uniform, wide-angle illumination light to the panel.

  2. Control electronics for receiving digital image data, outputting an analog signal voltage for each pixel, and outputting a timing pulse and a common voltage for the counter electrode of all pixels. A schematic diagram of a standard arrangement of LCD control electronics is shown in FIG. 1 (Ernst Lueder, Liquid Crystal Display Device, Wiley and Sons Ltd, 2001).

  3. It consists of two glass substrates facing each other. On one glass substrate, a pixel electrode array and an active matrix array that receives electronic signals from the control electronics and guides them to the pixel electrodes are installed. LC panel for displaying images by modulation. The other substrate is usually provided with a uniform common electrode and color filter array film. A liquid crystal layer having an arbitrary thickness (usually 2 μm to 6 μm) is sandwiched between the glass substrate and the liquid crystal layer because the alignment layer exists on the inner surface of the glass substrate. It may be arranged. The glass substrate generally causes an electrically induced alignment change in each pixel region of the LC layer to achieve the desired optical modulation of light coming from the backlight unit and the surroundings, and thereby image In order to produce, it arrange | positions between the some polarizing film and other optical compensation film | membrane arrange | positioned in the orthogonal direction.

  FIG. 2 schematically illustrates one embodiment of the invention disclosed in UK patent application No. 0804022.2 operating in a public display mode. In general, the LCD control electronics (also referred to as control electronics here) 1 is observed from the normal direction of the display surface (on the axis) in order to output a signal voltage depending on input image data. It is specifically adapted to the electro-optical characteristics of the LC panel 2 so as to optimize the perceived quality of the displayed image, ie, resolution, contrast, brightness, response time, etc. for the observer 3. Yes. The relationship caused by the display device (gamma curve) between the input image data value for a given pixel and the observed luminance (gamma curve) is the mapping between the display driver data value and the signal voltage and the LC panel. It is determined by the effect of combining the responsiveness of the luminance with respect to the signal voltage.

  The LC panel 2 generally has a plurality of LC domains and / or passive optical compensation films per pixel in order to keep the gamma curve of the display device as close to the on-axis responsiveness as possible for all viewing angles. It is comprised including. By doing so, the LC panel 2 provides substantially the same high-quality image with respect to the wide viewing area 5. However, the electrovisual response of the liquid crystal display device depends on the angle, and the fact that the off-axis gamma curve is different from the on-axis gamma curve is an inherent characteristic of the liquid crystal display device. This will generally not be a noticeable defect that is noticeable in the viewed image for off-axis observers 4 unless it causes contrast reversal, large color shifts, or contrast reduction.

  When the display device is operating in the public mode, a set of main image data 6 constituting a single image is input to the control electronics 1 in the form of a serial bit stream, usually during each frame period. The Then, the control electronics outputs a set of signal data voltages to the LC panel 2. Each of these signal voltages is guided to the corresponding pixel electrode by the active matrix array of the LC panel. The resulting collective electro-optic responsiveness of the pixels in the LC layer produces an image. Then, almost the same image is perceived by the on-axis observer 3 and the off-axis observer 4. At this time, it can be said that the display device is operating in the wide viewing angle mode. This situation is illustrated in FIG. 2, which can be said to be the standard operating method of an LCD.

  In a multi-view display device of the kind described in UK patent application No. 0804022.2 (also schematically shown in FIG. 3), in private mode, main image data 7 and side images 8 constituting the main image are displayed. Two image data sets with the constituting side image data 8 are input to the control electronics 1 during each frame period. Then, as described above, the control electronics outputs a set of signal data voltages including one data voltage for each pixel in the LC panel. However, here, the control electronics (display controller) uses an extended look-up table (LUT), and the signal data voltage output for each pixel in the LC panel forming one image as a whole. Depends on the data value of the corresponding pixel in both the main image 7 and the side image 8 (with respect to the spatial position in the image). The data voltage output for each pixel also depends on a third parameter determined by the spatial position of the pixel in the display device.

  The output voltage from the control electronics 1 causes the LC panel 2 to display an image composed of a combination of pixels. When this image is observed by the main observer 3, it becomes the main image. However, because the gamma curve is different for off-axis observer 4 (this is one feature of the LC panel), off-axis observations most clearly show side images that blur and / or degrade the main image. Perceive. By doing so, main image information can be presented only to an observer located inside a limited cone angle 9 centered on the normal of the display device. This situation is illustrated in FIG. Further details are described in British Patent Application No. 0804022.2.

  The image data compression and combination process and parameters described in the multi-view process of UK Patent Application No. 0804022.2 generate subtle color artifacts that are noticeable to the on-axis observer 3. This is particularly true in the case of an area consisting of diagonal lines having a width of one pixel in the input main image. This is due to the following facts. That is, such a process often produces an output image in which every other color subpixel is set to black, so that a black diagonal line with a width of one pixel is superimposed on the pattern, thereby , A line of pixels on one side of the diagonal can leave one or two color sub-pixels included in the line on for the entire length of the line. In this case, the colored line can be recognized by the eyes.

  This problem is illustrated in FIGS. FIG. 4 shows the result of the multi-view image process acting on the luminance value of a pixel group consisting of 2 × 2 pixels in a uniform main image and side image in a typical RGB stripe display. Show. Therefore, even if the multi-view process outputs image data, this output is changed at the sub-pixel level. However, the multi-view process changes the overall luminance and color for a pixel group composed of 2 × 2 pixels. It can be seen that the balance is maintained. The set of output pixels then looks the same as the input main image to an observer located at a sufficient distance so that only the averaged pixel brightness of the pixel group is visible.

  In FIG. 5, the input side image is uniform, but the input main image region is formed by dark diagonal lines, and the same multi-view process as described above is performed in the uniform side image. A state in which a pixel group composed of × 2 pixels is operated is shown. It will be appreciated that the multi-view process results in output image data that varies significantly in both overall brightness and color balance. This discrepancy is visible to the on-axis observer, and in this example, a clear green artifact is visible.

  Depending on the relative position of the diagonal lines for the bright and dark subpixel patterns, the appearance of the image artifacts will change. For example, in the case of a dark diagonal line in the input main image having a slope opposite to that in the example shown in FIG. 5, the output pixel subset appears to be magenta as a whole. All color subpixels within a plurality of composite white pixels are processed by a multi-view image process rather than patterning at the subpixel level as described in UK patent application 0804022.2. If the luminance is increased and the luminance of all the color components of the adjacent composite white pixels is decreased accordingly, all the oblique lines can be eliminated.

  British Patent Application No. 0804022.2 adds preprocessing steps 10 and 11 that operate on the input main image data set and side image data set, respectively, in the process of combining the multi-view images. Have been described. These pre-processing steps 10 and 11 consist of image blur filters to prevent the occurrence of these image artifacts. This change to the display process is shown in FIG. 6 of the accompanying drawings (corresponding to FIG. 17 of UK patent application No. 0804022.2).

  In a preferred embodiment of the present invention, using the image data processing performed by the multi-view image processing method and the knowledge of the image artifacts generated by this processing, the most efficient change and computational resources to make to the input image With minimal changes to use (for example, one of fewer gates, less memory capacity to buffer data, slower timing, shorter latency, and less power consumption) An improved image pre-processing method is provided that modifies the input image data set to prevent the appearance of artifacts in the output image.

  In a preferred method, pixel data of the input image is buffered by a plurality of pixels when received by the added image processing device 10 or 11. This makes it possible to sample all subsets of adjacent pixel data values within a “window” or “kernel” on the image. Each such window or kernel may be considered to comprise a subset of pixel data, each subset of pixel data comprising the same number of pixel groups, and each pixel group comprising at least one pixel. For example, each pixel group may include a single pixel, or may be a composite color pixel group including red, green, and blue pixels. In order to simplify the description, the pixel group may be simply referred to as a pixel here.

  A process consisting of the three steps described below is then applied to each subset.

  The first step (classification step) is used to classify the pixels in the subset as pixels having a “high” value or pixels having a “low” value. This classification step is a step of assigning pixels to one of a plurality of levels (in this case, two levels of “high” and “low”) that form a predetermined set based on brightness. You may think. If the pixel value is associated with a single monochrome pixel or a single black and white pixel, the pixel values of adjacent pixels are processed. However, in the case of the composite color pixel group, since each composite color pixel group includes red, green, and blue pixels, the process is sequentially applied to each color component pixel value of the adjacent composite pixel group.

  In the second step (comparison step or case detection step), the resulting high / low pattern is compared with a set of known patterns.

  If a matching pattern is found, a third step (calculation step) is performed on at least one of the pixels in the subset. The sampling window is then moved in the image to sample a new set of pixels. If no matching pattern is found, the pixel value is not changed and the sampling window moves to the next (if no matching pattern is found, at least conceptually, it simply receives the pixel data of the existing image and It may be considered that a filter that is used as new pixel data without changing is applied). When the window has scanned all the main input images and made any necessary changes to the image data values, the adjusted image data is output from the pre-processing circuit to the multi-view circuit.

  In an exemplary embodiment, the sampling window, or “kernel”, is a horizontal block of 4 × 1 pixels. In the classification step, the data values of all the pixels are averaged, and a pixel in the kernel having a data value higher than this average value is defined as having a high value, which is denoted as “1”. In addition, a pixel having a data value lower than the average is defined as having a low value, that is, “0”. This process is illustrated in FIG. 7, using pixel values in one row as an example.

Then, the 4 × 1 pattern composed of 0 value and 1 value is compared with a set of known patterns, and the operation corresponding to the current pattern is the second pixel (P _x ) from the left of the kernel. To be implemented. Then, the 4 × 1 window moves to the right by one pixel in the image, and the high / low classification, the pattern comparison, and the operation for the second pixel are repeated. This process continues until the kernel finishes scanning across all rows in the image. The modified data value obtained from the previous step may be used in subsequent steps. Alternatively, the original data value may be used. For example, in the first stage, the pixel 2 (second pixel from the left) of the 4 × 1 window may be changed. In this case, in the second stage, the pixel is in the 4 × 1 window. Pixel 1 is reached. In the first embodiment, the new value of pixel 1 is used in the calculation in the second stage. On the other hand, in the second embodiment, the original value of pixel 1 is used. For video display, this process occurs in real time, filtering each frame of the video sequence within the frame time, and outputting the corrected image to the display device.

  As described above, when each pixel group is formed by individual color component pixels, the above process is repeated for each color component. For example, the illustration of FIG. 7 may be considered to represent the respective values of only the red component of four adjacent pixels in a 4 × 1 window, or the respective values of only the green component or only the blue component. You may think that it represents the value of. In another embodiment, the window can consider and process values from multiple color components at once, thereby allowing more complex color patterns to be generated.

In an exemplary embodiment, for 10 of the 16 possible kernel patterns, artifacts will occur in the multi-view image unless the operation is performed on the second pixel from the left. In order to correct this, it has been found that it is necessary to perform an operation on the second pixel from the left. These patterns and the corresponding operations are shown in FIG. Note that P _x ′, that is, the pixel value used to calculate the new value of the second pixel from the left in the kernel is the value of the pixel P _x and its neighboring pixels P _x−1 and P _{x + 1} . And not the 0 or 1 value used to represent the high / low values in the first step of the process. It is also not a new corrected pixel value obtained from a previous step where the kernel is at a different position. It should also be noted in the table of FIG. 8 that one embodiment of the present invention can distinguish each pattern from its inverted pattern. For example, pattern 1 is distinguished from an inverted pattern (pattern 2) of pattern 1. Pattern 3 is distinguished from the inverted pattern (pattern 4) of pattern 3. Different filters or operations are applied to each pattern and the respective inversion pattern corresponding to that pattern. One embodiment of the present invention can implement such an application by looking at patterns in pixel data rather than patterns in absolute differences derived from pixel data (also implemented in GB-A-2445982). ing). However, it is not necessary to distinguish all individual patterns from their inverted patterns. The same filter or operation may be applied to a particular pattern and its inverse pattern, and in such a situation, it is not necessary to distinguish between them. It is also necessary to distinguish a particular pattern from its inversion pattern (eg, 1101 from 0010), but the filter applies only to one of the two (eg, 1101) and the other pattern (eg, 0010). May not need to be filtered. In this case, it is only necessary to detect the presence of one pattern, and it is not necessary to detect the presence of both patterns. In this context, the inverted pattern of pixel data may be considered a pattern that, when combined, generates a substantially uniform image. For example, “bright” is the reversal of “dark” and vice versa.

Arbitrary values 0 to 1 may be assigned to parameters used in calculating P _x ′ (BN, WN, BL, WL, BE, WE, WC, and BC). When a particular operation is carried out, these parameters, determines the amount of data values of P _x is shifted toward the data values of adjacent pixels. In order to minimize the amount of change applied to the image and to prevent any image artifacts from appearing in the multi-view image, these parameters are used to determine the optical characteristics (brightness, contrast, gamma) of the display device. It needs to be set accurately according to the curve. In order to optimize the efficiency of the filtering process, the parameter is a fraction of the integer divided by 2 ⁿ (ie m / 16), where n and m are positive integers, so that the division step is a simple bit shift. Or m / 32).

  One of the advantages of this embodiment over the prior art image blur filter is that the pixel to be computed is part of a dark or bright line having a width of one pixel by the multiple case detection step. Whether it is a pixel adjacent to a dark or bright line with a width of 1 pixel, whether it is the left edge of a dark or bright line with a width of 2 pixels, or light and dark with a pitch of 2 pixels It is possible to determine if it is part of a checkered pattern. These are the main image features that cause artifacts in the multi-view image process. Also, in order to optimally correct the artifacts that appear if they are not corrected, each feature requires that different operations be performed in each case. In particular, white features may be treated separately from black features. In addition, a pixel adjacent to a narrow feature may have a different amount of change from a pixel constituting the feature itself. By doing so, the appearance of both white and black lines can be improved using the same filter. That is, the ability to significantly improve the perceived quality of an image that has undergone a multi-view process is realized. For each case that matches the pattern, this can be achieved by using different operations with individually adjusted parameters.

  In the above, the process is described as running the kernel from left to right over the entire image, but the kernel is run in the reverse direction or the vertical direction without changing the overall performance of the process. In this case, corresponding changes may be made to the pixels in the kernel whose data values are changed at each step.

  Although the above classification steps have been shown to produce effective results, there are other ways to assign pixel data values of the input image to the 0 state / 1 state for comparison with a particular case pattern. It should also be noted that there are many methods that are equally effective and that may be more effective. A simple threshold data value may be applied in which a pixel is classified as “low” if the data value is less than the threshold, and a pixel is classified as “high” if the data value exceeds the threshold. A pixel having a data value in a predetermined band may be applied with a threshold having a band to which neither a “high” status nor a “low” status is assigned. A pixel in the subset may be assigned a “high” or “low” to match the relative data values for its neighboring pixels. For this method, “diffusion” may be applied, in which a pixel is not classified as “high” or “low” unless the difference between adjacent pixels is less than a predetermined amount. These methods, combinations thereof, and other methods that perform the same classification function are also within the scope of the present invention.

  Further, without providing an independent classification step, for example, the case detection step may directly process the image data. In this case, the case detection step may be modified, for example, if necessary, based on the image data itself by a predetermined case detection algorithm that takes image data as input (rather than pre-processed classification values). Determine what is needed. In this case, it can be considered that the first step and the second step are combined, and classification and case detection are performed as one step. In essence, this step classifies the pixel data into one of a plurality of categories each having an associated filter. It is also possible to combine all three steps into one. In this case, new pixel data need only be derived in some way based on the brightness pattern of the subset of pixels, and this may be achieved using any method or any number of steps. Another possibility is that in the first step, one or more “feature indicators”, each reflecting one or more corresponding measures associated with certain known image features, are derived from the data of the subset of pixels. It may be calculated. As the feature indicator, for example, the feature indicator may reflect how much the pixel appearance is “checkerboard”. For example, a high value may reflect a very high checkered pattern appearance, and a low value may reflect a less high checkered pattern appearance. In this case, these “feature indicators” are used to determine what image features actually exist in the second step (or what features are most prevalent) and what filters in the third step. May be determined to be most appropriate.

  It should be understood by those skilled in the art that the image processing method including the above-described embodiments may be executed in the form of software running on a processing unit that provides data to a display device. The process may also be performed in the form of hardware in an application specific integrated circuit (ASIC), field programmable gate array (FPGA), or other electronic circuit integrated in existing display control electronics. Absent. One of the advantages of the embodiment that does not use the computational resources of this embodiment is that the required circuit may be stacked on the glass substrate of the display device together with the transistor device of the pixel. This saves cost and allows the circuit to be easily adapted to each display device to be implemented.

An example of a circuit design that implements the required process is shown in FIG. In FIG. 9, it can be seen that the process described above is implemented in the following manner. That is, the pixel data value of the image corresponding to the current position of the kernel is stored in the buffer memory. These values are input to a high / low detector and an average value calculator for the first step (classification step) and a parameter multiplication process for the third step (calculation step). The output of the high / low detector determines which of the available parameters input from the register block is applied to the pixel value from the buffer to calculate the correct output value in the third step (calculation step). Is used by the pattern match selector in the second step (case detection step). Available parameters are shown in the figure as WN0 / 1, BN0 / 1, WL0 / 1, BL0 / 1, WE0 / 1, BE0 / 1, WC0 / 1, and BC0 / 1. Incidentally, shaped portion subscript of 0/1, the parameters, whether a parameter for application to P _x, varies depending on whether whether a pixel currently being changed, or the pixel of the immediately adjacent .
(For example, to use the values given in FIG. 8, WN1 is equal to 1-WN0).

  Although FIG. 9 shows that the bit depth of the input image data is 6 bits / color channel (that is, 18b per pixel), this figure is only an example of the embodiment and is shown in FIG. Functional processes can be applied to images of any bit depth.

  In the described embodiment, a technique using a simple threshold value can be used in the first step (classification step). In this case, it is also possible to input a BT value and a WT value to determine a data level threshold that distinguishes between a high state and a low state. In addition, these values are used in conjunction with the kernel average value to define bands above and below the kernel average value, so that pixel data values must always be outside this band in order to be classified as high or low. Good.

  The process described above significantly increases the perceived quality of images that undergo a multi-view process, and this process does not require significant computational resources, while providing better performance with more computational resources. It should also be noted that the process variations that can be achieved are possible. As described above, diagonal lines in the input image are characteristics that usually generate the most noticeable image artifact in the multi-view image. Therefore, use a processing method that can distinguish diagonal lines from horizontal and vertical lines, since only non-hatched lines need to be relatively small to preserve larger overall image clarity. Thus, it is desirable to apply the necessary corrections. This is not possible with a single-line kernel, but is used to increase the area of the kernel to cover more than one row and to compare and determine which operation to apply. If the number of specific patterns and the number of available operations are increased, the process can be preferentially processed even if the above-mentioned process remains basically the same. This provides better image quality in the output multiview image. Thus, the shape of the image scan kernel used may be 4 × 2 pixels, 5 × 2 pixels, 3 × 3 pixels, or any that is known to provide optimal performance without departing from the scope of the present invention. It may be a shape.

  In a second embodiment as an example embodiment, a window of 4 × 3 pixels is used to sample each pixel of the main image and surrounding pixels in the first step (classification step). In the same manner as described above, in the second step (case detection step), a plurality of possible cases include the pixel configuration in the window together with the step of converting the non-linear data value into the “pixel appearance” (eg, “gamma” power conversion). In the third step (calculation step), a blurring mechanism based on 3 × 3 pixels is used. As above, the case of a composite color pixel group, since each composite color pixel group comprises red, green, and blue pixels, the process is applied sequentially to each color component pixel value of the image. In the case where the pixel value is associated with a single monochrome pixel or a single black and white pixel, the pixel values of eight adjacent pixels are processed. In this example, a 4 × 3 window is used in the first step, for example instead of a 3 × 3 window, as used in the third step. This is because a 4 × 3 window should provide higher reliability in distinguishing white text from black text. However, it will be understood that any size window may be used. Also, for storage purposes, the smaller the window, the better.

  The first step (classification step) of this embodiment may be the same as in the previous method, but may be extended for use in a 4 × 3 pixel window. That is, an average pixel data value of the subset is calculated, pixels having a data value larger than the average value are denoted as “1” pixels, and pixels having a data value smaller than the average value are denoted as “0” pixels.

  In the second step (case detection step), the 3 × 3 pixel kernel automatically applies a greater degree of change to the diagonal feature than the horizontal or vertical feature. Having great functionality means that there are fewer individual cases that must be detected and specific operations applied. It has been found that only three general cases require the application of filtering operations specific to each case based on detection and 3 × 3 pixels. These cases can also be described as light narrow features on a dark background, dark narrow features on a light background, and checkered patterns with a single pixel pitch.

  A relatively simple method of determining the case of two fine features sums the number of “1” type pixels in a 4 × 3 window. If the 12 pixel window contains 6 or more “1” type pixels, this case may be defined as a dark feature on a light background. If the 12 pixel window contains less than 6 “1” type pixels, this case may be defined as a bright feature on a dark background.

  In the case where there is a checkered pattern with a pitch of one pixel, in the 4 pixels on the left side of the 4 pixels in the central horizontal line in the 4 × 3 window, the types of classified pixels appear alternately ( For example, 101 or 010), and the inverted pattern may alternately appear (for example, 010 or 101) in any of the three pixels above or below the left three pixels.

  In the third step (calculation step), the new data value is transferred to the second pixel from the left in the center row of the 4 × 3 pixel subset, and the original data value of this pixel and It may be written based on the data values of 8 pixels (ie, the data values of the leftmost 9 pixels of the 12 pixels in the 4 × 3 subset used in the second step). The 3 × 3 pixel operation used may be that described in the disclosure of UK patent application No. 0701325.3. Briefly described, an overall measure of the appearance of each pixel is calculated based on the weighted sum of the data value of the pixel in the main image and the data values of the 8 pixels immediately adjacent thereto.

  Next, in approximating the effect of the multi-view process on a pixel subset, the second overall apparent value is set to 0 for the same subset of pixels, but immediately next to adjacent pixels. Is calculated by The weighted sum values are normalized and, if necessary, set equal to each other by calculating a new value for the central pixel data value, P '(x, y). The relationship expressed by this equation is shown in FIG. Thus, the equation for finding a new data value for the central pixel P ′ (x, y) is given by:

Here, ω ₁ , ω ₂ , and ω ₃ are weighting parameters used, and P (x, y) and the like are pixel data values at respective coordinates in the main image. Similar to the previous example, when a new value for the pixel in the middle of the 3x3 window is calculated, this value is written to the output image ready to be input to the multi-view image process, and the process consists of three steps. The whole is repeated for the next pixel in the input main image until the entire image is scanned and processed by the window.

  This method can be optimized by selecting pixel weighting parameters so that a greater degree of change is automatically applied to features consisting of diagonal lines than features consisting of horizontal and vertical lines in the image. . By doing so, the number of specific cases that need to be considered in the second step (case detection step) is reduced. Also, as described above, in the three cases of a bright feature on a dark background, a dark feature on a white background, and a checkered pattern with a single pixel, an operation based on only three different 3 × 3 pixels It has been found that the appearance of the image can be improved by applying.

In the three cases, the weighting parameter of the process of equation (1) is the center of the apparent brightness of the pixels in the 3 × 3 subset as described in UK patent application No. 0701325.3. Assuming that it follows a two-dimensional Gaussian distribution as a function of distance from, it may be determined by quantifying the influence of surrounding pixels on the center pixel. For example, assuming a two-dimensional Gaussian distribution with a standard deviation of 0.5 pixel width, the weighting parameters ω ₁ = 0.0240, ω ₂ = 0.1070, and ω ₃ = 0.4759 are obtained.

  These same weighting parameters may be used in the third step (calculation step) in the case of both light features on a dark background and dark features on a white background. However, the degree of change and the overall trend of the process that emphasizes the appearance of narrow (single pixel) dark features, narrow bright features, or both, is the primary image data value apparent brightness The value may be controlled, for example, by converting using a gamma power function expressed by the following equation.

Here, L is an apparent luminance, D is a data value (0 to 255), gamma is a power coefficient, and each is a value before the obtained value is processed by a calculation based on 3 × 3 pixels.

  If the gamma value (conversion parameter) used in this conversion generates a relationship between the data value and the luminance that accurately reflects the responsiveness of the luminance to the pixel data value in the display device itself, 3 × 3 pixels Based on the darkness in the main image observed on the display device while still providing sufficient smoothing effect on the steep features of the image and preventing the introduction of color artifacts by the multi-view process And maintain the overall brightness of bright features.

  If the gamma value used is lower than the gamma value representing the brightness responsiveness to the data value of the display device itself, the overall tendency of the calculation based on 3 × 3 pixels described in Equation (1) is as follows: Emphasize the appearance of dark features in the main image. This is due to the fact that after processing, the dark pixels adjacent to the bright pixels remain dark, while the bright pixels adjacent to the dark pixels are significantly darker.

  If the gamma value used is higher than the gamma value representing the responsiveness of the luminance to the data value of the display device itself, the overall tendency of the calculation based on 3 × 3 pixels described in Equation (1) is Emphasize the appearance of bright features in the main image. This is due to the fact that after going through the process, the bright pixels adjacent to the dark pixels remain bright, while the dark pixels adjacent to the bright pixels are significantly brighter.

  If the 3x3 pixel based filter process enhances both the appearance of light and dark narrow features, the overall appearance of the main image after the multiview process has been applied is improved. It may be. In this case, if the second step (case detection step) suggests that the 3 × 3 window consists of image areas containing bright features on a dark background, a higher gamma value is used in the filter operation. It can be used. On the other hand, if the second step (case detection step) suggests that the 3 × 3 window consists of an image area containing a narrow dark feature on a light background, a lower gamma value is used. It can be used in filter operations, and in this way, the necessary effect is realized in either case.

  After the 3 × 3 pixel based filter process is applied to the gamma transformed pixel data value, the resulting output value is transformed using the inverse of equation (2) shown in equation (3) to obtain an equivalent New data values may be returned.

Where D ′ is the new equivalent data value, L ′ is the new apparent luminance value after processing by the filter based on 3 × 3 pixels, and gamma is the same power factor used in equation (2). .

  FIG. 11 shows the effect of the filter operation based on 3 × 3 pixels described in Expression (1) on two input main image regions including (a) black diagonal lines and (b) white diagonal lines. Note that the pixel data values shown in FIG. 11 are pixel data values that have been pre- and post-transformed according to equations (2) and (3) using gamma values of 1 and 2.4, respectively. To enhance how the selection of the gamma affects different ways of changing the bright and dark lines, and to emphasize both the appearance of light and dark narrow features in the image, calculate It can be seen from FIG. 11 why it is desirable to select the gamma value used in, according to the results of the previous case detection step.

  The result of the second step is that a checkerboard pattern having a pitch of one pixel exists in the region of the main image that is the sampling target of the 4 × 3 window in the second step (case detection step). Suggests that in the third step (calculation step), the equal weight assigned to the sum of the central pixel and the four neighboring pixels is changed to: An operation of a weighted total value value based on three pixels may be performed.

Again, the pixel data value P (x, y) etc. may be transformed by a non-linear method (e.g. the above-described gamma-dependent process) before the operation is applied.

  By doing this, this second example is more accurate than the first example in that the amount of unnecessary changes made to the input main image is reduced, and from the image that results from the multiview process. Provide a means to remove any unwanted color artifacts. Note that this reduction in the amount of unnecessary change is performed by performing memory necessary for buffering pixel data values of 4 × 3 blocks (not 4 × 1 blocks) and necessary pixel data change operations. Both of the processing powers to be generated occur in exchange for an increase in demands on the computing resources of the display device.

  In an example of the third implementation of the first embodiment, a 4 × 3 pixel window may again be used in the first step (classification step) and the second step (case detection step). Again, a 3 × 3 pixel window may be used in the third operation. However, removing the color artifacts from the final multiview image maintains comparable performance and minimizes the effect of blurring the main image while significantly reducing the computations performed in the third step. The number of cases detected in the second step may be increased from 3 to 6 compared to the previous example in order to be able to simplify and reduce the demand for computing resources of the display device. . This increase in the number of cases detected can simplify the blurring operation performed in all six cases to a standard normalized weighted sum of a 3x3 pixel subset. become. In this simplification, the weights of the pixels used are all fractions obtained by dividing the integer by 32, and it is not necessary to nonlinearly convert the pixel data value to an apparent luminance value at an arbitrary point. is there.

  In the first step (classification step), the embodiment may be the same as in the previous example. That is, an average pixel data value of the subset is calculated, pixels having a data value larger than the average value are denoted as “1” pixels, and pixels having a data value smaller than the average value are denoted as “0” pixels.

  In the second step (case detection step), in order to double the number of specific cases from 3 to 6, the three main cases of the previous example (checkered pattern, dark width on a light background) Narrow features and light narrow features on a dark background) are the same as the previous process (compared to known patterns for checkered cases, the number of pixels of type “1” for other patterns) In addition to detecting by summation, the classification of the central pixel of the 3 × 3 subset used in the third step (calculation step) is also considered. For example, a checkerboard pattern with a central pixel type of “0” is a different case, which is different from a calculation step applied to a checkered pattern with a central pixel of type “1”. Arithmetic steps are applied. An example of six detected cases is shown in FIG. 12 ("?" In the pixel classification suggests that the type of pixel at this position in the subset is independent of the case determined).

  In the third step (calculation step), using the normalized weighted sum calculation, using different weighting parameters for each of the six different cases distinguished by the second step (case detection step), 3 A new value for the center pixel of the x3 subset is calculated. In each calculation, the weighting parameter may be a fraction obtained by dividing the integer by 32, and non-linear conversion of the pixel data value is unnecessary, and this simplifies the calculation method. This process is illustrated in FIG. Note that the weighting parameters shown in FIG. 13 are weighting parameters that are known to produce good results in the final multi-view image for each case. As can be seen from FIG. 13, each of the six operations is a simple normalized weighted sum, except in the case of “adjacent pixels that are white text” where a negative weighting parameter is used for the corner pixels. Value. By doing this, if the central pixel is adjacent to the diagonal line, the major change required to remove color artifacts from these regions of the final multiview image is Added.

  An example of a circuit design for executing a necessary process is shown in FIG. In this figure, the process described above is similar to that shown in FIG. 9, except that the pixel buffer step and operation step processes have been expanded to incorporate windows of 4 × 3 and 3 × 3 pixels, respectively. It can be seen that it operates in a similar manner. In FIG. 14, the parameters input from the register block are indicated by A to F for the six different cases detected in the second step. Also, a subscript of 0, 1, or 2 is added, and the parameter can be applied to the central pixel in the third step (calculation step) or applied to one of the pixels adjacent in the horizontal direction. Whether it is possible or applicable to one of the pixels adjacent in the diagonal direction. For example, considering the case shown in detail in FIG. 13, if an adjacent pixel of a case = white text is expressed as “F”, F0 = 4, F1 = 9, and F2 = −2.

  In this case, even if any of the processes described above is executed, there is a possibility that there is an image area that generates a new pixel value outside the normal range (for example, 0 to 255). New values outside these ranges may simply be trimmed off so that they are equal to the end values of the range.

  If the inclusion of negative weighting parameters is not desired for a particular implementation of the process, an alternative method is to apply other significant methods to the input image data in specific cases (e.g. in filtering operations). In addition, a certain amount of data may be added to or subtracted from the pixel value to be calculated), or the average pixel data of 4 × 3 kernels calculated in the first step (classification step) of the process The value may be used as a parameter in the third step (calculation step) so that the degree of change can be adjusted to this average value.

  FIG. 14 shows that the bit depth of the input image data is 6 bits / color channel (that is, 18b per pixel), but this figure is only an example of the embodiment and is shown in FIG. Functional processes can be applied to images of any bit depth.

  In order to obtain optimal results, the image area consisting of bright features on a dark background and the opposite image area need to have different parameters applied in the third step (calculation step). From the above example, it will be clear that the main purpose of the second step (case detection step) is to distinguish these different regions. The type of LCD used has this information and is available for the main examples of these cases. In other words, if the image contains black text on a white background, the information that this image is such a case is displayed in the form of a “flag” embedded in the input image data or in the display mode by the user. As part of the options, it may be entered on the LCD. In this case, this information may be entered into the case detection process so that the first classification step and automatic selection of the correct parameters need not be performed.

  In the second embodiment of the present invention, a multi-view process is performed on the data set of the main image 7 and the data set of the side image 8 in the LCD control electronics 1 without making any changes to the input image. Is done. Any luminance resulting from this process is then compared by comparing the luminance value of the output multiview image with the luminance value of the input main image after being compressed by the first stage of the multiview process 12 in the additional image processing device 13. Undesired image artifacts are detected. Next, the additional image processing device 13 performs some correction on the multi-view image so that the artifact is not visually recognized, and further outputs the corrected multi-view image data set to the display device. The flow of this calculation process is shown in FIG.

  In an example of the method of the second embodiment, when the multi-view image and the original image 7 data are input to the additional image processing device 12, a 3 × 3 kernel is used in both images. , Each pixel and all eight pixels immediately adjacent to it are sampled. If the pixel value is associated with a single monochrome pixel or a single black and white pixel, the pixel values of eight adjacent pixels are processed. In the case of a composite color pixel group, since each composite color pixel group comprises red, green, and blue pixels, the process is applied sequentially to each color component pixel value of the image.

Then, the normalized weighted sum “A” of the pixel data values in the kernel of each image applies a weight to each pixel in the kernel according to the position of the image, and the process illustrated in FIG. It is calculated in the same process. This type of configuration using _three different weighting factors ω ₁ , ω ₂ and ω ₃ is shown in FIG. These normalized weighted “pixel apparent values” of the corresponding pixels in each image are compared, and if a discrepancy is found, the center pixel of the kernel in the multiview image is taken from each image's kernel. It is changed according to the required amount so as to make the obtained weighted sum equal. If A _mv is the sum of the normalized weighted kernels obtained from the multi-view image and A _ic is the sum of the normalized weighted kernels obtained from the compressed input main image, The new value P _{(x, y)} ′ of the center pixel of the kernel in the multiview image is given by:

Here, ω ₃ is a weighting factor for the central pixel of the kernel.

  When the new center pixel value is written to the multi-view image, the kernel moves one pixel and the operation is repeated with the new pixel value obtained from the previous step that provides adjacent pixel values in the new kernel. . In this way, the correction for the pixel value of the multi-view image is considered in the next calculation, and the image is not corrected excessively. The process is repeated until all pixels of the image have been scanned by the center pixel of the kernel.

In order to minimize the change in the output multi-view image caused by this process, while also minimizing image artifacts in the output image, it is desirable that no change is necessary in regions where the main image is a uniform image. For these reasons, it is also desirable that the sum of the weighted kernels does not vary from step to step due to the patterning that occurs in the multiview image by the multiview process in a uniform main image region. An example of a uniform region of luminance values of the compressed main image and the equivalent region of the output multi-view image, which would be caused by the process described in UK patent application No. 0804022.2, is shown in FIG. Show. From this figure it can be seen that even though the original image is uniform, the 3 × 3 kernel 14 in step (n) may have a different weighted sum than the kernel 15 in step (n + 1). Recognize. Therefore, it is desirable to select a weight value for the pixel within the kernel that produces a total value of equal weighted kernels for multiple regions of the multi-view image corresponding to a uniform region in the input main image. As an example of such weighting, ω ₁ = 1, ω ₂ = 2 and ω ₃ = 4 may be used.

  An image processing method that improves the perceived quality of an image displayed on a multi-view display device is thus provided. This improvement in quality is achieved by minimizing the degree of change that occurs in the input image, especially blurring of fine features, while removing color artifacts caused by multi-view processes in other image processing methods. Achieved. Various embodiments are described that provide various options that balance the overall quality of the perceived output multi-view image with the computational resources required to perform the process. It should be noted that the embodiments described herein include specific examples of processes that have been found to produce good results in the case of various resource specifications for specific display devices, but any specific use The optimal process for possible resource specifications varies from application to application. Accordingly, certain aspects of the embodiment (eg, a specific method for classifying the pixels used in the first step, the number and types of cases of the particular pixel pattern detected in the second step, or the first The specific calculation method performed to change the image data in the third step of the embodiment varies depending on the embodiment, and such variation indicates a detailed specification of a specific application example. It will be obvious to those skilled in the art and thus does not depart from the scope of the present invention.

  One embodiment of the present invention is particularly useful for removing color artifacts caused by the processes described in UK Patent Application No. 2428152A and UK Patent Application No. 0804022.2. An image processing filter is provided. GB-A-2445882 and corresponding applications in countries other than the UK describe blur filters for use in two-image display devices to process color artifacts. However, one embodiment of the present invention provides a first blur filter specifically designed to deal with color artifacts caused by the software privacy process, and as a result is introduced into the main image. Minimize the amount of blur.

  However, the present invention is not intended to be limited to software privacy, but is introduced by an optional image processing step in which the received pixel data is processed to create a multi-view effect for the viewer. It will be understood that it is applicable to the correction of artifacts that are made. For example, in GB-A-2445982, the processing steps performed using the technique according to one embodiment of the present invention involve interleaving a plurality of sets of received pixel data representing different images corresponding to each other. This is a step of presenting different images to a plurality of viewers at different viewing positions corresponding to different images, and the present invention is applicable in this context as described in GB-A-2445882.

  Two main embodiments have been described above. In the first main embodiment, the main image is blurred only by referring to the main image itself, and then a multi-view process is performed on the main image. In the second main embodiment, the multi-view processed image is compared with the input main image and corrected according to the comparison result.

  In the first embodiment, three examples are shown. Each example involves three steps: classification of input data, pattern matching, and application of an appropriate blurring operation. These three variants all differ in the balance between performance and ease of implementation / resource requirements.

  Some of the differences between the method embodying the present invention and the methods described in GB-A-2445882 and corresponding applications in countries other than the UK are listed below.

  Case detection filters described in GB-A-2445882 and corresponding applications in countries other than the UK have unidirectionality. That is, the process of blurring depends only on the pixel value of the pixel being processed and the pixel on the right side thereof. A filter according to an embodiment of the present invention uses the right side, left side, or both pixels to determine a new value.

  The filter according to an embodiment of the present invention can be handled differently for black lines, white lines, edges, and checkered patterns. The filters described in GB-A-2445882 and corresponding applications in countries other than the UK can only handle one specific case.

  GB-A-2445982 and corresponding applications in countries other than the UK use classification methods to classify whether the pixels in the kernel are images that are similar to or significantly different from the processed pixels. It is important that adjacent pixels have a higher or lower value than the pixel under consideration for which the operation is performed, and these pixel values are close. This is not sufficient as an embodiment of the present invention because it is not questioned whether it is different or different. In this regard, in one embodiment of the present invention, new pixel data is derived for a pixel in such a way that at least one pattern of pixel data and a corresponding inversion pattern can be distinguished, but GB-A The -2455982 method cannot make this distinction.

  Some of the differences between the second embodiment described here and GB-A-2445982 and the second method of corresponding applications in countries other than the UK are listed below.

  This second embodiment uses the actual output multi-view image to compare the apparent total values of the pixels rather than the assumed parallax barrier configuration. Pixels are not always set to 0, and some pixels are doubled in value, and the pattern of high / low pixels is inverted as the kernel moves.

  This second embodiment writes the correction to the image, moves the kernel, and uses the corrected pixel values obtained from the previous step to calculate a new weighted sum. In this way, the amendment is “carried”, unlike GB-A-2445882 and the corresponding application in countries other than the UK.

  It will be appreciated that the operation of one or more of the previously described members can be controlled by a program running on the display device or apparatus. Such an operation program may be stored in a computer-readable medium, or may be embodied in a signal such as a downloadable data signal provided from a website on the Internet. The appended claims should be construed as covering the operating program itself, or covering the operating program as a record, on a carrier wave, as a signal, or in any other form.

  For a more complete understanding of the nature and advantages of the present invention, reference should be made to the following detailed description taken together with the accompanying figures.

  Although the present invention has been described above, it is obvious that various modifications can be made. Such modifications do not depart from the spirit and scope of the present invention, and all such modifications that will be apparent to those skilled in the art are included within the scope of the following claims.

Claims (33)

A method of processing image data for multi-view display according to the display panel of the display device,
One image set of tables to picture element data is acquired a plurality of sets,
In a first process step, in order to create the effect of the multi-view to the viewer, to process pixel data of said plurality of sets,
In a second process step, each of the plurality of subsets in the pixel data comprises the same number of pixel groups, and for each subset each pixel group comprises at least one pixel among the plurality of pixel groups in said subset at least regarding one pixel group, depending on the pattern of the pixel data in the pixel group, to derive the new pixel data, conductor unloading the image blur filters having different in at least one of the pattern and its inverted pattern of The method performed by . In the first process step,
An on-axis gamma curve related to an on-axis observer observing from the normal direction of the display surface of the display panel, and an off-axis gamma related to an off-axis observer observing from a direction other than the normal direction. Based on the characteristic of the display panel that is different from a curve , one set of pixel data among the plurality of sets is modulated based on another set of pixel data, and thus for an observer on the axis , while variations in luminance due to the modulation is imperceptible are locally averaged, for the above off-axis observer, variation in luminance due to the modulation becomes perceptible without being locally averaged, wherein Item 2. The method according to Item 1. In the first process step, the observer on the axis by variation in luminance due to the modulation is imperceptible are averaged, a pair of perceived as having a single luminance pixel for the group, minutes increased luminance in one pixel group, make consistent and decrease in luminance of the other pixel group, the method of claim 2. The pixel groups of the pair, or Brightness of one pixel group above the highest luminance, or is Kano either bright degree of the other pixel group is the lowest brightness, according to claim 3 the method of. In the derivation of the new pixel data, depending on the pattern of the pixel data to select the filter, and apply the selected filter to at least a portion of the pixel group of the plurality of pixel groups in the subset and deriving the new pixel data, the method according to any one of claims 1-4. Comparing the pattern of the pixel data with a plurality of predetermined patterns;
Deriving a new pixel data in dependence on the result of the comparison step, the method according to any one of claims 1-5. In the derivation of the new pixel data, depending on the pattern of the pixel data to select the filter, and apply the selected filter to at least a portion of the pixel group of the plurality of pixel groups in the subset New pixel data is derived,
Each of said plurality of predetermined patterns, associated with a corresponding said filter,
Determine the matching pattern in the comparison step,
The filter associated with that pattern is selected for use in deriving the new pixel data, The method of claim 6. In the comparison step, a brightness measure for at least one pixel group immediately adjacent to each of the at least one pixel group for which new pixel data is being derived is determined, and a measure for distinguishing between brightness levels is determined. The method according to claim 6 or 7 . Each pixel group in the subset, in accordance with the pixel data is assigned to one of the predetermined multiple levels,
In the comparing step, the level of patterns assigned, is compared with the plurality of predetermined pattern, the method according to claim 6, 7 or 8,. A measure for allocating the pixel group to one of the predetermined plurality of levels, calculated on the basis of at least a portion of the pixel group of the pixel data of the plurality of pixel groups of the subset, to claim 9 The method described. A method of processing image data for multi-view display according to the display panel of the display device,
One image set of tables to picture element data is acquired a plurality of sets,
In a first process step, in order to create the effect of the multi-view to the viewer, to process pixel data of said plurality of sets,
In a second process step, each of the plurality of subsets in the pixel data comprises the same number of pixel groups, and for each subset each pixel group comprises at least one pixel, each pixel group in the above Symbol subset in accordance with the pixel data is assigned to one of Jo Tokoro of multiple levels,
Compare the assigned level pattern to multiple predetermined patterns,
Depending on the result of this comparison step, new pixel data is derived for at least one pixel group of the plurality of pixel groups of the subset ,
An image data processing method for calculating a scale for assigning a pixel group to one of the plurality of predetermined levels based on pixel data of at least a part of the plurality of pixel groups of the subset . The method according to claim 10 or 11 , wherein the scale is an average value. The pixel group, based on a comparison between its pixel data and the calculated measure is assigned to one of the predetermined multiple levels, the method according to claim 10, 11 or 12,. The pixel data, for use in the step of the derivation, based on the gamma power function to convert the luminance value of the apparent A method according to any one of claims 1 to 13. The method of claim 14 , wherein the pixel data is converted to the apparent luminance value using at least one conversion parameter determined depending on a pattern of the subset of pixel groups. 16. A method according to any one of the preceding claims, wherein the subset of pixels is continuous and extends substantially in one dimension. Pixel groups of the subset, is continuous, has a configuration extending in two dimensions, the method according to any one of claims 1-16. Each subset are separated by one pixel group from the other subsets, the method according to any one of claims 1 to 17. It said step of deriving new pixel data is to use information about the process that is performed in the first processing step, the method according to any one of claims 1 to 18. The new pixel data, a pattern of the pixel data representing the characteristics of the image, derived based on the pattern of ease Ige element data causes artifacts undesired color is generated by processing performed in the first process step 20. The method according to any one of claims 1 to 19 , wherein: In step above derivation, is being derive new pixel data said at least one respective pixel group is a pattern of pixel data representing the characteristics of the image, not desired processing performed in the first processing step color If it is determined whether to form a part of the pattern of pixel data that is likely to cause an artifact, and if it is determined to form part of the pattern, new pixel data is derived depending on the part of the pattern to process according to any one of claims 1 to 20. FIG. In steps described above derivation, the pattern of pixel data representing bright features on a relatively dark background, separately treated from the pattern of pixel data representing dark features on a relatively bright background, claim 1-21 The method according to one item. In step above derivation, is being derive new pixel data said at least one respective group of pixels is a dark or bright line having a width of one pixel group, adjacent to a dark or bright line having a width of one pixel group Of the left side of a dark or bright line having a width of two pixel groups, a checkered pattern consisting of light and dark having a pitch of one or two pixel groups, and an image feature consisting of diagonal lines whether forming part of at least one characteristic is determined from the pattern of the pixel data, the method according to any one of claims 1 to 22. 24. The method according to any one of claims 1 to 23 , wherein the second processing step is performed before the first processing step. The second process step is performed after the first processing step, the method according to any one of claims 1 to 23. 26. The method according to any one of claims 1 to 25 , wherein the new pixel data is derived only for one pixel group of the subset of pixel groups. Each pixel group comprises a composite color pixel group of color component pixels,
Is applied sequentially to each color component pixels, the method according to any one of claims 1 to 26. 28. The method of claim 27 , wherein the composite color pixel group comprises red, green, and blue color component pixels. The first processing step is performed to interleave received pixel data representing different images, and to present the different images to a plurality of viewers at different viewing positions. The method according to any one of claims 1 to 28 . 30. An apparatus configured to perform the method of any one of claims 1 to 29 . A display device comprising the apparatus according to claim 30 . 30. A program for controlling an apparatus to carry out the method according to any one of claims 1 to 29 . Transported by a transport medium,
The program according to claim 32 , wherein the carrier medium is a storage medium or a transmission medium.

Images