GB2428152A

GB2428152A - Processing image data for direction dependent viewing on a display device

Info

Publication number: GB2428152A
Application number: GB0513968A
Authority: GB
Inventors: Thomas Matthew Wynne-Powell; Paul Antony Gass; Diana Ulrich Kean
Original assignee: Sharp Corp
Current assignee: Sharp Corp
Priority date: 2005-07-08
Filing date: 2005-07-08
Publication date: 2007-01-17
Also published as: CN101339756A; CN1892323A; GB0513968D0; CN100489607C; CN101339756B

Abstract

An image is processed by modifying the respective data values of at least some of the image elements by adding or subtracting a splitting value to neighbouring image elements such that when the image is displayed on a display device, the values of neighbouring image elements are spatially averaged by the eye of a viewer. If the display device has a linear data value-to-luminance response, the image perceived by a viewer through spatial averaging is the same as the original image. For a display having a non-linear data value-to-luminance response, the perceived image is different to the original image. The splitting values may be chosen so that a mask image overlays the original image when viewed through a display having a non-linear data value to luminance response. The response characteristics of the display device may be dependent on the viewing angle such that the image may only be viewed when the viewing angle is normal to the display. The response characteristics may be switchable to allow the display to be switched between public and private viewing modes.

Description

Image Processing Method and Apparatus for Display Device The present

invention relates to a method of processing an image and an apparatus for processing an image. The present invention can be applied to a display device that is switchable between a private viewing mode and a public viewing mode.

Electronic display devices, such as monitors used with computers and screens built in to telephones and portable information devices, are usually designed to have a viewing angle as wide as possible, so that they can be read from as many viewing positions as possible.

However, there are some situations where it is useful to have a display that is visible from only a narrow range of angles. For example, where a person is reading a confidential or private document on the display of a mobile device in a crowded place, he would wish to minimise the risk of others around him also having sight of the document on the display.

It is therefore useful to have a display device that is switchable between two modes of operation. In a public' mode, the display device would have a wide viewing angle for general use. In a private' mode, the display device would have a narrow viewing angle, so that private information could be read in a public place.

For example, when certain secure web pages are accessed (e.g. bank site web pages), or when a certain PIN (personal identification number) is input to the keyboard (e.g. bank account PIN), the display could automatically go into the privacy mode. In the private mode, an indicator or icon could be shown on the screen to indicate that the private mode is active.

This concept can be applied to many other types of devices where a user may which to view confidential information, but cannot control who else may be watching. Examples are mobile phones, Personal Digital Assistants (PDAs), laptop PCs, desktop monitors, Automatic Teller Machines (ATM5) and Electronic Point of Sale (EPoS) equipment.

A number of devices are known which restrict the range of angles or positions from which a display can be viewed.

US 6,552,850 describes a method for the display of private information on a cash dispensing machine. Light emitted by the machine's display has a fixed polarisation state. The machine and its user are surrounded by a large screen of sheet polariser that absorbs light of that polarisation state but transmits the orthogonal state. Passers-by can see the user and the machine but cannot see information displayed on the screen.

One method for controlling the direction of light is the use of a "louvred" film. Such a film consists of alternating transparent and opaque layers in an arrangement similar to a Venetian blind. These layers may be perpendicular to the surface of the film or at some other angle. Like a Venetian blind, it allows light to pass through it when the light is travelling in a direction nearly parallel to the plane of the layers, but absorbs light travelling at large angles to the plane of the layers. Methods for the production of such films are described in USRE 27,617, US 4,766,023 and US 4,764,410.

Other methods exist for making films with similar properties to the Iouvred film. These are described, for example, in US 5,147,716 and US 5, 528,319.

The techniques described above may be used to restrict the range of angles from which a display can be viewed; in other words, they can be used to make a display private'.

However none of them gives a method by which the privacy function can easily be switched off to allow viewing from a wide range of angles.

Several methods are known for providing a display that can be switched between a public mode (with a wide viewing angle) and a private mode (with a narrow viewing angle).

US 2002/0158967 describes the use of a light control film mounted on a display so that the light control film can be moved over the front of the display to give a private mode, or mechanically retracted into a holder behind or beside the display to give a public mode. This method has the disadvantages that it requires moving parts that may fail or be damaged, and it adds significant bulk to the display.

One method for switching from public to private mode with no moving parts is to mount a light control film behind the display panel, and to place a diffuser that can be electronically switched on and off between the light control film and the panel. When the diffuser is inactive, the light control film restricts the range of viewing angles and the display is in the private mode. When the diffuser is switched on, it causes light travelling at a wide range of angles to pass through the panel and the display is in the public mode. It is also possible to mount the light control film in front of the panel and place the switchable diffuser in front of the light control film to achieve the same effect.

Switchable privacy devices of this type are described in US 5,831,698, US 6,211,930 and US 5,877,829. They share the disadvantage that the light control film absorbs a significant fraction of the light incident upon it, whether the display is in the public or the private mode. The display is therefore inefficient in its use of light. Since the diffuser spreads light through a wide range of angles in the public mode, these displays are also dimmer in the public mode than in the private mode, unless the backlight is made brighter to compensate.

Another disadvantage relates to the power consumption of such devices. In the public mode of operation, the diffuser is switched off. This would typically mean that a voltage is applied to a switchable polymerdispersed liquid crystal diffuser. More power is therefore consumed in the public mode than in the private mode. This is a disadvantage for displays that are used for most of the time in the public mode.

Another known method for providing a switchable public/private display is described in US 5,825,436. The light control device disclosed is similar in structure to the louvred film described above. However, each opaque element in the louvred film is replaced by a liquid crystal cell that can be electronically switched from an opaque state to a transparent state. The light control device is placed in front of or behind a display panel. When the cells are opaque, the display is in a private mode; when the cells are transparent, the display is in a public mode.

One disadvantage of this method relates to the difficulty and expense of manufacturing liquid crystal cells with an appropriate shape. Another disadvantage is that, in the private mode, a ray of light may enter at an angle such that it passes first through the transparent material and then through part of a liquid crystal cell. Such a ray will not be completely absorbed by the liquid crystal cell and this may reduce the privacy of the device.

Another method for producing a switchable public/private display device is disclosed in JP 3607272. The disclosed device uses an additional liquid crystal panel, which has patterned liquid crystal alignment. Different aligned segments of the panel modify the viewing characteristics of different areas of the display in different ways, with the result that the whole display panel is fully readable only from a central position.

GB-A-2405544 and JP 2005-078093 describe switchable privacy devices based on louvres, which operate only for one polarisation of light. The louvres are switched on and off either by rotating dyed liquid crystal molecules in the louvre itself or by rotating the plane of polarisation of the incident light using a separate element.

Co-pending British Patent Application No. 0401062.5 and PCT Patent Application No. PCT/1B2005/0501 70 disclose various backlight arrangements for use in a display device having the ability to switch the viewing angle between public and private modes, for example. Further known systems and techniques in this area are also described therein.

Co-pending British Patent Application No. 0408742.5 discloses a switchable privacy device that is constructed by adding one or more extra liquid crystal layers and polarisers to a display panel. The intrinsic viewing angle dependence of these extra elements can be changed by electrically switching the liquid crystal.

Co-pending British Patent Application No. 0427303.3 discloses a polarisation modifying layer (PML) that is placed behind the exit polariser of a liquid crystal display panel. Some parts of the PML are transparent. Other parts change the polarisation of light passing through them so that pixels viewed through these parts are inverted in colour (bright pixels becoming dark and dark pixels becoming bright). Data sent to pixels directly behind these parts are inverted so that when the display is viewed from a central position, the image appears normally. However, when the display is viewed from a different angle, different pixels are viewed through the retarder elements and the image is corrupted. Off-axis viewers see a confusing image, for example a random dot pattern. The PML may be made from liquid crystal and switched off to give a public mode.

Co-pending British Patent Application No. 0421227.0 discloses a device in which a guest host (dyed) LC layer with a patterned electrode is added to a standard TFT LC display. The dyed LC layer can be switched between an absorbing (private) and non- absorbing state (public). The dye molecules absorption is dependent upon the incident angle and polarisation of light. For a given polarisation and orientation the absorption of the dye increases with larger viewing angles resulting in low brightness at high angles (narrow mode).

Co-pending British Patent Application No. 0510422.9 discloses the combination of a privacy function and a 3D function provided by a single additional switch cell. The display has three operating states: a wide mode; a private mode; and a 3D mode. Both patterned and unpatterned LC alignment examples are described.

The concept of using a hologram to provide a privacy function which was first described in GB-A-2404991. However, due to unwanted diffraction of light from the display by the hologram, the colour of the image seen by viewers may be affected.

Furthermore, for applications using a touch screen mounted on the front of the display, the user's hand can block the illumination of the hologram and so reduce the effectiveness of the privacy mode.

Co-pending British Patent Application No. 0511536.5 discloses the use of an extra liquid crystal layer located between the existing polarisers of an LCD panel. In this location the extra switch cell can modify the greyscale curves for off-axis light. This provides a higher level of privacy for images than the techniques disclosed in, for example, copending British Patent Application No. 0408742.5.

JP 09230377 and US 5,844,640 describe a method of changing the viewing angle properties of a single layer LCD panel. This is achieved for a Vertically Aligned Nematic (VAN) LC mode. Electric fields in the plane of the display panel are used to control how the LC material tilts in a pixel area. The number and orientation of different tilt domains within a pixel can be controlled by the in-plane fields. A pixel with several tilt domains will have a wide viewing angle, while a pixel with one tilt domain will have a narrower viewing angle. The use of such a method to vary the viewing angle of a display is described. However, the viewing angle of a single tilt domain of the VAN mode described is generally not sufficiently narrow to provide a good privacy mode.

JP 3405972 describes a single LC panel which uses patterned LC alignment to provide a narrow viewing angle mode LCD. However, this narrow mode is fixed, and there is no wide viewing mode.

WO 03/015424 discloses a light switching apparatus that comprises a passive birefringent lens and a switchable polariser. By switching the polarisation, different directional distributions of output light are provided. However, when activated, the lenses do not discriminate in angle which light is imaged.

US 6,369,949 discloses an optically anisotropic micro-lens window. The imaging element described is not switchable, and consequently a device making use of this technology could not be switchable between public and private modes of operation.

Co-pending British Patent Application No. 0401190.4 discloses the use of multiple arrays of polarisation sensitive lenses in a polarisation optical conversion system.

US 5,109,219 describes a method for controlling the viewing angle of a LC display by converting a digital view angle parameter to an analogue bias voltage which is applied to the LC. However, this technique will only serve to modify the view angle characteristics of the display, and will not tend to hide the image at wide angles.

US 5,936,596 and JP 2002-263235 describe changing the voltage range applied to the pixels in an LC display to change the viewing angle. Lookup tables are used to change the display between narrow and wide viewangle modes. However, this method does not conceal displayed information as such when in the narrow mode, it only modifies the grey-scale mapping to distort the image.

The article "A Method for Concealment of Displayed Data", M. Dogruel, Displays, vol. 24, no. 3, October 2003, describes a method for concealing data shown on a display by time-sequentially rendering the image and its inverse at a rate faster than the human eye can perceive. The eye of a casual observer thus averages the images and therefore sees a uniform grey display screen. To see the private image, the user must wear shuttered glasses synchronised with the display, such that the inverse image is blocked. This method has a number of drawbacks: firstly, the user must wear shuttered glasses in order to observe the correct image; secondly, image privacy can also be compromised by rapidly moving a toothed object across the view of the display and thus obscuring some parts of the cancelling image; and thirdly a ghost image can be observed as it is very difficult to design the two images to cancel perfectly. This article also describes adding a third image to act as a confusing image, but this requires the display to run at three times the normal video rate.

Rocket Software, Inc. (http://www.rocketsoftware.com) have developed a software package that provides some level of privacy using the inherent properties of an LC display. The software modifies the image sent to the display by applying an extra patterning across the whole image that reduces the grey levels or contrast of the image.

Due to the non-linear response of the display, the level of reduction is such that, when viewed on-axis, the image is only slightly disturbed but, when viewed off- axis, the non- linear response of the display leads to an enhanced contrast patterning. However, this solution does inevitably affect the on-axis performance of the display in some degree, and the pattern visibility will disturb even the authorised user when using the display in the private mode. Further, in practice, the patterning is not sufficient to provide an adequate level of privacy off-axis.

In view of the above, it is desirable to provide a display device that is switchable between public and private modes and in which the quality of the image presented to the on-axis viewer in the private mode is not compromised while retaining an adequate level of privacy off-axis.

According to a first aspect of the present invention, there is provided a method of processing an image, represented by a plurality of image elements, for display on one or more display devices, comprising, in a first mode of operation, modifying the respective data values of at least some of the image elements such that when the modified image is displayed using a first display device having a first data value-to-luminance response to a first viewer in a first position relative to the first display device, the image perceived by the first viewer through spatial averaging is substantially the same as the original image, and such that when the modified image is displayed using a second display device having a second data value-to-luminance response, different to the first data value-to-luminance response, to a second viewer in a second position relative to the second display device, the image perceived by the second viewer through spatial averaging is different to the original image.

It may be that the first and second display devices are the same and the first and second positions are different.

It may also be that the first and second display devices are different. In such a case, the first and second positions may be the same or they may be different.

The first position may be within a narrow range of angles from an optimum viewing position for the first display device.

The second position may be outside a narrow range of angles from an optimum viewing position for the second display device.

The optimum viewing position may be substantially normal to a display panel of the display device.

The original image may be substantially hidden in the image perceived by the second viewer.

At least some of the data values may be modified in dependence upon a masking image.

Each data value may be modified in dependence upon the data value at a corresponding position of the masking image.

The image perceived by the second viewer may resemble at least to some extent the masking image.

The masking image may be such as to provide a high degree of visually confusing information to the second viewer.

The masking image may comprise random noise.

The method may comprise using different masking images in different time frames.

At least some of the data values may be modified in dependence upon a masking parameter.

The degree of modification may be determined at least in part by the masking parameter.

The method may comprise modifying the data values such that localised groups of displayed image elements are perceived by the first viewer through spatial averaging to have substantially the same overall luminance as those image elements would have done without such modification.

The degree of modification for each image element in a group may be determined in dependence upon the data value at a position in the masking image corresponding to the image element.

If any modification is to be performed for a group, the data value of at least one image element in the group may be increased while the data value of at least one other image element in the group may be decreased.

The amount of increase may be substantially the same as the decrease.

The amount of increase relative to the amount of decrease may be determined in dependence upon the first data value-to-luminance response.

The image elements designated for increase and decrease may be swapped in different time frames.

For each of the at least one image element an amount related to the corresponding respective masking image data value may be added to the image element data value, and for each of the at least other one image element an amount related to the corresponding respective masking image data value may be subtracted from the image element data value.

The amount may be equal to the corresponding masking image data value.

The amount may be determined in dependence upon the difference between the image data value and the maximum or minimum data value, whichever is closer.

The amount may be proportional to the difference multiplied by the corresponding masking image data value.

Each group may comprise two image elements.

The method may comprise averaging the data values of at least some of the image elements in a group.

The method may comprise averaging the data values of at least some of the masking image elements in a group.

Image elements having corresponding modifications performed on them may be arranged in lines of the image.

Image elements having corresponding modifications performed on them may be arranged in columns of the image.

Image elements having corresponding modifications performed on them may be arranged in a chequerboard pattern.

The method may comprise ensuring that a modified data value does not fall outside the normal range of data values allowed.

The method may comprise compressing the data value range of the image before modification.

The method may comprise compressing the data value range of the masking image before modification.

The second data value-to-luminance response may be a substantially nonlinear data value-to-luminance response.

The first data value-to-luminance response may be a substantially linear data value-to- luminance response.

Preferably, in a second mode of operation, the data values are not so modified.

The method may comprise, at least in the first mode of operation, modifying the data value-to-luminance response of the second display device to ensure that the second data value-to-luminance response is different to the first data value-to-luminance response.

The data value-to-luminance response of the second display device may be so modified by providing a layer for this purpose in the second display device.

The layer may be switchable.

The method may comprise providing a layer in the second display device to produce the second data value-to-luminance response.

It may be that only the data values of the image elements in a subportion of the image are so modified.

Each image element may relate to a plurality of colour components corresponding to a pixel of the display device.

Each image element may relate to a single colour component corresponding to a sub- pixel of the display device.

The first position may be at least a predetermined distance away from the first display device.

According to a second aspect of the present invention, there is provided an image processing apparatus for processing an image, represented by a plurality of image elements, for display on one or more display devices, the apparatus being adapted, in a first mode of operation, to modify the respective data values of at least some of the image elements such that when the modified image is displayed using a first display device having a first data value-to-luminance response to a first viewer in a first position relative to the first display device, the image perceived by the first viewer through spatial averaging is substantially the same as the original image, and such that when the modified image is displayed using a second display device having a second data value-to-luminance response, different to the first data value-to-luminance response, to a second viewer in a second position relative to the second display device, the image perceived by the second viewer through spatial averaging is different to the original image.

According to a third aspect of the present invention, there is provided a display device comprising an image processing apparatus according to the second aspect of the present invention.

According to a fourth aspect of the present invention, there is provided an operating program which, when loaded into an apparatus, causes the apparatus to become an apparatus according to the second aspect of the present invention or a display device according to the third aspect of the present invention.

According to a fifth aspect of the present invention, there is provided an operating program which, when run on an apparatus, causes the apparatus to carry out a method according to the first aspect of the present invention.

The operating program may be carried on a carrier medium. The carrier medium may be a transmission medium. The carrier medium may be a storage medium.

An embodiment of the present invention provides a privacy mode by confusing the displayed image at wider angles by means of a superimposed pattern. An embodiment of the present invention provides a low cost, switchable system for producing this patterning by image manipulation and using the inherent properties of a Liquid Crystal (LC) display.

Reference will now be made, by way of example, to the accompanying drawings, in which: Figure 1 is a block diagram illustrating an image display system embodying the present invention; Figure 2 is a flow diagram illustrating the operation of a first embodiment of the present invention; Figure 3 is a schematic diagram illustrating the modification of data values in an embodiment of the present invention; Figure 4(A) is a chart illustrating a linear data value-to-luminance response in an embodiment of the present invention; Figure 4(B) is a chart illustrating a non-linear data value-to-luminance response in an embodiment of the present invention; Figure 5(A) illustrates the scaling of the original and the masking images in the first embodiment of the present invention; Figure 5(B) illustrates the combining of the scaled original and the masking images in the first embodiment of the present invention; Figure 6 illustrates the scaling of a masking image in a second embodiment of the present invention; Figures 7(A) to (C) illustrate various data value modification arrangements suitable for use an in embodiment of the present invention; and Figure 8 illustrates a possible effect of a sharp change in masking image.

Figure 1 is a block diagram illustrating an image display system I embodying the present invention. The image display system I comprises an image processor 10, a display controller 20 and a display device 30. The display device 30 comprises a display panel 32 and a non-linear component 34.

An original image I is to be displayed on the display panel 32 of the display device 30.

The original image I is represented by a plurality of image elements, which may correspond to pixels of the display panel 32 or sub-pixels of the display panel 32. If the original image I is displayed directly on the display device 30, then the image will be viewable both by a viewer positioned on-axis at a position P1, and by a viewer positioned off-axis at a position P2. The first and second positions P1 and P2 are within first and second viewing regions RI and R2 respectively.

An embodiment of the present invention provides a first mode of operation in which the image seen by a first viewer located at position P1 relative to the display device 30 is substantially the same as the original image I, whereas the image seen by a second viewer located at position P2 relative to the display device 30 is different from the original image I. The masking image M is used for this purpose, as will now be explained in connection with a first embodiment of the present invention.

Operation of the first embodiment of the present invention will be described with reference to Figures 2 to 5. Figure 2 is a flow diagram providing an overview of the operations performed by the image processor 10 in the first embodiment. Before considering in detail the steps performed by the image processor 10, a more general description of the concept underlying an embodiment of the present invention will first be described with reference to Figures 3 and 4.

When a viewer is located more than a predetermined distance away from the display panel 32 of the display device 30, the viewer is unable to resolve each individual pixel or image element being displayed. A guideline for use in estimating an adequate viewer-display separation for this to be the case is provided in "Color and Light in Nature", D. Lynch & W Livingston, Cambridge University Press, 1995, which suggests that the eye's resolution is limited to I arc minute. Applying this to the present embodiment, two pixels or image element should preferably subtend an angle of less than 1 arc minute. It is to be understood that this is merely a guideline and that other resolutions may be applicable in different circumstances.

Instead of being able to resolve each individual pixel or image element, the human eye will spatially average a localised group of displayed image elements to perceive a single overall luminance. It should be noted thatthe localised group of displayed image elements may not correspond to arrangement of these image elements in the original image, for example because the image elements may be interlaced or otherwise re- arranged before display. An embodiment of the present invention takes advantage of this, together with the inherent data value-to-luminance response of the display device 30, as will now be explained.

Figure 3(A) shows one such localised group comprising two image elements having the same data value. Before display, the image processor 10 in an embodiment of the present invention splits the original data value equally into two new data values, such that the data value of one of the image elements is equal to the original data value minus the splitting amount, and the data value of the other image element is equal to the original data value plus the splitting amount.

When displayed on a display device having a linear data value-toluminance response, a viewer will perceive the two modified data values of the localised group to have the same overall luminance as those image elements would have done without such modification. This is because, due to the linear response of the display device, the single original data value maps to the same luminance as the average luminance of the two modified image elements. This is illustrated in Figures 3(B) and 4(A).

On the other hand, when the modified image elements in the localised group are displayed on a display device having a non-linear data value-toluminance response to a viewer in a predetermined position relative to the display device, the luminance of these image elements is no longer spatially averaged by the eye of the viewer to have the same overall luminance as those image elements would have done without such modification. Instead, the viewer perceives a luminance which differs from a straight average by an amount which depends on the non-linearity of the display. This is illustrated in Figure 4(B) and Figure 3(C).

The switching voltage applied to a pixel in a Liquid Crystal Display (LCD) device is usually compensated such that, when viewed on-axis, a change in the data sent to the pixel causes a proportional change in the observed luminance. However, light passing through the panel 32 at an angle other than the normal to the panel 32 will travel a different optical path length through the Liquid Crystal (LC) and will therefore be affected differently. This change in optical path length introduces a non- linear relationship between the pixel data and the observed luminance off- axis.

Because of this, the data value-to-luminance response to a viewer in a first position P1 substantially normal to the display panel 32 will be substantially linear, and the image perceived by such a viewer through spatial averaging will be substantially the same as the original image.

On the other hand, the data value-to-luminance response to a second viewer at position P2 off-axis relative to the display panel 32 will be non-linear, and the image perceived by such a viewer through spatial averaging will be different to the original image.

The required difference in the first and second data value-to-luminance responses is, as mentioned above, usual inherent in the properties of the display device itself. However, some displays are compensated to remove the non-linearity of the LC, such that the on- axis and off-axis responses are both substantially linear. For such devices, the off-axis non-linearity can be reintroduced by the introduction of the non-linear component 34 shown in Figure 1 to modify the viewing characteristics of the display. This non-linear component 34 can be a simple unpatterned LC layer. The non-linear component 34 can also be switchable, so that it need only be activated when required. It will be appreciated, however, that for display devices inherently having different luminance responses on- and off-axis the non-linear component 34 is not necessary, although the use of such a component may enhance the performance.

Alternatively, the LC panel itself may be operable to switch between two modes of operation, one in which the display has a constant luminance response and another in which the display has a non-linear luminance response for viewer P2. This would achieve the same effect, but without the need for the extra non-linear component 34.

This is described in more detail in our co-pending UK patent application No. [Agent's ref: P53551GB], which is incorporated herein by reference. The reader is referred to this co-pending application for further details of a suitable LC panel for use with an embodiment of the present invention, but in brief the co-pending application discloses one embodiment in which a display device comprises a liquid crystal display panel for displaying an image by spatial light modulation, and circuitry for switching liquid crystal in the panel between having a first configuration in a first (public) mode to cause an image displayed using the panel to be discernible from a wide range of viewing angles, and having a second configuration in a second (private) mode to cause an image displayed using the panel to be discernible substantially only from within a narrow range of viewing angles. The second configuration of liquid crystal is such as to cause an image-confusing pattern to be visible in the image discerned by a viewer outside the narrow range of angles. The liquid crystal in the first configuration comprises a plurality of lateral regions each having one of at least two different arrangements of liquid crystal. The first configuration regions are sized so as not to be resolvable by a viewer. The liquid crystal in the second configuration comprises a plurality of lateral regions each having one of at least two different arrangements of liquid crystal. The liquid crystal in the second configuration is arranged to provide the display panel with a substantially linear data value-to-luminance response for viewing angles within the narrow range and a substantially non-linear data valueto-luminance response for viewing angles outside the narrow range. The first configuration regions are adapted such that light travelling at an angle outside the narrow range of angles passes through at least two regions having different arrangements of liquid crystal so as to have a substantially linear data value-to-luminance response. The second configuration regions are adapted such that light travelling at an angle outside the narrow range of angles passes through a single region so as to have a substantially non- linear data value- to-luminance response.

As described above, in the first embodiment of the present invention, neighbouring data values arc modified to produce an image that, when viewed through a linear display device (on-axis) will be spatially averaged by the human eye back to the original image, but when viewed through a non-linear display device (off-axis) will introduce a component of the degree of splitting used. If the degree of splitting is varied across the image by an amount proportional to a second image, then when viewed off-axis the original and the second image will both be visible. The second image is the masking image M described above. If the masking M has a confusing patterning such as a chequerboard or company logo, then the original image will be substantially hidden to an off-axis viewer. This provides a private mode of operation, in which only the on- axis viewer has an undisturbed view of the original image.

A more detailed description of the first embodiment will now be provided with reference to Figures 2 and 5.

The image elements in the original image I would usually take any value in the range from 0 to 255. Because of this, the splitting as described above with reference to Figure 3 of a data value close to the minimum or maximum data value would potentially result in a modified data value falling outside the normal range of data values allowed. In order to prevent this, in step SI of the first embodiment, the original image I is scaled and centred so as to have a new, compressed data value range; this is illustrated in the top half of Figure 5(A). In step S2, the data value range of the masking image M is scaled such that the minimum data value is 0 and the maximum data value is equal to the minimum of the scaled original image; this is illustrated in the bottom half of Figure 5(A).

Figure 5(B) shows example cross-sections through the scaled original image I and the scaled masking image M. The data value of the masking image at any point determines the level of splitting to be used for the image element at a corresponding position of the scaled original image I. The level of splitting is proportional to the scaled data value of the masking image M, with neighbouring image elements of the scaled original image I being increased and decreased respectively by the splitting level determined from the scaled masking image M. The right-hand part of Figure 5(B) shows the result of the combination of the scaled original and masking images I and M, with the greatest degree of splitting occurring at the positions of the maxima of the masking image M. In the first embodiment, the splitting amounts are respectively added and subtracted from the scaled original image I by first inverting half of the scaled masking image M data values in step S3, and then adding the resulting pattern of data values to the scaled original image I in step S4. The resulting image is then viewed in S5 by the viewer.

The on-axis viewer will perceive an image through spatial averaging that is substantially the same as the scaled original image, while the offaxis viewer positioned at P2 will perceive an image that is different to the original image, resembling at least to some extent the masking image M. For a good privacy mode, the masking image would be such as to provide a high degree of visually confusing information to the off-axis viewer.

A second embodiment of the present invention will now be described with reference to Figure 6. The second embodiment is similar to the first embodiment, using the apparatus of Figure 1, and accordingly will only briefly be described here. The difference between the first and second embodiment results from the manner in which the original and masking images I and M are scaled and combined. In the first embodiment, the scaling of the original image I into a narrower range of data values results in a decrease in the contrast of the image, whereas the method that is performed in the second embodiment is such that the contrast of the original image is not sacrificed.

In the second embodiment, the degree of splitting is partially determined by how far the data value of a particular image element is from the closest edge of the allowed data value range. Image elements having data values in the middle of the range will be split the most, while image elements near 0 or 255 will be split the least. Image elements at either extreme of the range will not be split at all. This ensures that the modified data value does not fall outside the allowed range of data values.

Therefore, in the second embodiment, the difference between the original data value and the maximum or minimum data value, whichever is closer, is calculated, and this effectively sets the maximum level of splitting. This is illustrated in the top half of Figure 6.

The masking image M is then scaled according to the maximum level of splitting allowed for each image element to produce the scaled masking image M to be combined with the original image I. The actual combination can be performed in a similar way as described above with reference to the first embodiment.

Therefore, the overall process in the second embodiment is similar to the overall process in the first embodiment. Referring to Figure 2, in the second embodiment step Sl is omitted, while the scaling of the masking image M in step S2 is performed as described above with reference to Figure 6. Compared to the first embodiment, the second embodiment produces a strong effect on images having a pictorial content, but since saturated pixels undergo little or no splitting, the second embodiment will have no effect on pure black and white text.

Image elements having corresponding modifications performed on them, for example splitting upwards or splitting downwards, can be arranged to correspond to lines of the display device 30, as shown in Figure 7(A). Alternatively, corresponding image elements can be arranged in columns, or in a chequerboard pattern as shown in Figure 7(B). With the arrangements shown in Figures 7(A) and 7(B), each image element comprises three separate RGB colour components with each colour component being represented by a data value. Each data value of the image element is modified in the same way.

Alternatively, where each image element comprises three separate RGB colour components, each colour component may be treated independently, so that different colour component data values for the same image element may be split differently. One possible arrangement is shown in Figure 7(C) .

Other ways of combining the original and masking images I and M are possible. For example, as with the first embodiment, the contrast of the original image I could be reduced, but an asymmetric compression and splitting could be used rather than centring the compressed original image I. This would mean that the contrast of either the light or dark areas are retained preferentially, with the masking image M being compressed as in the first embodiment but also scaled according to the original image such that the modified data values do not fall outside the allowed range of data values. Other methods would be readily apparent to the skilled person, and any combination of the above methods can be used.

Although it is generally assumed above that the on-axis luminance response of the display device 30 is linear, such that equal splitting of data values will result in the viewer seeing substantially the original image through spatial averaging, if in fact the on-axis luminance response is non-linear, then the splitting can easily be compensated to account for this so that the on-axis viewer still sees substantially the original image through spatial averaging.

The above embodiments have been described with the emphasis on providing a privacy mode for a particular display device, such that a single image or sequence of images displayed on a single display device 30 is perceived differently by on-axis and off-axis viewers because of the different luminance responses seen. An embodiment of the present invention can also be applied to the situation where a single image is processed as described above for display on two separate display devices. The image is processed such that, when displayed on the first display device, the original image is seen undisturbed, but when the processed image is viewed on the second device, having a different luminance response to the first device, the original image is affected by the masking image used to process the original image. This would enable the original image to be viewed properly only by a display device having particular, known luminance response characteristics.

Any type of masking image M can be used, depending on the intended application. For example, the masking image could comprise a colour or black and white chequerboard pattern or random noise for a public/private mode application. The masking image could also comprise a logo or other image, text or any other form of information for display only on particular types of display device or to viewers located in particular positions. Animated masking images can also be used.

An embodiment of the present invention can be implemented in hardware. An embodiment of the present invention can also be implemented entirely in software providing the display panel has a luminance response which varies according to the viewing position, or providing that the different display panels used have different inherent luminance responses. A combination is also possible.

The image elements in a group, for example the two image elements shown in Figures 3(A) to (C), can be averaged before adding and subtracting the masking image, or it can be assumed that they take the same value. For a masking image, that is not aligned in any way with the image being displayed, it could be the case that the image elements in any localised group would be split by different amounts according to the corresponding data values of the masking image M. It would also be possible to average the masking image data values such that the same degree of splitting is applied to both or all image elements in a group. The number of image elements averaged together can be two or more. Although the example described above with reference to Figure 3 showed the image elements being considered in pairs, this is not essential, and any number of image elements can be considered in a localised group of image elements, it merely being necessary that the localised group of image elements are perceived by the on-axis viewer through spatial averaging to have substantially the same overall luminance as those image elements would have done without modification. The spatially averaged pixels could also be reversed time sequentially, so that in one frame a pixel may have the masking image added, and in a subsequent time frame the masking image would instead be subtracted (with an equivalent reversal for its neighbour).

Although the above description has referred to localised groups of displayed image elements, it is to be understood that this is a useful concept to adopt to ensure proper spatial averaging is provided, but does not necessarily imply the processing of image elements in separate, individual groups. The image elements can be processed without reference to any grouping, but with an appropriate global pattern of modification to ensure correct local spatial averaging (for example, as shown in Figures 7(A) to 7(C)).

The masking image should preferably have smoothly-varying data values, since any abrupt changes may have a minor effect on the perceived image on-axis in certain circumstances. Figure 8 illustrates why an abrupt change in the masking image could leave an artifact in the perceived image. This can be counteracted with suitable modification to a method embodying the present invention, for example a pre- processing stage to alter the masking image to avoid or reduce such an effect, or a change to the algorithm used to combine the masking image to prevent or reduce such an effect.

An embodiment of the present invention provides an electronicallyswitchable method for producing viewing angle restriction. A custom masking image can be used, which may be a moving image in order to provide enhanced confusion to unauthorised viewers. An embodiment of the present invention does not require shuttered glasses as do some known techniques, and can be applied to the whole or part of the display.

Modification to the display device itself is not usually required, but when it is required a complex patterned extra layer is not necessary, a single switchable cell being sufficient.

Privacy can be produced in both horizontal and vertical directions if there is non- linearity in both planes. The privacy level and area can be dependent on the content being displayed, and variable view-angle restrictions can be provided by changing the degree of splitting used. An embodiment of the present invention provides a low cost, switchable system for producing switching view-angle restriction.

An embodiment of the present invention can be applied to any type of display device, for example those on mobile phones, Personal Digital Assistants (PDA), Electronic Point of Sale (EPoS) kiosks, laptop computers or desktop monitors.

An operating program for implementing the invention can be stored on a computer- readable medium, although an operating program embodying the present invention need not be stored on a computer-readable medium and could, for example, be embodied in a signal such as a downloadable data signal provided from an Internet website. The appended claims are to be interpreted as covering an operating program by itself, or as a record on a carrier, or as a signal, or in any other form.

Claims

CLAIMS: I. A method of processing an image, represented by a plurality of

image elements, for display on one or more display devices, comprising, in a first mode of operation, modifying the respective data values of at least some of the image elements such that when the modified image is displayed using a first display device having a first data value-toluminance response to a first viewer in a first position relative to the first display device, the image perceived by the first viewer through spatial averaging is substantially the same as the original image, and such that when the modified image is displayed using a second display device having a second data value-to-luminance response, different to the first data value-to-luminance response, to a second viewer in a second position relative to the second display device, the image perceived by the second viewer through spatial averaging is different to the original image.
2. A method as claimed in claim 1, wherein the first and second display devices are the same and the first and second positions are different.
3. A method as claimed in claim 1, wherein the first and second display devices are different.
4. A method as claimed in claim 3, wherein the first and second positions are the same.
5. A method as claimed in claim 3, wherein the first and second positions are different.
6. A method as claimed in any preceding claim, wherein the first position is within a narrow range of angles from an optimum viewing position for the first display device.
7. A method as claimed in any preceding claim, wherein the second position is outside a narrow range of angles from an optimum viewing position for the second display device.
8. A method as claimed in claim 6 or 7, wherein the optimum viewing position is substantially normal to a display panel of the display device.
9. A method as claimed in any preceding claim, wherein the original image is substantially hidden in the image perceived by the second viewer.
10. A method as claimed in any preceding claim, wherein at least some of the data values are modified in dependence upon a masking image.
11. A method as claimed in claim 10, wherein each data value is modified in dependence upon the data value at a corresponding position of the masking image.
12. A method as claimed in claim 10 or 11, wherein the image perceived by the second viewer resembles at least to some extent the masking image.
13. A method as claimed in claim 10, 11 or 12, when dependent on claim 9, wherein the masking image is such as to provide a high degree of visually confusing information to the second viewer.
14. A method as claimed in claim 13, wherein the masking image comprises random noise.
15. A method as claimed in any one of claims 10 to 14, comprising using different masking images in different time frames.
16. A method as claimed in any preceding claim, wherein at least some of the data values are modified in dependence upon a masking parameter.
17. A method as claimed in claim 16, wherein the degree of modification is determined at least in part by the masking parameter.
18. A method as claimed in any preceding claim, comprising modifying the data values such that localised groups of displayed image elements are perceived by the first viewer through spatial averaging to have substantially the same overall luminance as those image elements would have done without such modification.
19. A method as claimed in claim 18, when dependent on claim 10, wherein the degree of modification for each image element in a group is determined in dependence upon the data value at a position in the masking image corresponding to the image element.
20. A method as claimed in claim 18 or 19, wherein, if any modification is to be performed for a group, the data value of at least one image element in the group is increased while the data value of at least one other image element in the group is decreased.
21. A method as claimed in claim 20, wherein the amount of increase is substantially the same as the decrease.
22. A method as claimed in claim 20, wherein the amount of increase relative to the amount of decrease is determined in dependence upon the first data value-to-luminance response.
23. A method as claimed in claim 20, 21 or 22, wherein the image elements designated for increase and decrease are swapped in different time frames.
24. A method as claimed in any one of claims 20 to 23, when dependent on claim 19, wherein for each of the at least one image element an amount related to the corresponding respective masking image data value is added to the image element data value, and for each of the at least other one image element an amount related to the corresponding respective masking image data value is subtracted from the image element data value.
25. A method as claimed in claim 24, wherein the amount is equal to the corresponding masking image data value.
26. A method as claimed in claim 24, wherein the amount is determined in dependence upon the difference between the image data value and the maximum or minimum data value, whichever is closer.
27. A method as claimed in claim 26, wherein the amount is proportional to the difference multiplied by the corresponding masking image data value.
28. A method as claimed in any one of claims 18 to 27, wherein each group comprises two image elements.
29. A method as claimed in any one of claims 18 to 28, comprising averaging the data values of at least some of the image elements in a group.
30. A method as claimed in any one of claims 18 to 29, comprising averaging the data values of at least some of the masking image elements in a group.
31. A method as claimed in any one of claims 18 to 30, wherein image elements having corresponding modifications performed on them are arranged in lines of the image.
32. A method as claimed in any one of claims 18 to 30, wherein image elements having corresponding modifications performed on them are arranged in columns of the image.
33. A method as claimed in any one of claims 18 to 30, wherein image elements having corresponding modifications performed on them are arranged in a chequerboard pattern.
34. A method as claimed in any preceding claim, comprising ensuring that a modified data value does not fall outside the normal range of data values allowed.
35. A method as claimed in claim 34, when dependent on claim 19, comprising compressing the data value range of the image before modification.
36. A method as claimed in claim 34 or 35, comprising compressing the data value range of the masking image before modification.
37. A method as claimed in any preceding claim, wherein the second data value-to- luminance response is a substantially non-linear data value-to-luminance response.
38. A method as claimed in any preceding claim, wherein the first data value-to- luminance response is a substantially linear data value-to-luminance response.
39. A method as claimed in any preceding claim, wherein, in a second mode of operation, the data values are not so modified.
40. A method as claimed in any preceding claim, comprising, at least in the first mode of operation, modifying the data value-to-luminance response of the second display device to ensure that the second data value-to-luminance response is different to the first data value-toluminance response.
41. A method as claimed in claim 40, wherein the data value-to-luminance response of the second display device is so modified by providing a layer for this purpose in the second display device.
42. A method as claimed in claim 41, wherein the layer is switchable.
43. A method as claimed in claim 40, comprising providing a layer in the second display device to produce the second data value-to-luminance response.
44. A method as claimed in any preceding claim, wherein only the data values of the image elements in a sub-portion of the image are so modified.
45. A method as claimed in any preceding claim, wherein each image element relates to a plurality of colour components corresponding to a pixel of the display device.
46. A method as claimed in any one of claims I to 44, wherein each image element relates to a single colour component corresponding to a sub-pixel of the display device.
47. A method as claimed in any preceding claim, wherein the first position is at least a predetermined distance away from the first display device.
48. An image processing apparatus for processing an image, represented by a plurality of image elements, for display on one or more display devices, the apparatus being adapted, in a first mode of operation, to modily the respective data values of at least some of the image elements such that when the modified image is displayed using a first display device having a first data value-to-luminance response to a first viewer in a first position relative to the first display device, the image perceived by the first viewer through spatial averaging is substantially the same as the original image, and such that when the modified image is displayed using a second display device having a second data value-to-luminance response, different to the first data value-to-luminance response, to a second viewer in a second position relative to the second display device, the image perceived by the second viewer through spatial averaging is different to the original image.
49. A display device comprising an image processing apparatus as claimed in claim 48.
50. An operating program which, when loaded into an apparatus, causes the apparatus to become apparatus as claimed in claim 48 or 49.
51. An operating program which, when run on an apparatus, causes the apparatus to carry out a method as claimed in any one of claims I to 47.
52. An operating program as claimed in claim 50 or 51, carried on a carrier medium.
53. An operating program as claimed in claim 52, wherein the carrier medium is a transmission medium.
54. An operating program as claimed in claim 52, wherein the carrier medium is a storage medium.