EP1204951A1

EP1204951A1 - Method and system for displaying remotely transmitted images

Info

Publication number: EP1204951A1
Application number: EP00953260A
Authority: EP
Inventors: Jean-François Le Goff; Thierry Poullain; Eric Poullain
Original assignee: E-Sual
Current assignee: E-Sual
Priority date: 1999-07-19
Filing date: 2000-07-19
Publication date: 2002-05-15
Also published as: FR2797370A1; AU6577700A; FR2797370B1; WO2001006459A1

Abstract

The invention concerns a method for displaying an image in a data processing equipment comprising the following steps: (a) connecting to a server system; (b) transferring to the equipment an initial digitised image (I) capable of being modified by processing; (c) transferring to the equipment a mask (M) associated with the image and capable of delimiting at least a predetermined zone (Z1, Z2, Z2') of said image wherein a modification processing can be performed; (d) using an input interface of the equipment, the user selects a modification, in particular a new colour, to be applied to the or each zone; (e) constructing a new image (I') from the initial image; and (f) transferring the new image to a display memory.

Description

"METHOD AND SYSTEM FOR DISPLAYING REMOTELY TRANSMITTED IMAGES"

The present invention relates generally to the processing of digitized images in a computer network environment, in particular client-server.

When, for example via an Internet connection, a client computer user wishes to view an image available on a server computer, this image is downloaded to the client computer via the network. It can be, for example, an image in millions of colors, with a depth of 24 bits per pixel, and known processing (for example according to the JPEG standard) makes it possible to transmit this image in compressed form, with the aim in particular minimize the transfer time and the space required for its storage, both on the server and client sides. Today, when an image of the same composition, but having different colors and / or textures, must be routed to the client station, then it is again the entire image which must be transmitted.

To take a concrete example, however in no way limiting, when the user of the client computer is connected to an e-commerce site which offers for sale items available in different colors, then the user who wants to be able to observe the product in his different colors must now download the corresponding image stored on the server site each time.

This state of the art has many drawbacks.

First, the design of an electronic catalog requires taking a photo by product, even if it is the same product available in different colors and / or textures.

As a corollary, the server site must include storage memories which can be considerable to be able to store all these images.

Then, as indicated, the server needs in terms of bandwidth are important, due to the need to transmit, with each new request from a client station, the entire image requested.

Finally, with the dominant transmission channels _^ today (analog telephone lines), the waiting times at the client station make consultation of the site extremely tedious. The object of the present invention is to propose a method and a system for displaying an image on a client station which, in certain contexts, make it possible to remedy these drawbacks.

Thus, the present invention proposes, according to a first aspect, a method of displaying an image on a display device of a data processing equipment, characterized in that it comprises the following steps:

(a) remote connection to a server system, (b) transfer, from the server system to the data processing equipment, of a digital starting image able to receive modification processing,

(c) transfer, from the server system to the data processing equipment, of a mask associated with said image, said mask being capable of delimiting at least one predetermined area of said image in which modification processing can be carried out, (d) using an input interface of the data processing equipment, user choice of a color and / or a texture to be applied to the or each zone, (e) construction of a new image from the initial image, and

(f) transfer of the image thus constructed to a display memory of said data processing equipment. Preferred, but not limiting, aspects of the process of the invention are as follows:

- the choice step is carried out in a table having several entries, each containing predefined color or texture data. - Said table is transferred from the server system to the data equipment during the transfer step (c).

the choice step is carried out from a choice list displayed on a display device of the data processing equipment, the number of which is equal to the number of entries in said table.

the choice step is carried out from a palette of colors displayed on a display device of the data processing equipment, and the method further comprises a step consisting in determining which of the entries of said table corresponds as close as possible to one or more colors chosen by the user.

the determination step consists in combining differences between the hue, saturation and brightness values of a color chosen by the user, and of an imposed color contained in each entry of the said table.

the determination step comprises the search for a minimum difference value (e) between the chosen color and the colors defined at the various entries of the table, the difference being each time determined according to the following formula:

e (Cl, C2) = MIN [ABS (hl-h2), ABS (hl-h2-l)] + ABS (sl-s2) + ABS (bl-b2)

where h, s, b hue, saturation and brightness values, normalized to be all between 0 and 1 Cl (ni, si, bl) First color to compare C2 (h2, s2, b2) Second color to compare ABS absolute value of ...

MIN minimum of ...

the method further comprises, before the determination step, a step of converting the values of red, green and blue of the color chosen by the user and of the color defined at each entry of the table into said values hue, saturation and shine. - Said mask comprises a plurality of zones capable of being modified individually, and the steps of choosing colors and building the new image are carried out for each of said zones.

- The mask has the format of an image with a depth of N bits, said areas being distributed in the depth of the image, and the size of the mask is equal to the size of the original image. each of the N bits associated with each pixel of the mask defines a mask layer.

each possible value of the bytes of N bits associated with each pixel of the mask defines a mask layer.

- one of the mask layers defines an area of pixels of the image which must not be modified.

- The mask has, at least for its transfer, the format of a plurality of images without depth, which define the respective areas and at least some of which have a size less than _. that of the starting image and are defined by two dimensions and coordinates of an origin point.

According to a second aspect, the invention provides a data processing system, comprising a central processing unit, memories and a display device, as well as a means of communication with a data server, said system comprising:

means for receiving a starting image from said data server and for storing it,

- means for receiving from said data server, in a manner associated with said starting image, a mask defining at least one zone of said image which can be modified in particular as to its color, - an input device for enabling a user to define a modification to appear in said or in each zone,

- processing means for placing in said zone or in each zone the chosen color or a color derived from the chosen color.

Preferred, but not limiting, aspects of the system according to the invention are as follows: the reception means are also able to receive from said data server, in association with said image and said mask, a table of color constraints, and said processing means are able to place in said zone or in each zone only one color contained in said color constraint table.

- The processing means are able to place in said zone the color directly chosen in the table by the user.

the processing means comprise means for determining a difference between a color chosen by the user and colors contained in the color constraint table, and are able to place the color having the slightest difference with respect to the color chosen in the associated area.

the deviation determination means are capable of calculating a difference between colors according to the following formula:

e (Cl, C2) = MIN [ABS (hl-h2), ABS (hl-h2-l)] + ABS (sl-s2) + ABS (bl-b2)

where h, s, b hue, saturation and brightness values, normalized to be all between 0 and 1

Cl (hl, sl, bl) First color to compare C2 (h2, s2, b2) Second color to compare ABS absolute value of ...

MIN minimum of ... - the processing means are capable of developing a color value derived from a color chosen in using brightness information from the starting image in the area concerned.

Other aspects, aims and advantages of the present invention will appear better on reading the following detailed description of a preferred embodiment thereof, given by way of non-limiting example and made with reference to the accompanying drawings, in which: FIG. 1 illustrates a representation of colors in the RGB format, FIG. 2 illustrates a representation of colors in the HSB format, and FIG. 3 is a diagram illustrating a process for changing image colors according to the present invention.

As a preliminary point, the present invention can be implemented for example in a client station constituted by a standard computer of the "PC" type, which will not be described here, this client station being able to connect, for example via the telephone network and the Internet, at any data server station.

Firstly, the purpose of a local application of colors and / or textures in accordance with the present invention is to allow one or more users of client stations connected to a server station via a network to view an image coming from the server station according to a arbitrary or predefined number of variations of colors and / or textures applied to one or more areas of the original image, without having to reload an image each time from the server station. The definition of the individually modifiable zones is carried out by means of a second image, called image mask, which is transmitted to the client station concerned at the same time as the original image.

The application of colors can be carried out by the user either freely, by choosing one or more colors from any palette of colors available to him on his workstation, or in a constrained manner, that is to say restricted to a set given colors or associated color sets, which is also transmitted to the client station from the server station at the same time as the original image and the image mask, and which contains a table of data of color constraints defining different colors or different sets of colors which are authorized to be associated respectively with the different areas of the image defined by the mask; in the latter case, means may be provided in the client station for choosing, from the palette available on his station, the color closest to the color defined by the color constraint data, according to a color search method such as 'will be described in detail below. We will first focus on two types of digital representation of color images, or color models, which are used in a preferred process according to the present invention, then on the description of a change process. of colors according to the present invention.

1) Color models The color models used are:

a) Model R, G, B (Red, Green, Blue for Red, Green, Blue)

This model separates the colors into 3 main components, red, green and blue, Perfectly suited for computer screens with cathode ray tube, composed of 3 guns representing the same colors, and associated graphics cards. The most common model is to represent each pixel in 3 components of 8 bits each, or 24 bits in total, allowing to represent 16 million colors.

A color is then represented in a 3-dimensional space, as illustrated in Figure 1, and the geometric shape associated with this model is a cube. This widely used model is however not very representative of human vision.

b) Model H, S, B (Hue, Saturation, Brightness, for Hue, Saturation, Brightness)

This model breaks down a color into 3 fundamental characteristics. The hue is the wavelength reflected or transmitted by an object, and is represented by an angle generally expressed in degrees and between 0 ° and 360 °, with convention

Red = 0 degrees, Green = 120 degrees and Blue = 240 degrees.

The tint value is normalized, so that the range of variations is between 0 and 1, the value 0 representing 0 °, and the value 1 representing 360 °. Saturation measures the strength of color. It is represented by a percentage between 0 and

100%. With 0% saturation, the color is desaturated, and appears as a gray level. At 100%, the color is completely saturated or pure.

Finally, the brightness represents the intensity of light, by a percentage value also between 0% and 100%. At 0%, there is no light, and the color appears black. At 100% the brightness is maximum. The geometric shape associated with this model is illustrated in Figure 2 of the drawings. It: appears as a double cone whose main axis is that of shine; the further we move laterally from this axis, the more the color is saturated; finally the angle formed by the projection of the color vector considered in a horizontal plane, with respect to a reference axis contained in this plane, gives the hue value.

This model is very representative of human vision, and more suitable for the application of algorithms related to color, such as segmentation for example.

2) Procedure for changing colors

The procedure for changing the colors of an original image in a client computer is carried out in five main steps:

a) loading of the program b) initialization c) calculation of the luminances of each region d) loading of the table of color constraints e) modification of the colors by the user. The program is loaded, for example, during an Internet session, according to the Java language specifications of the American company Sun Microsystems.

This step is only necessary in the environment of a navigation program for the Internet, or Intranet. In the case of other applications on other types of networks, this step may not exist, the program then being called like any other program on the user's machine.

Referring now to FIG. 3, the initialization step consists in loading an image I to be modified in the program. This image here represents an OB object intended to undergo the color changes. The input format for this image can be any; in particular, the image may have been compressed using various techniques (JPEG or Wavelets for example), and it is then decompressed before processing. In this example, it is an image of mxn pixels downloaded via the Internet from a server site.

The processes that will be described specifically below are designed to apply to bitmap type images, typically for images in millions of colors whose color depth is 3 x 8 bits (8 bits per Red, Green and Blue channel), i.e. there is a correspondence between each point of the image and the display on the screen (as opposed to an image of the type vector, where the display is made by applying functions). If an image mask M accompanies image I (which is done by an appropriate link), it is also loaded.

The input format of the mask M can be any. In this case, it is a bit card type mask, uncompressed or compressed without loss of information so that it can be faithfully restored. This image mask M, of size identical to the image M to be processed, contains one or more zones without overlapping with each other. The membership of a given point in a region is defined by the value of the pixel at the coordinates of this point. The convention used here is to associate a plane bit with each region, so that, if the mask M is given the same format as a bit map image with a depth of 8 bits, this mask can define eight masking layers Ml, M2, etc. each having one or more zones, the or each zone of the same layer having to be associated with the same color when processing color changes. In the present example, the masking layer M1 contains a zone Zl corresponding to a part of the object OB contained in the image I, and the masking layer M2 contains two zones Z2 and Z2 'corresponding to two other parts of l 'object, these two zones Z2 and Z2' must each time have the same color because they are contained in the same masking layer.

It will be noted here that this format of the mask can be used more finely so as to associate with each value of the byte associated with each pixel a masking layer, so as to thus produce up to 256 masking layers. However, in practice, it is advantageous for the masking layer associated with the value byte 0 is reserved to designate all the pixels that do not belong to any color change zone. Thus all the points of this layer remain intact during color changes. According to another variant, it can be provided that the different zones are defined by individual 1-bit images (black and white) the size of which can be smaller than that of the initial image, in particular if the zone in question is significantly larger. smaller than the whole picture. This part of the mask is then defined by a rectangle of given dimensions and starting from an origin also given. This makes it possible to reduce the volume of information to be transmitted over the network to the client station when the mask is loaded. Of course, after arriving at the client station, these different individual images at 1 bit depth can be used by the client station to reconstruct a mask with N layers of size equal to that of the starting image as described above. The original image I in RGB format is first converted into the HSB format described above, by a conversion process known per se and a copy of this transformed image is kept in a buffer memory, which serves as a reference to any new application of desired color.

The luminances of each zone of the mask are then calculated: thus, for each point of the starting image I, the average luminance is calculated for each region defined by the mask image M. If no mask image M is associated with image I, then the average luminance over the entire image is calculated. The average luminance of a region - or of the whole image - is calculated by adding the brightness values of all the pixels in the area (or of the image), then dividing this sum by the total number of pixels taken into account

(arithmetic average) . This information of average brightness is reiterated for each zone Zi defined by the mask, and the different values obtained are then stored in a table in buffer memory, and will be used thereafter for any new application of color. The following step consists in loading the table of color constraints, indicated at T in FIG. 3, which, as indicated, makes it possible to restrict the application of colors to a predetermined set of color sets. Each entry X in this table consists of n triples, namely a triplet (r _X γ, g _xγ , b _xγ ) per masking layer M _γ defined by the mask M, each triplet containing 3 values r _xγ , g _xγ and b _xγ defining in the RGB model the color to be applied in the zone or zones defined by this layer. When applying colors, this table once loaded in the buffer serves as a reference.

This table can be used in different ways. In a basic embodiment, the program offers the user, by a menu or any type of list to choose from, the possibility of directly choosing any one from the entries of the table.

For example, in the case illustrated in Figure 3, he can choose from the following list: white blue red yardstick in association with a color table which can be shown schematically as follows, in the same order:

Zone Zl Zones Z2 and Z2 'white light gray light blue dark blue red pink yellow orange

the colors indicated here being in practice defined by well-defined associated triplets.

Once the choice has been made (here choice of entry

"Red"), the method uses the values contained in the considered input of the color constraint table to modify the different zones and reconstruct an image I 'corresponding to the choice, this image then being displayed on the screen.

According to another approach, and in particular in the case where the OB object must be able to be represented in a very large number of colors, the program can be designed to offer the user an initially unconstrained choice of colors for the different zones. Zi. This approach will be described in more detail in the case where image I is associated with a single masking layer, with consequently a choice of color unique by the user.

This choice is made for example using a palette of colors displayed on the screen, in which the user makes a selection using the mouse. Once the user has made this choice, a process is then implemented to calculate the difference between the chosen color and each color in the color constraint table, in order to find the authorized color which is closest to the one requested.

The nearest color, that is to say the one having the minimum difference with the requested color, will then be applied.

The distance e (Cl, C2) between the two colors Cl and C2 is preferably calculated in the HSB model, that is to say after having converted the RGB triplets from the color constraint table into the HSB format, and after having - if necessary - converted the color requested by the user into this same HSB format. The following formula is preferably used:

e (Cl, C2) = MIN [ABS (hl-h2), ABS (hl-h2-l)] + ABS (sl-s2) + ABS (bl-b2)

Cl (hl, sl, bl) First color to compare

C2 (h2, s2, b2) Second color to compare

ABS minimum MIN absolute value of

We observe in the expression above that the distance on the hue value h, which is defined in terms of angle, is calculated odulo 360 °. Thus, for example, if hl = 0.91 and h2 = 0.12, the distance d (hl, h2) is well calculated as being 0.21 and not 0.79. Another possibility of the present invention is to allow the user, from a given image I and a given image mask M, but without color constraints in the sense described above, to independently create a image I 'having different colors, essentially chosen freely in the different zones.

The user's request can be formed for example by direct specification of n RGB triples corresponding to the n areas to be colored, or preferably by means of a graphic interface comprising one or more color palettes in RGB format. The first step is to convert the requested triples into HSB format, and to determine from HSB values the values to be applied to the different areas of the image.

However, it is advantageous that, for each specified region Zr, a relative difference between the required brightness value and the current value of the average brightness in the area considered is first calculated, according to the following relationship:

Pr = Lr / Br

r zone number, varying for example from 0 to n-1 Br mean value of the current brightness for all the points in the area Zr Lr brightness value requested by the user.

Then, for each pixel I (i, j) of the initial image I, the hue and saturation values H (i, j) and S (i, j) are applied directly, as a function of each zone Zr. But only the aforementioned variation Pr is applied to the brightness value of each pixel concerned, this in order to preserve the original texture of the starting image.

Indeed, the eye is very sensitive to the Bright / dark differences (light and shadow) of the pixels of an area, and it is important to keep for the pixels of the area concerned a distribution of brightness similar to that of the original image. By modifying the brightness values of the different pixels not in the absolute, but by multiplying them by a coefficient equal to the ratio between the requested brightness and the starting average brightness, we therefore preserve the light and shadow distributions d 'origin.

The relationships applied are as follows:

H (i, j) = H (i, j) if M (i, j) € Zo (no change) H (i, j) = Hr if M (i, j) e Zr

S (i, j) = S (i, j) if M (i, j) e Zo (no change) S (i, j) = Sr if M (i, j) e Zr

B (i, j) = B (i, j) if M (i, j) e Zo (no change) B (i, j) = B (i, j) xPr if M (i, j) e Zr

Once the image has been fully browsed, a conversion to the RGB format is carried out, then the image thus processed is memorized so that it can then proceed when it is displayed. This image then replaces the previous image.

The present invention finds application in many fields where variable images must be displayed at a client station after transfer from a server station.

In particular, in the context of publishing data online via the World Wide Web, the present invention allows an efficiency gain in following areas:

- shooting: for products existing in several colors, a single shooting of the product, in one of the possible colors is necessary. The product can be represented in other colors without requiring additional shots.

- Storage: The storage of images on the data server is optimized, since a single image, accompanied by its mask and "manufacturer" palettes, is necessary, regardless of the color palette of the product considered.

- data transmission: The bandwidth requirements of the server are considerably reduced, since a single image and its mask are transmitted regardless of the number of colors to be proposed. The net result is a possible increase in the number of simultaneous customers, and a shorter response time for a given number of customers.

- increased interactivity: The application of colors being completely carried out locally on the client's workstation, the latter benefits from greater interactivity and new functionalities. Of course, the present invention is in no way limited to the embodiment described above and shown in the drawings, but a person skilled in the art will know how to make numerous variants or modifications to it. In particular, while the case has been described above where an image I transmitted, and processed in accordance with the present invention, already has colors (original colors), which will be changed as indicated, provision may be made for not transmit a raw image I, that is to say a black and white or grayscale image, without colors and / or without textures, and to apply the colors corresponding to a predetermined entry in the color table in the client station before the image is displayed.

Claims

1. A method of displaying an image on a display device of a data processing equipment, characterized in that it comprises the following steps:

(a) remote connection to a server system,

(b) transfer, from the server system to the data processing equipment, of a digital starting image (I) capable of receiving modification processing,

(c) transfer, from the server system to the data processing equipment, of a mask (M) associated with said image, said mask being able to delimit at least one predetermined area (Zl, Z2, Z2 ') of said image in which modification processing can be carried out,

(d) using an input interface of the data processing equipment, choice by the user of a modification, in particular of a new color, to be applied to the or each zone,

(e) construction of a new image (I ′) from the initial image, and

(f) transfer of the image thus constructed to a display memory of said data processing equipment.

2. Method according to claim 1, characterized in that the choice step is carried out in a table (T) having several entries each containing predefined color or texture data (r _< ,, g _{ , b ,,) -

3. Method according to claim 2, characterized in that said table (T) is transferred from the server system to the data equipment during the transfer step (c).

4. Method according to one of claims 2 and 3, characterized in that the choice step is performed from a choice list displayed on a display device of the data processing equipment, the number is equal to the number of entries in said table (T).

5. Method according to claim 3, characterized in that the selection step is carried out from a color palette displayed on a display device of the data processing equipment, and in that it comprises furthermore a step consisting in determining which of the entries of said table corresponds as closely as possible to one or more colors chosen by the user.

6. Method according to claim 5, characterized in that the determining step consists in combining differences (e (Cl, C2)) between values of hue, saturation and brightness of a color chosen by the user and of an imposed color contained in each entry of said table.

7. Method according to claim 6, characterized in that the determining step comprises the search for a minimum difference value (e (Cl, C2)) between the chosen color and the colors defined at the different inputs. from the table, the difference being each time determined according to the following formula:

e (Cl, C2) = MTN [ABS (hl-h2), ABS (hl-h2-l)] + ABS (sl-s2) + ABS (bl-b2)

where h, s, b hue, saturation and brightness values, normalized to be all between 0 and 1 Cl (ni, si, bl) First color to compare C2 (h2, s2, b2) Second color to compare ABS absolute value of

MIN minimum of

8. Method according to one of claims 6 and 7, characterized in that it further comprises, before the determining step, a step of converting the values of red, green and blue (r, g, b ) of the color chosen by the user and of the color defined at each entry of the table in said values of hue, saturation and brightness (h, s, b).

9. Method according to one of claims 1 to 9, characterized in that said mask comprises a plurality of zones (Zl, Z2, Z2 ') capable of being modified individually, and in that the steps of choosing colors and construction of the new image are executed for each of said areas.

10. Method according to claim 9, characterized in that the mask has the format of an image with a depth of N bits, said areas (Zl, Z2, Z2 ') being distributed in the depth of the image, and in that the size of the mask is equal to the size of the original image.

11. Method according to claim 10, characterized in that each of the N bits associated with each pixel of the mask defines a mask layer (Ml, M2).

12. Method according to claim 11, characterized in that each possible value of the bytes of N bits associated with each pixel _^ of the mask defines a mask layer (Ml, M2).

13. Method according to one of claims 11 and 12, characterized in that one of the mask layers defines an area of pixels of the image which must not be modified.

14. Method according to claim 9, characterized in that the mask has, at least for its transfer, the format of a plurality of images without depth, which define the respective areas and at least some of which have a size smaller than that of the starting image and are defined by two dimensions and coordinates of an origin point.

15. Data processing system, comprising a central processing unit, memories and a display device, as well as a means of communication with a data server, said system comprising: means for receiving a starting image (I) from said data server and for storing it,

means for receiving from said data server, in a manner associated with said starting image, a mask (M) defining at least one zone (Zl, Z2, Z2 ') of said image which can be modified in particular as to its color, an input device to allow a user to define a modification to appear in said or in each zone,

- processing means for placing in said zone or in each zone the chosen color or a color (r _xγ , g _xγ , b _XY ) derived from the chosen color.

16. System according to claim 15, characterized in that the reception means are also capable of receiving from said data server, in association with said image and said mask, a table (T) of color constraints, and in that said means are able to place in said zone or in each zone only one color (r _xγ , g _xγ , b _xγ ) contained in said table of color constraints.

17. System according to claim 16, characterized in that the processing means are able to place in said zone the color (r _xγ , g _γ , b _xγ ) directly chosen in the table by the user.

18. System according to claim 16, characterized in that the processing means comprise means for determining a difference (e (Cl, C2)) between a color chosen by the user and colors (r _xγ , g _xγ , b _xγ ) contained in the table of color constraints, and are able to place the color having the least deviation compared to the color chosen in the associated zone.

19. The system as claimed in claim 18, characterized in that the means for determining the difference are capable of calculating a difference between colors according to the following formula:

e (Cl, C2) = MTN [ABS (hl-h2), ABS (hl-h2-l)] + ABS (sl-s2) + ABS (bl-b2)

Cl (ni, si, bl) First color to compare C2 (h2, s2, b2) Second color to compare ABS absolute value of

MIN minimum of

20. The system as claimed in claim 15, characterized in that the processing means are capable of developing a color value derived from a chosen color by using brightness information (b) of the starting image in the area concerned.