FR2797370A1 - METHOD AND SYSTEM FOR DISPLAYING REMOTE TRANSMITTED IMAGES - Google Patents

Abstract

A method of displaying an image in data processing equipment comprises the following steps: (a) connection to a server system, (b) transfer to the equipment of a digitized starting image (I) capable of receiving modification processing, (c) transfer to the equipment of a mask (M) associated with the image and capable of delimiting at least one predetermined zone (Z1, Z2, Z2 ') of said image in which a processing of modification can be carried out, (d) using an input interface of the equipment, choice by the user of a modification, in particular a new color, to be applied to the or each zone, (e) building a new image (I ') from the starting image, and (f) transferring the new image to a display memory.

Description

The present invention relates generally to image processing

scanned in an environment

  computer network including client-server.

  When, for example via an Internet connection, a client station user wishes to view an available image on a server station, this image is downloaded to the client station via the network. It may be for example an image in millions of colors, with a depth of 24 bits per pixel, and known processes (for example according to the JPEG standard) make it possible to transmit this image in compressed form, for the purpose in particular to minimize the transfer time and the space needed for storage, both

server side only client side.

  Today, when an image of the same composition, but having different colors and / or textures, must be sent to the client computer, then it is up to

  the whole picture that needs to be transmitted.

  To take a concrete example, however non-limiting, when the user of the client station is connected to an ecommerce site that offers for sale articles available in different colors, then the user who wants to be able to observe the product in its different colors today must download the image every time

  corresponding stored in the server site.

  This state of the art known presents

many disadvantages.

  First of all, the design of an electronic catalog requires shooting by product, even if it is the same product available in different colors and / or textures. As a corollary, the server site must have storage memories that can be considerable to be able to store all these images. Then, as mentioned, the bandwidth requirements of the server are important because of the need to transmit to each new! request from a client item, the entire

the requested image.

  Finally, with the dominant transmission channels today (analogue telephone lines), wait times at the customer station make the

  site consultation extremely tedious.

  The object of the present invention is to propose a method and a system for displaying an image on a client station which, in certain contexts, make it possible to

remedy these disadvantages.

  Thus, according to a first aspect, the present invention proposes a method for displaying an image on a display device of a data processing equipment, characterized in that it comprises the following steps: (a) remote connection to a server system, (b) transferring, from the server system to the data processing equipment, a digitized initial image adapted to receive modification processing, (c) transfer, from the server system to the equipment of data processing, of a mask associated with said image, said mask being able to delimit at least one predetermined zone of said image in which modification processing can be performed, (d) using an interface of input of the data processing equipment, choice by the user of a color and / or texture to be applied to the or each zone, (e) construction of a new image from the image of departure, and (f) transfer of the image i constructed to a display memory of said data processing equipment. Preferred, but not limiting, aspects of the method of the invention are the following: the selection step is carried out in a table having several entries each containing data

  pre-defined color or texture.

  said table is transferred from the server system to the data equipment during the step of

transfer (c).

  the selection step is performed from a list of choices displayed on a display device of the data processing equipment, the number of which is

  equal to the number of entries of said table.

  the selection step is performed from a color palette displayed on a display device of the data processing equipment, and the method further comprises a step of determining which of the entries of said table corresponds closer to one or more colors

chosen by the user.

  the determination step consists in combining differences between hue, saturation and gloss values of a color chosen by the user and an imposed color contained in each input of

said table.

  the determination step comprises the search for a minimum difference value (e) between the chosen color and the colors defined at the different entries of the table, the difference being determined each time according to the following formula: CI, C2) = MIN [ABS (hl-h2), ABS (hl-h2-1)] + ABS (sl-s2) + ABS (bl-b2) oh, s, b values of hue, saturation and gloss, standardized to all between 0 and 1 Cl (hl, sl, bl) First color to compare C2 (h2, s2, b2) Second color to compare

ABS absolute value of ...

MIN minimum of ...

  the method further comprises, before the determination step, a step of converting the values of red, green and blue of the color chosen by the user and the color defined at each entry of the table into said values; hue, saturation and

of brilliance.

  said mask comprises a plurality of zones which can be modified individually, and the steps of color selection and construction of the new image

  are executed for each of said zones.

  the mask has the format of an image of a depth of N bits, said zones being distributed in the depth of the image, and the size of the mask is

  equal to the size of the original image.

  each of the N bits associated with each pixel of the

  mask defines a mask layer.

  each possible value of the multiplets of N bits associated with each pixel of the mask defines a mask layer. one of the mask layers defines a zone of

  pixels in the image that should not be changed.

  the mask has, at least for its transfer, the format of a plurality of images without depth, which define the respective zones and at least some of which have a size smaller than that of the initial image and are defined by two dimensions and

  coordinates of a point of origin.

  According to a second aspect, the invention proposes a data processing system, comprising a central processing unit, memories and a display device, as well as a means of communication with a data server, said system comprising: means for receiving from said data server a start image and for storing it; means for receiving from said data server, in association with said start image, a mask defining at least one area of said image which can be modified in particular as regards its color, an input device to enable a user to define a modification to appear in said or in each zone, - processing means for placing in said zone or in each zone the chosen color or a

  color derived from the chosen color.

  Preferred, but not limiting, aspects of the system according to the invention are the following: the reception means are also able to receive from said data server, in association with said image and said mask, a table of color constraints, and said processing means are able to place in said zone or in each zone only a color contained in said color constraint table. the processing means are able to place in the said zone the color directly chosen in the table

by the user.

  the processing means comprise means for determining a difference between a color chosen by the user and colors contained in the color stress table, and are able to place the color exhibiting the slightest deviation from the color

  chosen color in the associated area.

  the deviation determination means are capable of calculating a difference between colors according to the following formula: e (CI, C2) MIN [ABS (h1-h2), ABS (h1-h2-1)] + ABS (sl- s2) + ABS (bl-b2) o h. s, b hue, saturation and gloss values, normalized to all between 0 and 1 Cl (hl, sl, bl) First color to compare C2 (h2, s2, b2) Second color to compare

ABS absolute value of.

MIN minimum of ...

  the processing means are capable of producing a color value derived from a chosen color by using a gloss information of the image of

departure in the area concerned.

  Other aspects, objects and advantages of the present invention will become more apparent upon reading the

  following detailed description of an embodiment

  preferred thereof, given by way of non-limiting example and with reference to the accompanying drawings, in which: Figure 1 illustrates a representation of colors in the RGB format, Figure 2 illustrates a representation of colors in the HSB format , and FIG. 3 is a diagram illustrating an image color change method according to the present

invention.

  As a preliminary, the present invention may be implemented for example in a client station consisting of a standard computer type "PC", which will not be described here, this client station being able to connect, for example via the telephone network and the Internet, to a data server station of any type. In the first place, a local application of colors and / or textures according to the present invention is intended to allow one or more users of client stations connected to a server station via a network to view an image from the server station following a an arbitrary or predefined number of color and / or texture variations applied to one or more zones of the original image, without having to reload each time an image from the server station. The definition of the individually modifiable zones is performed via a second image, called image mask, which is transmitted to the client station concerned at the same time as the original image. The application of the colors can be carried out by the user either freely, by choosing one or more colors in any color palette at his disposal on his workstation, or constrained, that is to say, restricted to a set given colors or color sets associated, which is also transmitted to the client station from the server station together with the original image and the image mask, and contains a data table of color constraints defining different colors or different sets of colors which are allowed to be respectively associated with the different areas of the image defined by the mask; means may in this latter case be provided in the customer station polur choose, in the pallet available on his station, the color closest to the color defined by the data of color constraints, according to a color search method such as it will be described in detail in

the following.

  We will first focus on two types of digital representation of color images, or color models, which are involved in a preferred process.

  according to the present invention, and then on the description of a

  color change method according to the present

invention.

  1) Color models The color models used are: a) Model R, G, B (Red, Green, Blue for Red, Green, Blue) This model separates colors into 3 main components, red, green and blue, Perfectly suited to CRT computer screens, consisting of 3 guns representing the same colors, and associated graphics cards. The most common model is to represent each pixel in 3 components of 8 bits each, ie 24 bits in total, allowing to

  represent 16 million colors.

  A color is then represented in a 3-dimensional space, as illustrated in FIG. 1, and the geometric shape associated with this model is a cube. This widespread model is, however, not very representative of the

human vision.

  b) Model H, S, B (Hue, Saturation, Brightness, for Hue, Saturation, Brilliance) This model breaks down a color into 3 basic characteristics. Tint ("hue") is the wavelength reflected or transmitted by an object, and is represented by an angle generally expressed in degrees and between 0 and 360, with by convention

  Red = 0 degrees, Green = 120 degrees and Blue = 240 degrees.

  The hue value is normalized such that the range of variation is between 0 and 1, the value 0 representing 0, and the value l representing 360. Saturation measures the strength of color. It is represented by a percentage between 0 and%. At 0% saturation, the color is desaturated, and appears as a gray level. At 100%, the color is totally saturated or pure. Finally, the brightness represents the intensity of light, with a percentage value also between 0% and 100%. At 0%, there is no light, and the

  color appears black. At 100% the brightness is maximum.

  The geometric shape associated with this model is illustrated in Figure 2 of the drawings. It appears as a double cone whose main axis is that of brilliance; the further one moves away from this axis, the more the color is saturated; finally the angle formed by the projection of the color vector considered in a horizontal plane, with respect to a reference axis contained

  in this plane, gives the hue value.

  This model is very representative of the human vision, and more appropriate for the application of algorithms

  related to color, such as segmentation for example.

  2) Color change procedure The procedure for changing the color of an original image in a client computer is done in five main steps: a) loading the program b) initialization c) calculating the luminances of each region d) loading the color constraints table

  e) change of colors by the user.

  For example, during an Internet session, the program is loaded according to the Java language specifications of the American company Sun Microsystems. This step is only necessary in the environment of a browser program for the Internet, or Intranet. In the case of other applications on other types of networks, this step may not exist, the program being called as any

  what other program on the user's machine.

  Referring now to Figure 3, the initialization step consists of loading an image I to be modified in the program. This image represents here an object OB intended to undergo the changes of colors. The input format of this image can be any; in particular, the image may have been compressed according to various techniques (JPEG or Wavelets for example), and

  it is then decompressed before the treatment.

  In this example, it is an image of mxn pixels downloaded via the Internet from a server site. The treatments that will be specifically described below are designed to apply to bitmap type images, typically for images in millions of colors whose color depth is 3x. 8-bit (8 bits per Red, Green and Blue channel), ie there is a correspondence between each point of the image and the on-screen display (as opposed to an image of type vector, o

  the display is performed by applying functions).

  If an image mask M accompanies the image I (which is done by an appropriate link), it is also loaded. The input format of the mask M can be any. This is a bitmap type mask, uncompressed or compressed without loss of information so that it can be faithfully restored. This image mask M, identical in size to the image M to be processed, contains one or more zones without overlapping with each other. The membership of a given point in a region is defined by the value of the pixel at the coordinates of that point. The convention used here is to associate a planar bit with each region, so that, if the mask M is given the same format as a bit map image with a depth of 8 bits, this mask can define eight layers of masking Ml, M2, etc. each having one or more zones, the or each zone of the same layer to be associated with the same color during the color change processing. In the present example, the masking layer M1 contains a zone Z1 corresponding to a part of the object OB contained in the image I, and the masking layer M2 contains two zones Z2 and Z2 'corresponding to two other parts of the 'object, these two zones Z2 and Z2' before each time to possess the same color of the fact

  that they are contained in the same masking layer.

  It should be noted here that this format of the mask can be exploited more finely so as to associate with each value of the byte associated with each pixel a masking layer, so as to achieve up to 256 masking layers. However, in practice, it is advantageous for the masking layer associated with the byte value 0 to be reserved to designate all the pixels that

  do not belong to any color change area.

  So all the points of this layer remain intact

during color changes.

  According to another variant, it is possible for the different zones to be defined by individual 1-bit images (black and white) whose size may be smaller than that of the initial image, especially if the zone in question is substantially larger. small as the whole picture. This part of the mask is then defined by a rectangle of given dimensions and from an origin also given. This makes it possible to reduce the volume of information to be transmitted on the network to the client station when the mask is loaded. Of course, after arriving at the client station, these individual images at 1 bit depth can be used by the client station to reconstruct an N layer mask of size equal to

  that of the original image as described above.

  The original image I in RGB format is first converted into the HSB format described above, by a conversion process known per se and a copy of this transformed image is stored in a buffer memory, which serves as a reference to any news

desired color application.

  The luminances of each zone of the mask are then calculated: thus, for each point of the initial image I, the average luminance is calculated for each region defined by the mask image M. If no mask image M is associated with image I, so we calculate the average luminance over the entire image. The average luminance of a region - or the entire image - is calculated by summing the brightness values of all the pixels in the area (or image), and then dividing that sum by the total number of pixels taken into account (arithmetic mean). This medium gloss information is reiterated for each zone Zi defined by the mask, and the different values obtained are then stored in a buffer table, and will be used thereafter for any new color application. The next step is to load the color constraint table, indicated at T in FIG. 3, which, as indicated, restricts the application of colors to a predetermined set of color sets. Each entry X of this table consists of n triplets, namely a triplet (rxy, g: .y, bxy) per mask layer My defined by the mask M, each triplet containing 3 values rxy, gxY and b: <y defining in the RGB model the color to apply

  in the zone or zones defined by this layer.

  When applying colors, this table a

  loaded in buffer is used as a reference.

  This table can be used in different ways. In a basic embodiment, the program offers the user, through a menu or any type of optional list, the possibility of choosing directly

  any of the entries in the table.

  For example, in the case illustrated in Figure 3, it can make a choice in the following list: white blue red yellow in combination with a color table which can be schematically as follows, in the same order: Zone ZI Zones Z2 and Z2 'white light gray light blue dark blue red pink yellow orange the colors shown here being in practice

  defined by well-defined associated triplets.

  Once the choice has been made (here choice of the "red" input), the method uses the values contained in the relevant entry of the color constraint table to modify the different zones and reconstruct an image I 'corresponding to the choice, this

  image is then displayed on the screen.

  According to another approach, and in particular in the case where the object OB must be able to be represented in a very large number of colors, the program can be designed to offer the user a choice initially

  not constrained by colors for different zones Zi.

  This approach will be described in more detail in the case where the image I is associated with a single masking layer, with therefore a single color choice

by the user.

  This choice is made for example using a color palette displayed on the screen, in which the user makes a selection using the mouse. Once the user has made this choice, a process is then implemented to calculate the difference between the chosen color and each color of the color constraint table, in order to find the nearest authorized color. the requested one. The closest color, ie the one with the minimum distance to the requested color, will be applied. The distance e (C1, C2) between the two colors C1 and

  C2 is preferably calculated in the HSB model, that is,

  that is, after converting the RGB triplets in the color constraint table to the HSB format, and after - if necessary - converting the requested color to

  the user in the same HSB format.

  The following formula is preferably used: e (C1, C2) = MIN [ABS (h1-h2), ABS (h1-h2-1)] + ABS (s1-s2) + ABS (b1-b2) oh, s , b hue, saturation and gloss values, normalized to all between 0 and 1 Cl (hl, sl, bl) First color to compare C2 (h2, s2, b2) Second color to compare ABS absolute value of MIN It is observed in the above expression that the distance to the hue value h, which is defined in terms of angle, is calculated modulo 360. Thus, for example, if h1 = 0.91 and h2 = 0.12, the distance d (h1, h2) is well calculated to be 0.21 and not 0.79. Another possibility of the present invention is to allow the user, from a given image I and M image mask, but without color constraints in the sense described above, to autonomously create a image I 'having different colors, chosen for the most part freely in the

different areas.

  The user's request can be formed, for example, by direct specification of n RGB triplets corresponding to the n zones to be colored, or preferably via a graphical interface comprising one or more color palettes in RGB format. The first step consists of converting the requested triplets into the HSB format, and determining from the HSB values the values to be applied to the different zones.

of the image.

  However, it is advantageous for each specified region Zr to first calculate a relative difference between the requested brightness value and the current value of the average brightness in the area under consideration, as follows: Pr = Lr / Br r number of the zone, varying for example from 0 to n-1 Br mean value of the current brightness for all the points of the zone Zr

  The brightness value requested by the user.

  Then, for each pixel I (i, j) of the initial image I, the hue and saturation values H (i, j) and S (i, j) are applied directly, as a function of each zone Zr. But only the aforementioned variation Pr is applied to the brightness value of each pixel concerned, this in order to preserve the original texture of the original image. Indeed, the eye is very sensitive to the differences Bright / dark (light and shadow) of the pixels of a zone, and it is important to preserve for the pixels of the zone concerned a distribution of the brightness similar to that of the original image. By modifying the brightness values of the different pixels not in the absolute, but by multiplying them by a coefficient equal to the ratio between the requested brightness and the initial average brightness, the distributions are thus preserved.

shade and light of origin.

  The relations applied are thus the following ones: H (i, j) = H (i, j) if M (i, j) e Zo (no change) H (i, j) = Hr if M (i, j) e Zr S (i, j) = S (i, j) if M (i, j) c Zo (no change) S (i, j) = Sr if M (i, j) e Zr B (i, j) = B (i, j) if M (i, j) e Zo (no change) B (i, j) = B (i, j) xPr if M (i, j) e Zr Once the image completely traveled, a conversion in the RGB format is performed, then the image thus processed is stored in order to then proceed to display. This image then comes to replace

the previous image.

  The present invention finds application in many fields where variable images must be displayed at a client station after transfer

from a server station.

  In particular, in the context of publishing data online via the World Wide Web, the present invention makes it possible to gain efficiency in the following areas: - shooting: for products existing in several colors, a single take of product view,

  in one of the possible colors is necessary.

  The product can be represented in other colors

  without requiring additional shots.

  - storage: The storage of images on the data server is optimized, since a single image, accompanied by its mask and "manufacturer" palettes, is necessary, whatever the

  color palette of the product under consideration.

  - Data transmission: The bandwidth requirements of the server are considerably reduced, since only one image and its mask are transmitted independently of the number of colors to be proposed. The net result is a possible increase in the number of simultaneous clients, and a shorter response time for a given number of clients.

  - Increased interactivity: The application of the colors being totally carried out locally on the customer's workstation, this one benefits from a greater interactivity

  great, and new features.

  Of course, the present invention is not limited to the embodiment described above and shown in the drawings, but the skilled person will know

  make many variations or modifications.

  In particular, while we have described above the case where an image I transmitted, and processed in accordance with the present invention, already has colors (original colors), which will be changed as indicated, we can predict not to transmit a raw image I, that is to say a black and white or grayscale image, without colors and / or without textures, and to apply the colors corresponding to a predetermined entry of the color table in the customer post beforehand

the display of the image.

Claims (20)

  A method of displaying an image on a display device of a data processing equipment, characterized in that it comprises the following steps: (a) remote connection to a server system, (b) transferring, from the server system to the data processing equipment, a scanned initial image (I) adapted to receive modification processing, (c) transfer, from the server system to the data processing equipment, d. a mask (M) associated with said image, said mask being able to define at least one predetermined zone (Z1, Z2, Z2 ') of said image in which modification processing can be carried out, (d) using an input interface to the data processing equipment, the user's choice of a modification, in particular of a new color, to be applied to the or each zone, (e) the construction of a new image (I ') from the starting image, and (f) transfer of the image thus constru ite to a display memory of said processing equipment of data.   2. Method according to claim 1, characterized in that the selection step is performed in a table (T) having several entries each containing predefined color or texture data (r: y, g:.: Y, b ;: y). 3. Method according to claim 2, characterized in that said table (T) is transferred from the server system to the data equipment during the step transfer (c).   4. Method according to one of claims 2 and   3, characterized in that the selection step is performed from a list of choices displayed on a display device of the data processing equipment, whose number is equal to the number of entries of said table (T).   5. Method according to claim 3, characterized in that the selection step is performed from a color palette displayed on a display device of the data processing equipment, and in that it comprises in addition a step of determining which of the entries of said table corresponds as closely as one or more colors chosen by the user.   6. Method according to claim 5, characterized in that the determining step consists of combining gaps (e (Cl, C2)) between hue, saturation and brightness values of a color chosen by the user. and an imposed color contained in each entry of said table.   7. Method according to claim 6, characterized in that the determining step comprises the search for a minimum difference value (e (Cl, C2)) between the chosen color and the colors defined at the different entries of the table. , the difference being in each case determined according to the following formula: e (Cl, C2) = MIN [ABS (h1-h2), ABS (h1-h2-1)] + ABS (s1-s2) + ABS (b1) -b2) * oh, s, b hue, saturation and gloss values, normalized to all between 0 and 1 Cl (hl, sl, bl) First color to compare C2 (h2, s2, b2) Second color to compare ABS absolute value of MIN minimum of   8. Method according to one of claims 6 and   7, characterized in that it further comprises, before the determining step, a step of converting the values of red, green and blue (r, g, b) of the color chosen by the user and of the color defined at each entry of the table in the said values   hue, saturation and gloss (h, s, b).   9. Method according to one of claims 1 to 9,   characterized in that said mask comprises a plurality of zones (Z1, Z2, Z2 ') adapted to be individually modified, and that the steps of color selection and construction of the new image are   executed for each of said zones.   The method according to claim 9, characterized in that the mask has the format of an image of a depth of N bits, said zones (Z1, Z2, Z2 ') being distributed in the depth of the image, and in that the size of the mask is equal to the size of the original image. 11. Method according to claim 10, characterized in that each of the N bits associated with each   Mask pixel defines a mask layer (Ml, M2).   12. Method according to claim 11, characterized in that each possible value of the multiplets of N bits associated with each pixel of the mask   defines a mask layer (Ml, M2).   13. Method according to one of claims 11 and   12, characterized in that one of the mask layers defines an area of pixels of the image not to be modified. 14. The method of claim 9, characterized in that the mask has, at least for its transfer, the format of a plurality of images without depth, which define the respective areas and at least some of which have a size smaller than that of the starting image and are defined by two   dimensions and coordinates of a point of origin.   A data processing system, comprising a central processing unit, memories and a display device, as well as means for communicating with a data server, said system comprising: means for receiving from said data server; data to a starting image (I) and to store it, - means for receiving from said data server, associated with said starting image, a mask (M) defining at least one zone (Z1, Z2, Z2 ') - of said image which can be modified in particular as to its color, - an input device to allow a user to define a modification to appear in said or in each zone, - processing means for placing in said area or in each area the chosen color or a   color (rxy, g: <y, bxy) derived from the chosen color.   16. System according to claim 15, characterized in that the receiving means are also able to receive from said data server, in association with said image and said mask, a table (T) of color constraints, and in that said means of treatment are able to place in said zone or in each zone only one color (r: <y, g: <. y, b: .y)   contained in said color constraint table.   17. System according to claim 16, characterized in that the processing means are adapted to place in said zone the color (rxy, gxy, b: .xy)   directly chosen in the table by the user.   18. System according to claim 16, characterized in that the processing means comprise means for determining a difference (e (C1, C2)) between a color chosen by the user and colors (rxy, gxy, bxy) contained in the color constraint table, and are able to place the color with the slightest deviation from the   chosen color in the associated area.   19. System according to claim 18, characterized in that the gap determining means are able to calculate a difference between colors according to the following formula: e (CI, C2) = MiN [ABS (h1-h2), ABS ( hl-h2-l)] + ABS (sl-s2) + ABS (bl-b2) oh, s, b hue, saturation and gloss values, normalized to all between 0 and 1 Cl (hl, sl , bl) First color to compare C2 (h2, s2, b2) Second color to compare ABS absolute value of MIN minimum of 20. System according to claim 15, characterized in that the processing means are capable of producing a color value derived from a chosen color by using a gloss information (b) of   the starting image in the area concerned.