SE511669C2 - Ways to optimize the choice of colors when presenting an image - Google Patents

Abstract

The present application relates to a method for optimising the choice of colours, included in the colour palette, when presenting a picture with fewer colours than those available in an original form of the picture. All colours are present as a combination of primary colours, for instance red, green and blue (RGB), and one generates a multidimensional histogram with said primary colours along the axes, in which the colours of all pixels are inserted. Then the entire histogram is considered as a colour block and the operations involving trimming of colour blocks and division of colour blocks are carried out until a predetermined number of blocks is available, whereupon each block is allowed to be represented by a colour which is representative of the block. Finally, the picture is presented with the thus-selected colours.

Description

511 669 2 10 15 20 25 30 35 can be used with a GIF image, but it can also reduce the number of colors in other applications and gives a very good result.

The invention solves the problems set forth by having the design as set forth in the following independent patent claim. Suitable embodiments of the invention are set forth in the other patent claims.

The invention will be described in more detail below with reference to the accompanying drawings, where Fig. 1 shows an example of a histogram of red colors, Fig. 2 shows a color cube, a three-dimensional histogram, Fig. 3 shows a color cube with two color blocks, Fig. 4 shows trimming of color blocks, Fig. 5 shows splitting of color blocks, Fig. 6 shows how unfortunate splitting can give rise to small adjacent blocks, Fig. 7 shows merging of small adjacent blocks and Fig. 8 shows how merging can lead to overlapping.

First, various concepts that will be used in the invention must be defined. In the embodiment of the invention that will be described in more detail below, the input data is a True Color image, i.e. a rectangle with pixels where each pixel consists of 24 bits. Other image formats that are read in can first be converted into such an image. Alternatively, the invention can be allowed to act directly on other image formats in a corresponding manner.

The palette optimization according to the invention can be used as soon as the number of colors in the input data exceeds the number of colors to be presented in the output image.

In the example, each pixel is RGB encoded. Other formats can be converted internally to RGB.

This means that the 24 bits are divided into 8 bits red level, 8 bits green and 8 bits blue level. The level for a single color can then vary between 0 and 255. 0 means off and 255 corresponds to full intensity. In other applications of the invention, other divisions into three colors can be used.

The output data is an image with an indexed color palette. The output image has the same width and height as the input image, and when everything works as it should, it looks about the same, while the number of colors has been significantly reduced. 10 15 20 25 30 35 3 511 669 The input data contains the entire color code for a pixel stored in the pixel itself. There is therefore no need for a palette. The output data contains a table with 2 to 256 color codes (in the example with 24-bit RGB). An address, an index, to a code in this table consists of 1 to 8 bits. The color code table is called a palette, and a pixel in the output image consists of an index to the palette.

The present invention is essentially a program for optimizing the palette, i.e. filling the palette with 24 bit RGB codes that are as well chosen as possible. For an image of a forest, for example, there will be many different shades of green in the palette.

In addition, an image must be created where the pixels are indexes to the palette, but that is a simpler task.

The invention works with a three-dimensional histogram of colors. The true histogram is a three-dimensional color cube with, in the example, red, green and blue levels along each axis. Often one sees histograms of one color level at a time, e.g. the red level, see Fig. 1. However, specifying histograms for the red, green and blue levels separately is a simplification. Fig. 2 shows a color cube - a true 3D histogram.

For each 24-bit color there is a cell in the color cube, and in that cell is stored the number of pixels in the input image that have that color. You can think of the color cube as containing a "pixel cloud", with varying density.

The color cube described above has 256 * 256 * 256 = 16,777,216 cells and each cell can be a 32-bit integer. The color cube then takes up just over 67 Megabytes of RAM. For a theoretical mathematician and/or supercomputer owner, this is no problem at all, but in practical use in image processing it is. If, on the other hand, the 24-bit color is reduced by 18 bits, a cube of 64 * 64 ' 64 = 262,144 cells is obtained. This then takes just over one megabyte, if each cell has 32 bits, and just over half a Megabyte for cells with 16 bits. This size is completely reasonable. 18-bit color is better than Hi Color and is actually unnecessarily good for the current application.

With 16 bits per cell, there is a small risk that parts of the color cube will "saturate", i.e. that some cells will hit the ceiling, since 2" -1 is no more than 65,535. This does not pose any significant inconvenience. As an appropriate size for the color cube, 64 " 64 " 64 cells and a sixteen-bit unsigned integer for each cell are suggested in the present case.

In other applications it may be appropriate to use alternative color cubes of other sizes. For example, if the output image is to contain only colors from the standard Netscape palette, a suitable cube size would be 6 ' 6 * 6 = 216 cells (and the cells can then be 32 bits since each cell contains more pixels when there are fewer cells).

The three-dimensional histogram must be able to be reset. There must therefore be a function that sets all cells in the color cube to zero. When you then add an image to the histogram, you roughly do the following for all pixels in the image (input data): - Find out the red, green and blue levels for the current pixel.

- Convert the color levels to appropriate indices in the color cube, e.g. convert 8 bits to 6 bits.

- The cell pointed to by the three indices is incremented by one, unless it is saturated.

- Continue with the next pixel.

By allowing multiple images to be added to the histogram, it may be possible to optimize a common palette for them. This can be useful when working with animations.

There must of course be a function to read cells in the color cube. The function takes three indices (not color levels) as arguments, and returns the number of pixels in the designated cell.

The invention may further include a number of less important functions. Examples of such functions are. - ignore almost empty cells There may be a function to reset the cells in the color cube that only contain a few single pixels, say 1 to 10. The function should be selectable. If the input image contains single stray pixels with odd colors, it can be avoided that these unnecessarily take up valuable space in the finished palette, provided that the stray pixels are uninteresting.

If you optimize a small image with even color distribution, you must be careful not to reset the entire color cube or a large part of it. 10 15 20 25 30 35 511 669 - Selectable saturation If the cells consist of 16 bits, an automatic saturation level of 65,535 pixels per cell is introduced. The user can be allowed to set his own saturation level of say 1000 to 65,535 pixels. This can avoid large solid-colored areas becoming too dominant when assigning palette colors.

- Color cube statistics The color cube can also answer some statistical questions, such as how many pixels it contains in total, how many cells have some content, etc. However, extensive statistics are not necessary here. Things like min, max, mean and standard deviation for red, green and blue are handled by the color blocks, which are described below.

A color block is a rectangular subvolume of the color cube. For each color cube (there is only one), there are between 1 and 256 color blocks, see Fig. 3. The largest size of a color block is equal to the entire color cube and the smallest size is a single cell in the color cube.

Each color block corresponds to exactly one color in the palette and vice versa. The list of 1 to 256 color blocks is the working tool in the palette optimization function according to the invention. The volume of a color block can for example be described by six variables, which are indices to the color cube: red_min, red_max, green_min, green_max, blue_min and blue_max.

The palette optimization according to the invention starts from a single color block of the same size as the entire color cube. A number of trimming, splitting and merging steps are then performed to optimize a palette.

Trimming assumes that a color block should never be allowed to contain any empty edges. A side that only contains zero-set cells is therefore scaled away, see Fig. 4 regarding such trimming. Each block has six sides that can be scaled away if they are empty, if necessary in several steps. If a block is completely empty, it will shrink to zero size during trimming and be removed. (Completely empty blocks should never actually occur.) Ignored pixels, see above, may end up outside all blocks during trimming.

When these pixels are later to be assigned a color in the palette, the closest color is used. 511 669 10 15 20 25 30 35 After trimming, the following statistical variables are already known: red_min, red_max, green_min, green_max, blue_min and blue_max. These are the same variables that describe the size of the block.

Additionally calculated: pixels diagonal color_center red_center green_center blue_center color_size red_size green_size blue_size color_mean red_mean green_mean blue_mean color_deviation red_deviation green_deviation blue_deviation Total number of pixels in the block. This is one of the two measures of the block's size.

The length of the space diagonal of the block. This is the second measure of the size of the block and is a measure of how disparate colors the block contains. (The volume of the block is not used as a measure, since a long narrow block can be very large along one axis and still have a small volume.) The center point of the respective side of the block. red_mid = (red_min + red_max) /2 The length of the respective sides of the block. red_size = (red_max + 1) - red_min The "Midpoint". color_mean is the average value of the respective color level in the block.

Standard deviation for each color level.

When the local statistics have been collected for all blocks, some global statistics are calculated: pixels_average The average value of the number of pixels in the blocks. (Total number of pixels in the cube divided by the number of blocks.) 10 15 20 25 30 35 511 669 diagonal_average The average value of the diagonals of the blocks.

We can now calculate some additional local statistics for each block: = pixels/pixels_mean The pixel size of the block in relation to the other blocks. pixels_relative = diagonal I diagonalI_mean The diagonal, or "color size", of the block in relation to the other blocks. diagonal_relative The diagonal and pixel size of the block are two measures that are not directly comparable to each other, and moreover they vary in different ways when the volume of the block changes.

The purpose of producing relative sizes is to make it easier to weigh the two dimensions against each other. Such a comparison is necessary because the invention involves the step of splitting color blocks, i.e. the largest color block in any selected sense is selected and divided into two smaller ones.

As mentioned, a block has two measures of its size in relation to the other blocks: pixels_relative and diagonal_relative.

It is the user of the program who decides which measure should apply and it may be possible to choose a compromise. For example, if you choose 60% priority on minimizing the diagonals of the blocks and 40% on the pixels, then for each block: weighted_size = 0.6 * diagonal_relative + 0.4 * pixels_relative If it is important to have an even distribution of the colors in the palette, you choose a high priority for the diagonal size (the color measure). If it is more important to have good color resolution for areas that are large, you choose a higher priority for the pixel measure.

The block with the largest weighted score is selected for splitting. A block consisting of only one cell is indivisible and must never be selected for splitting. If there are only single-cell blocks, the palette optimization must be stopped prematurely.

When splitting a block, see Fig. 5, the total number of blocks increases by one. The block being split is cut in half by a line perpendicular to either the red, green or blue axis. The largest axis is chosen for splitting. Here again, the size can be measured in two ways, the length of the side (Weller) or the corresponding color level standard deviation. A weighted average can be calculated in the same way as for weighted_size above. red_weighted_size = priority * (red_size/ mean_size) + (1 - priority) * (red_deviation|see I mean_deviation), where mean_size and mean_deviation apply locally to the block.

A simpler and perhaps more robust measure is red_weighted_size = priority * red_size + (1 - priority) * red_deviation.

Or even more robust: red_weighted_size = red_size + (1 - priority) * red_deviation When an axis, red, green or blue, has been selected for splitting, the location for the split must be determined. This can be the center of the axis or the mean value of the color level or a combination of the two, e.g.: red_split = priority ' red_mid + (1 - priority) * red_mean| The same priority value should be able to be used for selecting the largest block, selecting the split axis and selecting the split coordinate. The split coordinates are of course rounded to a whole number of cells before splitting. The two new blocks are trimmed and statistically processed. Then a new largest block is selected for splitting.

If the priority for the pixel content of the blocks is zero, the algorithm only considers the cell size of the blocks. This can speed up the program, as obtaining detailed pixel statistics becomes unnecessary.

When the number of blocks has reached a predetermined number, between 2 and 256, the first step of palette optimization is complete. After this, you can move on to creating the palette, see further on in the description.

In many cases, however, it is advisable to also perform a second step of merging small blocks before creating the palette. If you are unlucky when splitting the blocks, two small blocks may happen to be close to each other when the desired number of blocks has been reached, see Fig. 6. This is not optimal.

By merging the two small blocks into one, which is still quite small, the splitting process can be continued one more step, and the available space in the palette 10 15 20 25 30 35 9 511 669 is better utilized. This procedure can be repeated as long as there are two blocks that can be merged, without the merged block becoming the largest, see Fig. 7.

The function is a bit combinatorially difficult, but not impossible to implement. The ambitious user selects at each time the pair of blocks that after merging gives the smallest size. When merging blocks there is a certain risk that overlapping blocks will occur, see Fig. 8. Such overlapping can either be allowed or prohibited.

The pixel density of the color cube is defined as the total number of pixels divided by the volume of the entire color cube. This density measure can be calculated once and for all and is then constant during the palette optimization process. The pixel density of a color block can be defined in a similar way, but this measure may change slightly when the block is trimmed, split, or merged.

The pixel density of a block can be normalized by dividing it by the pixel density of the entire color cube. The normalized or relative pixel density, pixeldensity_re|ative, is greater than one (>1) if the block is denser than the color cube on average, and less than one (<1) if the block is sparser than that. The measure does not change when other blocks change shape.

There are some disadvantages to using the relative diagonal and pixel content of the blocks. As soon as something happens to a single block, these relative sizes can change for all blocks.

When merging blocks, a fairly large number of combinatorial possibilities must be examined in order to be able to determine whether there is any pair of blocks that together have a size that is smaller than the largest individual block. The algorithm becomes difficult if all blocks have different sizes before and after each possible merger. It would then be good if one could instead use a size measure that is constant for a certain block, as long as the block itself does not change. The absolute diagonal and the absolute number of pixels are such constant measures, the problem is only to be able to weigh them against each other. The diagonal increases linearly with the size of the block, while the number of pixels increases approximately with the cube.

The pixel density (relative or absolute) is not directly dependent on the size of the block.

By multiplying the density by the diagonal, a pixel measure is obtained that has the same 511 669 19 10 15 20 25 30 35 dimension as the diagonal. weighted_size = priority' diagonal + (1 - priority) * diagonal * pixel|density_relative A little more cautiously, you can instead use weighted_size = diagonal + (1 - priority) * diagonal " pixeldensity_relative It can be a precautionary measure to never let the size of a block be smaller than its diagonal.

After trimming, splitting and merging, the palette creation remains. Conceptually, there is always a one-to-one relationship between the color blocks and the colors in the palette. The color for a certain block can be, for example, red_split, green_split and blue_split as above. Before these levels are entered into the palette table, they must be converted to 8 bits per level. A slightly better color precision can sometimes be achieved if, for example, red_split is not rounded to a whole cell before being converted to 8 bits.

Finally, an output image is constructed, where the pixels consist of indices to the palette. For each pixel, its original RGB value is retrieved from the input image. The RGB value is converted to an index to a cell in the color cube. A color block is selected, either a block that contains the cell, or else the block that is closest. The index to the palette that corresponds to the selected block is the correct output for the current pixel.

As an addition, the user can use a so-called dither function when constructing the output image. The purpose is to achieve apparent multiple colors through rasterization. How this is done is known to those skilled in the art and is not described here.

If the input data consists of several images, the output data can do the same. The principles are the same as for a single image, with the only difference being that the pixels are read from and written to several image rectangles instead of one. Often when working with GlF animations, you want to optimize a common palette for a series of images.

A somewhat simpler version of the algorithm described here has been implemented in Common Lisp on a Texas Explorer ll Lisp machine. Images from a Teragon image processing system in 24-bit color were displayed on the Texas machine, which could display 8-bit color. The experiment was successful. The palette optimization worked well. New versions of the algorithm can be expected to be implemented as Windows 95/NT applications or possibly an application in Java.

Claims (11)

10 15 20 25 30 35 511 669 1 1 Patent claims: 1. Ways to optimize the choice of colors, included in the so-called the color palette, when an image is presented with fewer colors than those found in an original shape of the image, all colors being present as a combination of base colors, such as red, green, and blue (RGB), where the current image determines the current combination of base colors in each pixel and generates a multidimensional histogram with said base colors along the axes, in which the colors of all pixels are entered, so that for each possible color the histogram indicates the number of pixels with that color, characterized by originally looking at the whole histogram as a color block and performs the operations of color block trimming and color block division, trimming means that block pages containing only zeroed cells are peeled off and division that the currently largest block, in a predetermined sense, is divided by a plane perpendicular to the base color axis is parallel to the, in a predetermined sense, the longest block side and at a midpoint, in a predetermined to the side, wherein trimming is performed after each division, and said division of blocks is performed until a predetermined number of blocks is present, after which each block is allowed to be represented by a color representative of the block and finally the image is presented with the colors so selected. . 2. A method according to claim 1, characterized in that after a division it is checked whether it is possible to find two adjacent blocks which can be merged without the merged block becoming the largest block and if so merging them, after which division takes place. , followed by a new check if it is possible to merge blocks, etc. 3. A method according to claim 2, characterized in that the blocks which are merged are allowed to partially overlap each other. Method according to any one of claims 1-3, characterized in that the size of the blocks, weighted_size, is compared based on the size of the space diagonal of the blocks in relation to other blocks, diagonal relative, or the pixel size of the blocks relative to other blocks, pixel_relative, or a predetermined combination of them. Method according to one of the claims! 1_-3, characterized in that the size of the blocks, weighted_size, is compared based on the size of the space diagonal of the blocks, to diagonal, or this quantity multiplied by the relative pixel density, pixel density_relative, for the blocks or a predetermined combination of them. 6. A method according to any one of claims 1-5, characterized in that the length of the sides, color_weighted_large |, is compared based on the length of resp. page in a block in relation to the length of the corresponding page on average, color__size divided by average size, or the standard deviation for resp. color in a block relative to the corresponding standard deviation on average, color deviation divided by mean deviation, or a predetermined combination of them. 7. A method according to any one of claims 1-5, characterized in that the length of the pages, color_weighted_size, is compared based on the length of resp. page in a block, color_size, or the standard deviation of resp. color in a block, color_reviation, or a predetermined combination of them. 8. A method according to any one of claims 1-7, characterized in that the position for dividing a page, color_split, is determined based on the center of the page, color_center, or the color level average, color_means, or a predetermined combination of them. 9. A method according to claim 8, with calculation of the size of the blocks according to claim 4 or 5, the length of the sides according to claim 6 or 7 and the position of division according to claim 8, characterized in that said quantities are calculated as in resp. requirements specified predetermined combination of quantities and where the same ratio between the first mentioned quantity in question and the second quantity is used in the three cases. 10. A method according to any one of claims 1-9, characterized in that the color which is to represent a block consists of the point of intersection between the three planes which would be used if the block were to be divided along its three color axes, color1_split, color2_split resp. color3_split, usually red_split, green_split and blue_split. 11. A method according to any one of claims 1-10, characterized in that the cells in the histogram which contain only a few pixels, preferably at most 10 pieces, are reset before trimming and division begins and in cases where they end up outside a block during the division procedure is used at the final presentation in resp. case a color that is closest to the original color.