SE511669C2 - Ways to optimize the choice of colors when presenting an image - Google Patents

Abstract

The present application relates to a method for optimising the choice of colours, included in the colour palette, when presenting a picture with fewer colours than those available in an original form of the picture. All colours are present as a combination of primary colours, for instance red, green and blue (RGB), and one generates a multidimensional histogram with said primary colours along the axes, in which the colours of all pixels are inserted. Then the entire histogram is considered as a colour block and the operations involving trimming of colour blocks and division of colour blocks are carried out until a predetermined number of blocks is available, whereupon each block is allowed to be represented by a colour which is representative of the block. Finally, the picture is presented with the thus-selected colours.

Description

511 669 2 10 15 20 25 30 35 can be used with a GIF image, but it can also reduce the number of colors in other applications and thereby gives a very good result.

In the invention, the problems posed are solved by giving it the design which appears from the following independent patent claim. Suitable embodiments of the invention appear from other patent claims.

The invention will be described in more detail below with reference to the accompanying drawing, in which Fig. 1 shows an example of a histogram of red colors, Fig. 2 shows a color cube, a three-dimensional histogram, fi g. Fig. 3 shows a color cube with two color blocks, Fig. 4 shows trimming of color blocks, Fig. 5 shows division of color blocks, Fig. 6 shows how unfortunate division can give rise to small adjacent blocks, Fig. 7 shows merging of small adjacent blocks and Figs. 8 shows how merging can lead to overlap.

First, various concepts that will be used in the invention must be defined. The data in the embodiment of the invention which will be described in more detail below is a True Color image, i.e. a rectangle of pixels where each pixel consists of 24 bits. Other image formats that are loaded can first be converted to such an image. Alternatively, the invention may operate directly on other image formats in a corresponding manner.

The palette optimization according to the invention can be used as soon as the number of colors in the input data exceeds the number of colors to be presented in the education.

In the example, each pixel is RGB encoded. Other formats can be converted internally to RGB.

This means that the 24 bits are divided into 8 bits red level, 8 bits green and 8 bits blue level. The level of an individual color can then vary between 0 and 255. 0 means off and 255 corresponds to full intensity. In other applications of the invention, other divisions in three colors can be used.

Output is an image with an indexed color palette. The image has the same width and height as the image, and works everything as it should, it looks roughly the same, while the number of colors is significantly reduced. 10 15 20 25 30 35 3 511 669 The input data contains the entire color code of a pixel stored in the pixel itself. There is therefore no need for a palette. In the output there is a table with 2 to 256 color codes (in the example with 24 bit RGB). An address, an index, of a code in this table consists of 1 to 8 bits. The color code table is called a palette, and a pixel in the education consists of an index to the palette.

The present invention is essentially a program for optimizing the palette, i.e. fill the palette with 24 bit RGB codes that are as well selected as possible. For an image of a forest, ddet_t.ex. there are many different shades of green in the palette.

In addition, an education will be created where the pixels are the index to the palette, but it is a simpler task.

The invention works with a three-dimensional histogram of colors. The true histogram is a three-dimensional color cube with, in the example, red, green and blue levels along each axis. You often see histograms over one color level at a time, e.g. the red level, see Fig. 1. However, specifying the histograms for the red, green, and blue levels separately is a simplification. Fig. 2 shows a color cube - a real 3D histogram.

For every 24-bit color, there is one cell in the color cube, and in that cell the number of pixels in the input image that has just that color is stored. It is conceivable that the color cube contains a "pixel cloud", with varying density.

The color cube described above has 256 * 256 * 256 = 16 777 216 cells and each cell can be a 32 bit integer. The color cube then takes up just over 67 Megabytes of RAM. For a theoretical mathematician and / or supercomputer, this is not a problem at all, but in practical use in image processing it is. If, on the other hand, the 24-bit color is reduced by 18 bits, a cube of 64 * 64 '64 = 262 144 cells is obtained. It then takes just over one ï agabyte, if each cell has 32 bits, and just over half a megabyte for cells with 16 bits. This size is perfectly reasonable. 18 bit color is better than Hi Color and is really unnecessarily good for the current application.

With 16 bits per cell, there is a small risk that parts of the color cube are "saturated", ie. that some cells strike the ceiling, since 2 "-1 is not more than 65,535. This does not constitute a significant inconvenience. As the appropriate size of the color cube is proposed in the present case 64" 64 "64 cells and a sixteen-bit integer without signs of each cell.

In other applications, it may be appropriate to use alternative color cubes of other sizes. If e.g. the output image should only contain colors from the standard 511 669 4 10 15 20 25 30 35 Netscape palette, then a suitable size of the cube will be 6 '6 * 6 = 216 cells (and the cells can then have 32 bits because each cell contains fl er pixels when there are fewer cells).

It must be possible to reset the three-dimensional histogram. There must therefore be a function that sets all cells in the color cube to zero. When you then want to add an image to the histogram, you basically do the following for all the pixels in the image (input data): - Find out the red, green and blue levels for the current pixel.

- Convert the color levels to appropriate indices in the color cube, e.g. dec 8 bits to 6 bits.

- The cell designated by the three indices is incremented by one, unless it is saturated.

- Continue with the next pixel.

By allowing multiple images to be added to the histogram, it may be possible to optimize a common palette for them. It can be good when working with animations.

Of course, there must be a function to read the cell in the color cube. The function takes three indices (not color levels) as arguments, and returns the number of pixels in the designated cell.

The invention may further comprise a number of less important functions. Examples of such functions are. - ignore almost empty cells There may be a function to reset the cells in the color cube that contain only a few pixels, say 1 to 10 pieces. The function must be selectable. If the input image contains single scatter pixels with odd colors, you can avoid these unnecessarily taking up valuable space in the finished palette, provided that the scatter pixels are uninteresting.

If you optimize a small image with an even color distribution, you have to be careful not to reset the entire color cube or a large part of it. 10 15 20 25 30 35 511 669 - Selectable saturation If the cells consist of 16 bits, an automatic saturation level of 65,535 pixels per cell is introduced. The user can be allowed to set their own saturation level of say 1000 to 65,535 pixels. One can then avoid that large monochromatic surfaces become too dominant when assigning palette colors.

- Color cube statistics The color cube can also answer some statistical questions, e.g. how many pixels it contains in total, how many cells have any content, etc. However, comprehensive statistics are not necessary here. Things such as minimum, maximum, mean and standard deviation for red, green and blue are handled by the color blocks described below.

A color block is a rectangular subvolume of the color cube. For each color cube (there is only one) there are between 1 and 256 color blocks, see fi g. 3. The largest size for a color block is equal to the entire color cube and the smallest size is a single cell in the color cube.

Each color block corresponds exactly to one color in the palette and vice versa. The list of 1 to 256 color blocks is the work tool in the palette optimization function according to the invention. the volume of a color block can e.g. is described by six variables, which are indices of the color cube: red_min, red_max, green_min, green_max, blue_min and blue_max.

The palette optimization according to the invention is started on the basis of a single color block with the same size as the entire color cube. Then a number of steps are performed regarding trimming, splitting and merging to optimize a pallet.

Trimming assumes that a color block should never be allowed to contain any blank edges. A page that only contains reset cells is therefore peeled off, see fl g. 4 regarding such trimming. Each block has six sides that can be peeled off if empty, if necessary in several steps. If a block is completely empty, it will shrink to zero size during trimming and will be removed. (Completely empty blocks should never really appear.) Ignored pixels, see above, may end up outside all blocks when trimming.

When these pixels are later to be assigned a color in the palette, you have to take the one that happens to be closest. 511 669 10 15 20 25 30 35 After trimming, the following statistical variables are already known: red_min, red_max, green_min, green_max, blue_min and blue_max. These are the same variables that describe the size of the block.

Also calculated: pixels diagonal color_mid red_mid green_mid blue_mid color_size red_size green_size blue_size color_means red_means green_means blue_means color_division red_division green_division blue_division Total number of pixels in the block. This is one of the two measures of block size.

The length of the block's space diagonal. This is the second measure of the size of the block and is a measure of how disparate colors the block contains. (The volume of the block is not used as a measure, as a long narrow block can be very large along an axis and still have a small volume.) Resp. side center of the block. red_mid = (red_min + red_max) / 2 The length of the block resp. pages. red_size = (red_max + 1) - red_min "Center". color_means is the mean of resp. color level in the block.

Standard deviation for each color level.

When the local statistics are collected for all blocks, some global statistics are calculated: pixels_means The average value of the number of pixels in the blocks. (Total number of pixels in the cube divided by the number of blocks.) 10 15 20 25 30 35 511 669 diagonal_ mean The mean value of the diagonals of the blocks.

You can now calculate some additional local statistics for each block: = pixels / pixels_means The pixel size of the block in relation to the other blocks. pixels_relative = diagonal In diagonal_medium The block's diagonal, or "color size", relative to the other blocks.diagonal_relative The diagonal and pixel size of the block are two dimensions that are not readily comparable to each other, and also vary in different ways when the block volume changes.

The purpose of producing relative sizes is to make it easier to weigh the two dimensions against each other. Such a comparison is necessary, since the invention comprises the step of splitting color blocks, i.e. that the largest color block in any selected sense is selected and divided into two smaller ones.

As stated, a block has two measures of its size in relation to the other blocks: pixel_re | relative and diagonal_relative.

It is the user of the program who decides which measure should apply and it may be possible to choose a compromise. If one e.g. selects 60% priority on minimizing the diagonals of the blocks and 40% on the pixels, this applies to each block: weighted_size = 0.6 * diagonallative + 0.4 * pixels_relative If it is important to evenly distribute the colors in the palette, choose a high priority for the diagonal size (the color measure). If it is more important with good color resolution for areas that are large-Hailden, you choose a higher priority for the pixel size.

The block with the largest weighted sßfleken is selected for splitting. A block consisting of only one cell is indivisible and must never be selected for cleavage. If there are only single-cell blocks, the palette optimization must be stopped prematurely.

When splitting a block, see fi g. 5, increases the total number of blocks by one. The block that is split is cut in half with one that is perpendicular to either the red, green or blue axis. The largest axis is selected for division. Here, the size can again be measured in two ways, the length of the page (Weller) or the standard deviation of the corresponding color level 511 669 8 10 15 20 25 30 35. A weighted average value can be calculated in the same way as for weighted_size above. red_weighted_size = priority * (red_size / average size) + (1 - priority) * (red_awike | see In mean deviation), where mean size and mean deviation apply locally to the block.

A simpler and perhaps more robust measure is red_weighted_size = priority * red_size + (1 - priority) * red_reviation.

Or even more robust: red_weighted_size = red_size + (1 - priority) * red_division When an axis, red, green or blue, has been selected for division, the position of the division must be determined. It can be the center of the axis or the mean value of the color level or a combination of the two, eg: red_split = priority 'red_center + (1 - priority) * red_smith | The same priority value should be able to be used both for selection of the largest block, selection of splitting axis and for selection of splitting coordinates. The cleavage coordinates are of course rounded to a whole number of cells before division takes place. The two new blocks are trimmed and statistics processed. Then a new largest block for splitting is selected.

If the priority of the pixel content of the blocks is zero, the algorithm only takes into account the cell size of the blocks. The program can then be speeded up, as obtaining detailed pixel statistics becomes unnecessary.

When the number of blocks has reached a predetermined number, between 2 and 256, the first step of pallet optimization is completed. After this, you can move on to creating the palette, see further later in the description.

In many cases, however, it is advisable to also perform a second step regarding merging small blocks before creating the palette. Namely, if you are unlucky in the block splitting, two small blocks can by chance be close to each other, when the desired number of blocks has been reached, see fi g. 6. This is not optimal.

By merging the two small blocks into one, which is still quite small, the cleavage process can continue one step further, and the available space in the pallet 10 15 20 25 30 35 9 511 669 is better utilized. This procedure can be repeated as long as there are two blocks that can be merged, without the merged block being the largest, see fi g. 7.

The function is a bit combinatorially difficult, but not impossible to implement. The ambitious user selects at each occasion the pair of blocks that, after merging, give the smallest size. When merging blocks, there is a certain risk of overlapping blocks, see fi g. 8. Such overlap can either be allowed or prohibited.

The pixel density of the color cube is defined as the total number of pixels divided by the volume of the entire color cube. This density measure can be calculated once and for all and is then constant during the palette optimization process. The pixel density of a color block can be similarly reduced, but that dimension may change slightly when the block is trimmed, split or merged.

The pixel density of a block can be normalized by dividing it by the pixel density of the entire color cube. The standardized or relative pixel density, pixel density_re | ative, is greater than one (> 1) if the block is denser than the color cube on average, and less than one (<1) if the block is sparser than that. The dimension does not change when other blocks change shape.

This has some disadvantages in using the relative diagonal and pixel content of the blocks. As soon as something happens to a single block, these relative sizes can change for all blocks.

In the event of a merger of blocks, a fairly large number of combinatorial possibilities must be investigated, in order to be able to ascertain whether there are any pairs of blocks that together have a size that is smaller than the largest individual block. The algorithm becomes cumbersome, all blocks have different sizes before and after each possible merger. i It would then be good if you could instead use a size measure that is constant for a certain block, as long as the block itself does not change. The absolute diagonal and the absolute number of pixels are such constant measurements, the problem is just to be able to weigh them against each other. The diagonal increases linearly with the size of the block while the number of pixels increases approximately with the cube.

The pixel density (relative or absolute) is not directly dependent on the size of the block.

By multiplying the density by the diagonal, a pixel dimension is obtained which has the same dimension as the diagonal. weight_size = priority 'diagonal + (1 - priority) * diagonal * pixe | density_relative A little more cautiously you can instead use weight_size = diagonal + (1 - priority) * diagonal "pixel density_relative It can be a precaution to never let a block size decrease than its diagonal.

After trimming, splitting and merging, the creation of the palette remains. On an ideological level, there is always a one-to-one relationship between the color blocks and the colors in the palette. The color of a certain block can e.g. be red_split, green_split and blue_split as above. Before entering these levels into the palette table, they must be converted to 8 bits for the habit level. A slightly better color precision can sometimes be achieved, if e.g. red_split is not rounded to an entire cell before it is converted to 8 bits.

Finally, an output data image is constructed, where the pixels consist of an index to the palette. For each pixel, its original RGB value is retrieved from the input image. The RGB value is converted to an index to a cell in the color cube. A color block is selected, either a block that contains the cell, or the block closest to it. The index to the palette that corresponds to the selected block is the correct output for the current pixel.

As an add-on, the user can use a so-called. dither function in the construction of the output data image The purpose is to achieve seemingly more colors by rasterization. How this is done is known to those skilled in the art and is not described here.

If the input consists of several images, the output can do the same. The principles are the same as for a single image, with the only difference that the pixels are read from and written to several image rectangles instead of one. Often when working with GlF animations, you want to optimize a common palette for a series of images.

A slightly simpler variant of the algorithm described here has been implemented in Common Lisp on a Texas Explorer ll Lisp machine. Images from a 24-bit Teragon image processing system were displayed on the Texas machine, which could display 8-bit color. The experiment was successful. The palette optimization worked well. New versions of the algorithm can be expected to be executed as Windows 95 / NT applications or possibly an application in Java.

Claims (11)

10 15 20 25 30 35 511 669 1 1 Patent claims: 1. Ways to optimize the choice of colors, included in the so-called the color palette, when an image is presented with fewer colors than those found in an original shape of the image, all colors being present as a combination of base colors, such as red, green, and blue (RGB), where the current image determines the current combination of base colors in each pixel and generates a multidimensional histogram with said base colors along the axes, in which the colors of all pixels are entered, so that for each possible color the histogram indicates the number of pixels with that color, characterized by originally looking at the whole histogram as a color block and performs the operations of color block trimming and color block division, trimming means that block pages containing only zeroed cells are peeled off and division that the currently largest block, in a predetermined sense, is divided by a plane perpendicular to the base color axis is parallel to the, in a predetermined sense, the longest block side and at a midpoint, in a predetermined to the side, wherein trimming is performed after each division, and said division of blocks is performed until a predetermined number of blocks is present, after which each block is allowed to be represented by a color representative of the block and finally the image is presented with the colors so selected. . 2. A method according to claim 1, characterized in that after a division it is checked whether it is possible to find two adjacent blocks which can be merged without the merged block becoming the largest block and if so merging them, after which division takes place. , followed by a new check if it is possible to merge blocks, etc. 3. A method according to claim 2, characterized in that the blocks which are merged are allowed to partially overlap each other. Method according to any one of claims 1-3, characterized in that the size of the blocks, weighted_size, is compared based on the size of the space diagonal of the blocks in relation to other blocks, diagonal relative, or the pixel size of the blocks relative to other blocks, pixel_relative, or a predetermined combination of them. Method according to one of the claims! 1_-3, characterized in that the size of the blocks, weighted_size, is compared based on the size of the space diagonal of the blocks, to diagonal, or this quantity multiplied by the relative pixel density, pixel density_relative, for the blocks or a predetermined combination of them. 6. A method according to any one of claims 1-5, characterized in that the length of the sides, color_weighted_large |, is compared based on the length of resp. page in a block in relation to the length of the corresponding page on average, color__size divided by average size, or the standard deviation for resp. color in a block relative to the corresponding standard deviation on average, color deviation divided by mean deviation, or a predetermined combination of them. 7. A method according to any one of claims 1-5, characterized in that the length of the pages, color_weighted_size, is compared based on the length of resp. page in a block, color_size, or the standard deviation of resp. color in a block, color_reviation, or a predetermined combination of them. 8. A method according to any one of claims 1-7, characterized in that the position for dividing a page, color_split, is determined based on the center of the page, color_center, or the color level average, color_means, or a predetermined combination of them. 9. A method according to claim 8, with calculation of the size of the blocks according to claim 4 or 5, the length of the sides according to claim 6 or 7 and the position of division according to claim 8, characterized in that said quantities are calculated as in resp. requirements specified predetermined combination of quantities and where the same ratio between the first mentioned quantity in question and the second quantity is used in the three cases. 10. A method according to any one of claims 1-9, characterized in that the color which is to represent a block consists of the point of intersection between the three planes which would be used if the block were to be divided along its three color axes, color1_split, color2_split resp. color3_split, usually red_split, green_split and blue_split. 11. A method according to any one of claims 1-10, characterized in that the cells in the histogram which contain only a few pixels, preferably at most 10 pieces, are reset before trimming and division begins and in cases where they end up outside a block during the division procedure is used at the final presentation in resp. case a color that is closest to the original color.