RU2481635C2 - Adaptive method to create and print colour anaglyph images - Google Patents

Abstract

FIELD: information technologies.

SUBSTANCE: in the method a test colour sample is printed on a device of anaglyph images printing, functions of spectral transmission are assessed for the left and right filter of glasses for a printed test sample, a stereomate is generated, an anaglyph image is generated for a stereomate, and the anaglyph image is printed.

EFFECT: adaptation for printing of an anaglyph image with preservation of 3D perception of a printed image for specific stereo glasses.

7 cl, 7 dwg

Description

The invention relates to the field of image processing, and more specifically, the invention relates to methods for preparing color anaglyph images for reproducing a three-dimensional effect on paper using special stereo glasses with various color filters.

The prior art various approaches to solving the problem of creating anaglyph images. However, all these solutions describe methods that use the previously known spectral transmission functions of glasses and do not take into account the ratio of glasses and printer colors to each other. Also, these methods do not improve the quality of the anaglyph image and do not try to reduce the ghost-effect when viewing the anaglyph image.

Existing methods for creating and printing three-dimensional color anaglyph images have two types of artifacts: defects associated with incorrect conversion of disparity (or depth), and defects associated with improper color mixing. Separate methods solve the problem of incorrect disparity assessment by aligning the stereo pair, correcting its colors and modifying the disparity card. In this case, the alignment of stereo pairs provide a fairly good quality stereo image. Users can control this process through the user interface. For example, they can shift stereopair images relative to each other, changing the depth of three-dimensional perception.

Nevertheless, the question of incorrect color mixing remains open. The key point is the problem of matching the color of the anaglyph image with the spectral characteristics of the glasses and the colors of the printing device. All known methods use the previously known spectral transmission functions, the same for creating any anaglyph images, while the colors of the stereo glasses filters are very diverse. As a result, if the spectral characteristics of the glasses do not fully correspond to the colors of the printer, when viewing an anaglyph image through stereo glasses, there is a double effect. At the moment, no simple ways to solve this problem have been found, in addition, the existing software for creating anaglyph images does not take into account the size of the image when printing. Although it is obvious that the size of the printed image affects the features of three-dimensional perception.

As one of the most promising should be mentioned US patent 7680322 [1], which describes a method of printing images for viewing using stereo glasses. This is realized by generating individual images for the left and right eyes, performing the image alignment and modification procedures, and then creating an anaglyph image. However, this method does not allow the adaptation of the spectral transmission functions of stereo glasses with respect to the colors of the printed anaglyph image, and the printer does not select the colors for achieving the best quality three-dimensional perception of the anaglyph image. However, this method is closest to the claimed invention and can be considered as a prototype.

The problem to which the invention is directed is to develop a method for synthesizing anaglyph images adapted for specific stereo glasses and preserving their three-dimensional perception for these glasses. Moreover, this method should allow the possibility of implementation in a color printing device or in the software of printing devices.

The technical result is achieved through the use of a new method for creating and printing color anaglyph images, while the inventive method consists of the following steps:

- print a test color sample on a printing device designed to print anaglyph images;

- evaluate the spectral transmission functions of the left and right filter glasses for the printed test sample;

- form (prepare) a stereo pair;

- generate anaglyph image for stereo pair taking into account the functions of spectral transmission;

- print anaglyph image.

When forming stereopairs, the size of the printed anaglyph image is taken into account and the following steps are performed:

- align the right and left images of the stereo pair geometrically;

- adjust the colors of the right and left images of the stereo pair;

- blur at least one of the two color channels of the right and left images of the stereo pair;

- evaluate the disparity map;

- modify the disparity map in order to reduce the disparity values that exceed the threshold;

- change the right and left images of the stereo pair in accordance with the modified disparity map.

The formation of a stereo pair, if necessary, is performed in an alternative way:

- align the right and left images of the stereo pair geometrically;

- adjust the colors of the right and left images of the stereo pair;

- blur at least one of the two color channels of the right and left images of the stereo pair;

- evaluate the depth map;

- modify the depth map in order to reduce the depth range of the scene, if it exceeds a predetermined threshold;

- change the right and left images of the stereo pair in accordance with the modified depth map.

There are four main differences between the proposed method and the existing ones:

- a user assessment of the spectral transmission functions of the glasses filters is performed and their further approximation by the spectral transmission functions for filters existing on the market;

- creates adaptive anaglyph images from stereo pairs for specific glasses;

- the print size of the anaglyph image is taken into account when preparing a stereo pair;

- when preparing an anaglyph image, color channels are blurred.

Figure 1. An example of printing an anaglyph image by methods previously known.

Figure 2. An example of printing anaglyph image of the claimed method.

Figure 3. The block diagram of the implementation of the proposed method.

Figure 4. Scheme of a system that implements the inventive method.

Figure 5. A color sample for assessing the visibility / invisibility of its colors through eyeglass filters.

6. Flow chart for spectral transmittance functions for glasses filters.

7. Examples of ratings on a scale of visibility / invisibility of color perception.

A block diagram of the main steps of the algorithm is presented in Fig.3. The first step 301 is to print a test sample of a color sample (standard or template) on a printer. The sample contains a series of colors, for example, in the form of a table on the plane. Color tone (or just color) from 0 to 360 ° / Saturation from 0 to 1 HSL color space. This space quite well represents the dependence of color on wavelength (λ). Columns of the table - color tone with values varying from 0 ° to 330 ° (step 10 °). Rows (rows) of the table are colors with saturation / purity from 1 to 0.2 (step 0.2). Then, in step 302, the transmission characteristics of the glasses filters for the printed sample are evaluated. This procedure is necessary to achieve the best consistency between the stereo glasses filters and printer inks. Suppose there are two filters, red and blue. To obtain the approximate spectral transmission function of the glasses filters, the degree of visibility / invisibility of the colors on the sample is evaluated through the left and then the right filters. The procedure for printing a test color sample and evaluating the spectral transmission functions can be performed once for these stereo glasses. The user sequence for evaluating the spectral transmittance functions is described in FIG. 6. After establishing the levels of visibility by the user, it is determined which function of spectral transmission of real filters produced in the industry is closest to the function obtained as a result of evaluation by the user. The next step 303 is the formation (or preparation) of the stereo pair, which includes the geometric alignment of the stereo pair, the correction of its colors, estimation of the depth or disparity map, blurring of the image along two color channels and reducing the range of values of the disparity map or depth map. After completing all the preliminary steps, at step 304, it becomes possible to generate an anaglyph image taking into account the spectral transmission functions of the stereo glasses filters. Anaglyph image is directly generated using the projection method described in Eric Dubois's article "A projection method to generate anaglyph stereo images", Proc. of ICASSP, IEEE International Conference "Acoustic, Speech, and Signal Processing", vol. 3, pp. 1661-1664, 2001 [2]. After all these steps, the anaglyph image is printed at step 305.

Thus, a color sample printed on a specific printer is analyzed by a user who looks at him through stereo glasses (through each filter separately). In this way, you can create anaglyph images for any stereo glasses with color filters that the user has. In this case, anaglyph images will provide a good three-dimensional effect and have a minimum of artifacts when viewed on a computer monitor, so when printing on paper.

A block diagram of a system that implements the inventive method for creating and printing color anaglyph images is presented in Figure 4. Such a system takes into account the capabilities of modern printing devices. A central processing unit 401 (CPU) controls all modules of the system. Created images are printed on a printer 402. The method parameters are set by the user interface module 403. There are several options for implementing the user interface. The created image is stored in random access memory 404 (RAM). The central processor 401 controls all modules of the system and performs the creation of an anaglyph image by executing instructions stored in read-only memory 405 from ROM. Hard disk 406 is optionally used to store image files. A data bus 407 provides data transfer between system modules.

To evaluate the spectral transmission function of the glasses filters f ₁ (λ) and f _r (λ), where λ is the wavelength, a test color sample (standard or template) printed on this printing device is used. In a preferred embodiment of the invention, printing a color test sample on a printing device consists of the following steps:

- print a series of colors with different saturation / purity and color tone;

- print user instructions.

Figure 5 presents the test color sample and user instructions for its use. The instruction includes, for example, the following: “Please look at the first row of the sample through the left filter of glasses and for all the squares in the row, assign ratings (from 0 to 5, 0 - completely invisible, 5 completely visible). If in this row there are no invisible squares, please go to the next rows one by one until you find the row in which they are and conduct visibility assessments as described earlier. Do the same for the right filter points. " Printing instructions can be performed both on the same page as a series of colors, and on the back of the page, as well as on a separate page.

The sample contains a series of colors, for example, in the form of a table on the plane. Color tone (or just color) from 0 to 360 ° / Saturation from 0 to 1 HSL color space. This space in the best way, unlike other color spaces, represents the dependence of color on wavelength (λ). Columns of the table - color tone with values varying from 0 ° to 330 ° (step 10 °). The rows of the table are colors with saturation / purity from 1 to 0.2 (step 0.2). For example, glasses with red and blue filters are the most common. In order to obtain the spectral transmission function of stereo glasses filters, it is necessary to evaluate the invisibility of the colors from the sample near red and blue, observing them through the red and then the blue filters, respectively. If the colors merge with the background, i.e. invisible, then they are completely skipped by the filter. In the general case, the method for evaluating the spectral transmission functions allows one to evaluate the transmission characteristics for any type of stereo glasses with any filters.

The spectral transmission functions of the left and right filter points are evaluated by performing the following steps:

- estimate approximately the spectral transmission function for the left filter points according to the values corresponding to the user's choice of visible and invisible colors on the test color sample when viewed through the left filter;

- approximately evaluate the spectral transmission function for the right filter points according to the values corresponding to the user's choice of visible and invisible colors on the test color sample when viewed through the right filter;

- approximate approximate estimates of the spectral transmission functions by the closest functions of real filters from a given set.

In a preferred embodiment of the invention, the spectral transmission functions of the glasses filters for colors printed on a printing apparatus are evaluated as follows (see FIG. 6). Typically, a color with a color tone of 10 ° ± 10 ° corresponds to red with a wavelength of 700 nm ± 27.50 nm, blue = 180 ° ± 10 ° corresponds to blue with a wavelength of 495 nm ± 5 nm, yellow = 50 ° ± 10 ° corresponds to yellow with a wavelength of 564 nm ± 11 nm, green = 120 ° ± 10 ° corresponds to green with a wavelength of 530 nm ± 5 nm. The evaluation of the spectral transmittance functions begins with viewing the first row of the test color sample (step 602). The first row of the test color sample corresponds to saturation / purity = 1, lightness = 1, and color from 0 ° to 330 °. Let the left stereo glasses filter be red. The user needs to look at step 603 this row through the red glasses filter, taking into account that three squares with colors = 10 ° ± 10 ° (j = 1, 2, 3) correspond to red color with some variation of its wavelength. If the color from the range of 10 ° ± 10 ° (j = 1, 2, 3) is invisible (see step 604), then the maximum of the spectral transmittance function is in the range of 700 nm ± 27.50 nm (see step 606) if the colored squares are visible , then you need to select the next row (see step 605) with less saturation / purity and view this row similarly (see steps 603, 604). If the colors are still visible, then it is necessary to select the series with the lowest saturation / purity for analysis (see step 616). The maximum value of the spectral transmittance function depends on the choice of saturation / purity (row number in the test color sample). Saturation / Purity = 1; 0.8; 0.6; 0.4; 0.2 one-to-one correspond to the maxima of the spectral transmission functions = 0.9; 0.85; 0.65; 0.50; 0.45. When a number of test color samples is determined, then the degree of visibility of each color square is analyzed. The current position in the selected row (the number of the square in the row) is j = 1 (see step 617). If this is not the last position in the row (see step 618), the user should analyze the degree of visibility of the color square (see step 619) and give a rating (from 5 to 0) to each square. The rating system is presented in Fig.7. Completely invisible color squares have a rating of 5, fully visible squares have a rating of 0. Each rating of 0, 1, 2, 3, 4, 5, depending on the maximum transmittance (MaxTrans) at the current saturation / purity, corresponds to the value of spectral transmittance 0, 0.20 , 0.45, 0.60, 0.85, 1.0. Next, go to the next row position (see step 621). Thus, the spectral transmittance function for the red filter of the glasses data is estimated.

Let the right stereo glasses filter be blue. For a blue filter, the spectral transmittance function is estimated in a similar way. The blue values are in the range 180 ° ± 10 ° (495 nm ± 5 nm). The user must repeat the whole sequence of actions, as in the case of evaluating the spectral transmission function of the red filter, but in this case, to select a series, pay attention to the squares with a color tone of 180 ° ± 10 ° (j = 18, 19, 20), which correspond to blue with some variation in wavelength (see step 603). All subsequent steps are performed, as in the case of evaluating the spectral transmission function of the red filter of stereo glasses. Similarly, you need to act for any color filter stereo glasses.

- оценка позиций максимумов функций F_i, где F_i - функция спектрального пропускания для левого F_l или для правого F_r реального, например Roscolux, фильтра, f_i - функция оценки спектрального пропускания для левого f_l и правого f_r фильтров очков. Затем для максимумов соответствующих одинаковой длине волны необходимо применить следующее условие для выбора реального фильтра:

- условие ближайшего значения пропускаемости.Thus, an approximate estimate of the spectral transmission functions is obtained. The direct use of the obtained functions when creating an anaglyph image leads to improper color mixing and, accordingly, low quality restoration of three-dimensional objects on an anaglyph image. In order to achieve high quality in the proposed method for creating anaglyph images, the well-known spectral transmission functions of real filters manufactured in the industry are used, for example, the filter functions of Roscolux (http://www.rosco.com/us/filters/roscolux can be used. cfm) [3]. Therefore, approximate estimates of the spectral transmission functions (f) are approximated by the closest functions (F _i ) of real filters from a given set. The approximation consists of the following steps. Choose F _i as:

- assessment of the positions of the maxima of the functions F _i , where F _i is the spectral transmittance function for the left F _l or for the right F _r real, for example, Roscolux, a filter, f _i is the spectral transmittance estimation function for the left f _l and right f _r glasses filters. Then, for the maxima corresponding to the same wavelength, the following condition must be applied to select a real filter:

- condition of the closest transmittance value.

A method of forming or preparing a stereo pair includes the following steps:

- align the right and left images of the stereo pair geometrically;

- adjust the colors of the right and left images of the stereo pair;

- blur at least one of the two color channels of the right and left images of the stereo pair;

- evaluate the disparity map;

- modify the disparity map in order to reduce the disparity values exceeding the threshold;

- change the right and left images of the stereo pair in accordance with the modified disparity map.

An alternative way to prepare a stereo pair includes the following steps:

- align the right and left images of the stereo pair geometrically;

- adjust the colors of the right and left images of the stereo pair;

- blur at least one of the two color channels of the right and left images of the stereo pair;

- evaluate the depth map;

- modify the depth map in order to reduce the depth range of the scene, if it exceeds a predetermined threshold;

- change the right and left images of the stereo pair in accordance with the modified depth map.

In a preferred embodiment of the invention, the preparation of a stereo pair is as follows (see Figure 3). Geometric alignment of the stereo pair is performed as described in the article Hsien-huang P. Wu and Chih-Cheng Chen "Projective Rectification with Minimal Geometric Distoration", pp.530, I-Tech, 2007 [4]; The color correction of the stereo pair images is performed by finding the correspondence of the histograms, as, for example, described (http://paulbourke.net/texture_colour/equalisation/) [5]. Evaluation of the disparity map is implemented, for example, using the statistical method described in the article by Hongshi Yan and Jian Guo Liu "Robust Phase Correlation Based Sub-pixel Disparity Estimation", 4th SEAS DTC Technical Conference - Edinburgh 2009 [6]. The disparity map is converted to the depth map, and vice versa. For a number of reasons, a depth map is sometimes used, sometimes a disparity map.

Two color channels (red and blue) of the left and right images of the stereo pair are blurred using an averaging filter. Filter parameters depend on the size of the printed anaglyph image. The green color channel is not blurred, since the human eye perceives green best of all colors.

Averaging filter:

,

,

where I (x, y) is the value of the pixel intensity of a certain color channel (blue, red, green), I (x, y) ∈ [0; 255];

(x, y) - position of the pixel in the image, (x, y) ∈A × B; A, B - image size;

- среднее значение интенсивностей пикселей в некоторой прямоугольной окрестности с размерами N_x, N_y; размеры окрестности зависят от размера печатаемого анаглифного изображения;

- the average value of the intensities of the pixels in a rectangular neighborhood with sizes N _x , N _y ; the size of the neighborhood depends on the size of the printed anaglyph image;

(n _x , n _y ) is the position of the pixel in the neighborhood for averaging with sizes N _x × N _y , N _x × N _y ⊂A × B.

The range values of the disparity map are reduced depending on the size of the printed anaglyph image. This is necessary to reduce the double effect when viewing an anaglyph image through stereo glasses, which is formed as a result of interference of waves reflected from the surface of a printed anaglyph image:

,

,

where D (i, j) is the disparity value at the position (i, j) ∈A × B before the transformation, A, B is the image size;

- значение диспарантности в позиции после преобразования;

- the value of the disparity in the position after the conversion;

a and P are constants whose values depend on the size of the printed anaglyph image, 0 ≤ a ≤1, 0 <P <1.

When resizing an anaglyph image, it is important to convert the disparity values for the best quality of three-dimensional image perception. Changing the disparity is non-linear: increasing the size of the image should not lead to huge disparity values compared to the original values, reducing the size of the image should not lead to the zeroing of the disparity values and the disappearance of the three-dimensional effect. A similar correction can be made to the depth map.

In addition, using the user interface, you can align the stereo pair manually, change the parallax to modify the depth of the three-dimensional image, evaluate the transmission characteristics and select the size of the anaglyph image for printing.

In a preferred embodiment of the invention, an anaglyph image is generated from a stereo pair using a projection method taking into account the spectral transmission functions of the filters for these stereo glasses. Anaglyph image is generated according to the formula:

,

,

,

,

,where x is the position of the pixel in the image, x∈A × B; A, B - image size,

,

,

,

F _l (λ) and F _r (λ) are the spectral transmission functions of real filters for the left and right filters, λ is the wavelength; W is a matrix of weighting factors of size 6 × 6,

- функции сложения цветов для стандартного колориметрического наблюдателя (спектральная чувствительность глаза), принятые международной комиссией по освещению, табличные значения которых можно узнать в Color Physics for Industry, Second Edition, Edited by Roderick McDonald, 1997 [7]. k=1, 2, 3 - индексы для каждой компоненты цвета (красный, зеленый, синий), d_j(λ) - спектральное распределение энергии одного из стандартных колориметрических излучений, табличные значения которого можно узнать в [7]. j=1, 2, 3 - индексы для каждой компоненты цвета (красный, зеленый, синий); V(x)=[V₁₁(x)V_l2(x)V_l3(x)V_r1(x)V_r2(x)V_r3(x)]^T, N - нормализующая матрица, необходимая для выполнения условия

,

, j=1, 2, 3 - индексы для каждой компоненты цвета, а именно, красный, зеленый, синий,

,

.

- color addition functions for a standard colorimetric observer (spectral sensitivity of the eye), adopted by the international lighting commission, the tabular values of which can be found in Color Physics for Industry, Second Edition, Edited by Roderick McDonald, 1997 [7]. k = 1, 2, 3 are the indices for each color component (red, green, blue), d _j (λ) is the spectral energy distribution of one of the standard colorimetric radiation, the tabular values of which can be found in [7]. j = 1, 2, 3 - indices for each color component (red, green, blue); V (x) = [V ₁₁ (x) V _l2 (x) V _l3 (x) V _r1 (x) V _r2 (x) V _r3 (x)] ^T , N is the normalizing matrix necessary to satisfy the condition

,

, j = 1, 2, 3 - indices for each color component, namely, red, green, blue,

,

.

Thus, the invention includes a method for synthesizing anaglyph images adapted for specific stereo glasses and printing them while maintaining three-dimensional perception for these glasses. The method may be implemented in a color printing device or in software of printing devices.

It should be noted that the above embodiment of the invention is set forth for purposes of illustration only and it is clear to those skilled in the art that various modifications, additions and substitutions are possible without departing from the scope and meaning of the present invention disclosed in the attached claims.

Claims (7)

1. A method of creating and printing color anaglyph images, comprising the following operations: print a test color sample on a printing device for printing anaglyph images; evaluate the spectral transmission functions of the left and right filter glasses for the printed test sample; form a stereo pair; generate an anaglyph image for a stereo pair taking into account the functions of spectral transmission; print anaglyph image. 2. The method according to claim 1, characterized in that they print a test color sample on a printing device designed to print anaglyph images, by performing the following steps: print a series of colors with different purity / saturation and color tone; print user instructions. 3. The method according to claim 1, characterized in that the spectral transmission functions of the left and right glasses filters are evaluated by performing the following steps: approximately the spectral transmission functions for the left glasses filter are estimated by the values corresponding to the user's choice of visible and invisible colors on a test color sample with viewing through the left filter; Estimating approximately the spectral transmittance function for the right glasses filter based on the values corresponding to the user's choice of visible and invisible colors on the test color sample when viewed through the right filter; approximate approximate estimates of the spectral transmission functions by the nearest functions of real filters from a given set. 4. The method according to claim 1, characterized in that they form a stereo pair by performing the following steps: align the right and left images of the stereo pair geometrically; adjust the colors of the right and left images of the stereo pair; blurring at least one of the two color channels of the right and left images of the stereo pair; evaluate the disparity map; modifying the disparity map in order to reduce the disparity values exceeding the threshold; change the right and left images of the stereo pair in accordance with the modified disparity map. 5. The method according to claim 1, characterized in that they form a stereo pair by performing the following steps: align the right and left images of the stereo pair geometrically; adjust the colors of the right and left images of the stereo pair; blurring at least one of the two color channels of the right and left images of the stereo pair; evaluate a depth map; modifying the depth map in order to reduce the depth range of the scene if it exceeds a predetermined threshold; change the right and left images of the stereo pair in accordance with the modified depth map. 6. The method according to claim 1, characterized in that the blur parameters of at least one of the two color channels of the right and left stereopair images and modification parameters of the disparity map and depth map depend on the size of the printed anaglyph image on paper. 7. The method according to claim 1, characterized in that an anaglyph image for a stereo pair is generated taking into account the spectral transmission functions of the stereo glasses filters using the following expression:

where x is the position of the pixel in the image, x∈A × B; A, B - image size,

F_one(λ) and F_r(λ) are the spectral transmission functions of real filters for the left and right filters, λ is the wavelength; W - matrix of weighting factors of size 6 × 6,

- color addition functions for a standard colorimetric observer (spectral sensitivity of the eye), k = 1, 2, 3 - indices for each color component (red, green, blue), d_j(λ) is the spectral energy distribution of one of the standard colorimetric radiation, j = 1, 2, 3 are the indices for each color component (red, green, blue); V (x) = [V_eleven(x) V₁₂(x) V₁₃(x) V_r1(x) V_r2(x) V_R3(x)]^T, N is the normalizing matrix necessary to satisfy the condition

 j = 1, 2, 3 - indices for each color component, namely, red, green, blue,

,

.

Images