JP5959911B2 - Quantization apparatus and control method thereof - Google Patents

Description

The present invention relates to a technique for quantizing the scalar quantity such as your Keru pixel values in the image.

  Conventionally, a quantization technique is known that reduces the number of bits required for recording a scalar quantity by approximating a given scalar quantity with any of several arbitrarily determined representative values. Of course, as the number of representative values increases, the approximation error during quantization decreases, but the number of bits necessary to record one scalar quantity increases accordingly. Even if the number of representative values is the same, an error generated by quantization varies depending on what value is selected for each representative value. Therefore, what value is set as the representative value is a key for accurately quantizing the data. At the time of quantization, the correspondence relationship of which representative value corresponds to each value in the low bit representation is often recorded as a table, and this table is called a representative value table.

  When quantization is performed, a common representative value table is generally used for a plurality of scalar quantities. For example, in the case of a gray image, the luminance value of each pixel is one scalar quantity, and the luminance value of all the pixels of the image is generally quantized using a common representative value table. Therefore, the representative value table is generally determined so that deterioration due to quantization is reduced for a plurality of scalar quantities.

  A well-known technique for determining a representative value table is Lloyd-Max quantization (Patent Document 1). In Lloyd-Max quantization, the sum of squared errors E of each scalar quantity and a representative value approximating the scalar quantity E:

を最小にすることを目的にして代表値を定める。ここでｓ_i（ｉ＝１，…，Ｋ）は量子化対象のスカラー量で、ｓ_i'は量子化の際に、ｓ_iを近似するために割り当てられる代表値である。

A representative value is determined for the purpose of minimizing. Here, s _i (i = 1,..., K) is a scalar quantity to be quantized, and s _i ′ is a representative value assigned to approximate s _i during quantization.

JP-A-2005-341555

  As described above, when the number of representative values to be assigned at the time of quantization is determined, deterioration due to quantization is determined by the representative value table. As a criterion for degradation, as in the Lloyd-Max method, it is common to use a difference between a scalar quantity before quantization and a representative value approximating the scalar quantity.

  However, on the other hand, there are cases in which it is not appropriate to evaluate degradation based on such criteria. For example, when the luminance value of each pixel of an image is quantized, an allowable area including an important object in the image has a small allowable error, and other areas have a large allowable error.

  The present invention has been made in consideration of the above-described problems of the conventional example, and calculates an allowable range of representative values to be assigned to each scalar quantity to be quantized, and performs quantization in consideration of the allowable range. It is intended to provide a technique for determining a plurality of representative values.

In order to solve this problem, for example, the quantization apparatus of the present invention has the following configuration. That is,
N scalar quantities S (1) to S (N) to be quantized, each having a value which can be taken by each scalar quantity, are input, and each scalar quantity is one of a plurality of representative values. Representing a quantizer that quantizes,
Obtaining means for obtaining information for specifying an allowable range of change when quantized for each scalar quantity S (n) (n is any one of 1,..., N);
Setting means for setting an allowable range according to the information for each scalar quantity to be quantized;
An input means for inputting a scalar quantity to be quantized;
The interest was entered with the input means scalar S (n), when the allowable range for the set pre-Symbol focused scalar S (n) by the setting unit expressed as Tmin (n) ~Tmax (n) , wherein It is assumed that a plurality of scalar amounts in the range represented by Tmin (n) to Tmax (n) are input with respect to the target scalar amount S (n), and the number of input values of T1 to TI is the number of input. A counting means for cumulatively counting;
Determining means for determining the plurality of representative values according to the count values of T1 to TI obtained by the counting means when the N scalar amounts are input by the input means;
Quantization means for quantizing the N scalar quantities using a plurality of representative values obtained by the determining means.

  According to the present invention, the allowable range of the assigned representative value is calculated for each scalar quantity to be quantized, and a plurality of representative values for quantization are determined in consideration of the allowable range. As a result, it is possible to reduce the deterioration due to the quantization by performing the quantization reflecting the error characteristic allowed for each scalar quantity.

The flowchart which shows the processing content of 1st Embodiment. The flowchart which shows the detail of the calculation method of S102 of FIG. The flowchart which shows the detail of S107 of FIG. The figure which shows the example of a quantization object image. The figure which shows the example of the tolerance | permissible_range and count value calculated | required in 1st Embodiment. The figure which shows the structural example of the system which can reproduce | regenerate the free viewpoint image of 2nd Embodiment. The figure which shows the positional relationship of the camera which image | photographs a multi-viewpoint image, and a to-be-photographed object. The figure which shows the example of the color image image | photographed from multiple viewpoints. The figure which shows the parallax image used as quantization object. The figure explaining the relationship between the parallax tolerance | permissible_range and a color image. The figure which shows the example of the tolerance | permissible_range calculated | required by 2nd Embodiment. The flowchart which shows the detail of S107 of FIG. 1 in the modification of 2nd Embodiment. The flowchart which shows the detail of S102 of FIG. 1 in 2nd Embodiment. 14 is a flowchart showing details of S1302 in FIG. The flowchart in the case of automatically determining the number of representative values. The block block diagram of a computer.

  Hereinafter, embodiments according to the present invention will be described in detail with reference to the accompanying drawings. Each embodiment described below shows an example in the case where the present invention is specifically implemented, and is one of the specific embodiments having the configurations described in the claims.

[First Embodiment]
A block diagram of an information processing apparatus that functions as a quantizing apparatus according to the first embodiment shown in FIG. 16.

  The CPU 1601 controls the entire computer using computer programs and data stored in the RAM 1602 and the ROM 1603 and executes the above-described processes described as being performed by the multi-viewpoint image encoding apparatus. The RAM 1602 is an example of a computer-readable storage medium. The RAM 1602 has an area for temporarily storing computer programs and data loaded from the external storage device 1607, the storage medium drive 1608, and the network interface 1610. Further, the RAM 1602 has a work area used when the CPU 1601 executes various processes. That is, the RAM 1602 can provide various areas as appropriate. The ROM 1603 is an example of a computer-readable storage medium, and stores computer setting data, a boot program, and the like. The keyboard 1604 and the mouse 1605 can be operated by a computer operator to input various instructions to the CPU 1601. The display device 1606 is configured by a CRT, a liquid crystal screen, or the like, and can display a processing result by the CPU 1601 with an image, text, or the like. For example, the gray image to be quantized can be displayed, and the quantized result can be displayed.

  The external storage device 1607 is an example of a computer-readable storage medium, and is a large-capacity information storage device represented by a hard disk drive device. The external storage device 1607 stores an OS (Operating System), computer programs and data for causing the CPU 1601 to perform the processes shown in FIG. 1, the above-described various tables, databases, and the like. Computer programs and data stored in the external storage device 1607 are appropriately loaded into the RAM 1602 under the control of the CPU 1601 and are processed by the CPU 1601.

  The storage medium drive 1608 reads a computer program and data recorded on a storage medium such as a CD-ROM or DVD-ROM, and outputs the read computer program or data to the external storage device 1607 or the RAM 1602. Note that part or all of the information described as being stored in the external storage device 1607 may be recorded on this storage medium and read by this storage medium drive 1608.

  An I / F 1609 is an interface for inputting a gray image to be quantized from the outside, and is USB (Universal Serial Bus) as an example. A bus 1610 connects the above-described units.

  In the above configuration, when the computer is turned on, the CPU 1601 loads the OS from the external storage device 1607 to the RAM 1602 according to the boot program stored in the ROM 1603. As a result, an information input operation can be performed via the keyboard 1604 and the mouse 1605, and a GUI can be displayed on the display device 1606. When the user operates the keyboard 1604 or mouse 1605 and inputs an instruction to start an application program for gray image quantization stored in the external storage device 1607, the CPU 1601 loads the program into the RAM 1602 and executes it. Thus, the computer functions as an image processing apparatus that quantizes the image.

  Hereinafter, an example in which image data to be quantized is already stored in the external storage device 1607 and the quantized data is also stored in the external storage device 1607 as a file will be described. However, the image data to be quantized may be downloaded from a network or read by an image scanner or the like. Further, the quantized data may not be stored as a file in the storage device but may be transmitted over the network. That is, the input source of the image data to be quantized and the type of output destination of the image data after quantization do not limit the present invention.

  FIG. 1 is a flowchart illustrating a procedure executed by the CPU 1601 when functioning as an image processing apparatus that performs quantization of image data in a predetermined format. Hereinafter, the quantization processing according to the first embodiment will be described with reference to FIG.

  A quantization method capable of suppressing an error occurring in a region of interest (region of interest) will be described by taking as an example the case where the luminance value of each pixel of the gray image shown in FIG. The number of representative values is an important parameter for quantization, but in the present embodiment, the number of representative values is selected from three representative values for simplicity. Note that each pixel of the gray image in FIG. 4A is represented by 8 bits, and its luminance value is represented by 256 gradations from 0 to 255. A halftone dot region 401 in FIG. 4A is a region in which two colors are mixed, and pixels having a luminance value “249” and a luminance value “245” are mixed. The horizontal line area 402 in FIG. 5 is a monochrome area, and all the pixels have a luminance value “9”. Similarly, the hatched area 403 is a single color area in which all pixels have a luminance value of “5”.

  First, in S101 of FIG. 1, the CPU 1601 sets a scalar amount to be quantized. In this step, a plurality of scalar quantities are selected, and the selected scalar quantities are quantized using a common representative value table by subsequent processing. For example, when one gray image is quantized using a common representative value table, the luminance values of all the pixels of the gray image are scalar quantities to be quantized. In this embodiment, in order to simplify the description, not the entire image shown in FIG. 4A but the luminance values of the pixels of one line indicated by numbers 1 to 15 shown in FIG. And

  In S102, an allowable range of representative values assigned to the given quantization target scalar quantity is calculated. Here, the definition of the allowable range in the present embodiment will be described. The allowable range is a range that can be allowed as a change in each scalar amount accompanying quantization, and is a range that can be regarded as sufficiently small if it can be approximated by a value within the allowable range.

  In the present embodiment, in S102, when the luminance value of each pixel is quantized, an allowable range is determined depending on whether or not the pixel is in a region of interest specified by the user. In another embodiment, the allowable range may be given directly from the outside of the apparatus together with the scalar amount to be quantized. Details of S102 in the present embodiment are shown in FIG.

  In S201 of FIG. 2, the attention area is designated. A region of interest is particularly important in an image, and is a region where image quality degradation due to quantization is to be suppressed. In the present embodiment, it is assumed that the attention area is specified by an input from the user. For example, the image to be quantized is displayed on a display or the like, and is specified by surrounding a part of the image with a touch panel or a mouse. In the following, it is assumed that a rectangular area including the areas 402 and 403 in FIG. In S202, one pixel of the quantization target image is selected. In S203, it is determined whether or not the pixel selected in S202 is within the attention area specified in S201. As a result of the determination in S203, if the pixel is in the region of interest (within the region of interest), in S204, the allowable range of the luminance value of the pixel is set as the luminance value ± Din. Conversely, if the pixel is outside the region of interest (outside the region of interest), in S205, the allowable range of the luminance value of the pixel is set as the luminance value ± Dout. As a matter of course, it is desirable that Dout> Din ≧ 0. In the present embodiment, it is assumed that Dout is set to 2 and Din is set to 1. In step S206, it is determined whether an allowable range is set for all pixels. If there is a pixel for which the allowable range is not set, the allowable range is set for all the pixels by repeating the process of selecting the pixel and setting the allowable range in S202.

  Returning to the flowchart of FIG. In S103, the count value of each luminance value is obtained using the allowable range obtained in S102. The method for obtaining the count value will be described in detail below.

  First, the graph shown in FIG. 5A is obtained from the allowable range of each pixel obtained in S102. In FIG. 5A, the horizontal axis is a scalar quantity (luminance value in this embodiment), and the vertical axis is a number (pixel number in this embodiment) assigned to the scalar quantity to be quantized. Hereinafter, when the pixel number is i, the pixel value is expressed as pixel P (i).

  A hatched line 501 in FIG. 5A indicates that the luminance value of the pixel P (10) is “9”. White bands on the left and right of the oblique line 501 indicate the allowable range of the pixel P (10). The pixel P (10) is a pixel selected in the attention area in S201 of FIG. 2, and the allowable range is set to 9 ± 1, that is, 8 to 10 in S204, and the white band extends over the range. A hatched line 502 represents the luminance value of the pixel P (4), and the white band around it indicates the allowable range of the pixel P (4). Since the pixel P (4) is a pixel that has not been selected as a region of interest, the allowable range is 249 ± 2, that is, the range of 247 to 251. In FIG. 5 (a), the luminance values of the pixels are indicated by diagonal lines similarly to the other pixels, and the allowable range of the pixels is indicated by white bands including the diagonal lines.

  The count value represents the number of pixels including each luminance value in the allowable range, and is obtained by summing in the vertical direction in the graph of FIG. FIG. 5B shows the count value for each luminance value derived from FIG. For example, in FIG. 5A, the pixels including the luminance value 5 within the allowable range are the pixels P (7), P (8), and P (9). Therefore, in FIG. The count value is “3”.

  As another embodiment for obtaining the count value in S103, weighting may be performed when taking the sum in the vertical direction in the graph of FIG. For example, “2” may be added to the hatched area in FIG. 5A and “1” may be added to the white band area to obtain the count value for each luminance. In this case, the shape of the histogram indicating the count value for each luminance in FIG. 5B changes, but otherwise, the same processing as in this embodiment may be performed.

  In the next S104, one representative value is determined. As the representative value, the luminance value having the highest count value is selected. In this embodiment, as is clear from FIG. 5B, “247” is selected as the first representative value.

  In the next S105, the count target is reduced. This is because when determining the next representative value, pixels having a representative value within the allowable range are not considered when recalculating the count value in S103. In this embodiment, since “247” has already been selected as the representative value, the pixels P (1) to P (5) and P (11) to P (15) that include “247” in the allowable range are included. Reduce from counting.

  In next step S106, it is determined whether the representative value has reached the target number or the pixel to be counted has been eliminated. In the present embodiment, as described above, it is necessary to select three representative values, and since only one is selected and the pixels to be counted still remain, this condition is not satisfied, and S106 The determination is false (False), and the process returns to S103 again.

  The process of selecting the second and subsequent representative values may be basically performed in the same manner as when the first representative value is determined in steps S103 to S105. As described above, the difference after the second week is that when the count value is calculated in S103, the pixels reduced from the count target in S105 are ignored. For example, in this embodiment, when obtaining the second representative value, the count value obtained in S103 is as shown in FIG. FIG. 5C shows a count value for only the pixels P (6) to P (10) that cannot be approximated by an error within the allowable range by the first representative value 247 in FIG. 5A. It is what I have sought. The luminance value that maximizes the count value may be determined as a representative value in S104. In FIG. 5C, there are three luminance values 4, 5, and 6 that maximize the count value. In this way, a supplementary description will be made of the processing in the case where there are a plurality of candidates for the luminance value with the maximum count value, that is, the representative value. For example, for each representative value candidate, the sum of squares of the difference between the representative value candidate and the luminance value before quantization of the pixel that includes the representative value candidate within the allowable range is calculated. A method of selecting the smallest value as the representative value is conceivable. In the case of this embodiment, the luminance values 4, 5, and 6 that are candidates for representative values are all included in the allowable range of the pixels 7, 8, and 9, and the luminance values before the quantum of the pixels 7, 8, and 9 The luminance value 5 that minimizes the sum of squares of the difference from 5 is selected as the second representative value.

  When the above processing is repeated, representative values are selected in the order of “247”, “5”, and “9”. Then, the determination in S106 is determined to be true. In this case, the process proceeds to S107, and quantization is performed by replacing the luminance value of each pixel with a representative value. Details of S107 in the present embodiment are shown in FIG.

  In S301, it is determined whether each pixel has a representative value within an allowable range. Naturally, the allowable range is obtained in S102. If there is a representative value within the allowable range, the process proceeds to S302; otherwise, the process proceeds to S303.

  In S302, a representative value within the allowable range is assigned. At this time, if there are a plurality of representative values within the allowable range, the representative value closest to the luminance value before quantization is assigned. In S303, since there is no representative value in the allowable range, a representative value closest to the luminance value before quantization is assigned among the representative values outside the allowable range.

  In the quantization process of FIG. 3, when reducing the amount of image data of the quantization result, the representative values “5”, “9”, and “247” are 00, 01, 10 in 2 bits. And a file in which the pixel value to be quantized (8 bits) is replaced with 2-bit data representing the closest representative value is generated. In this case, information indicating that pixel values 5, 9, 247 are assigned to 00, 01, 10 may be stored in the file header.

  The quantization method in the present invention has been described above based on the flowchart of FIG. In the above embodiment, the scalar quantity to be quantized is 15 pixel values as shown in FIG. 4B and the allowable range is ± 1, ± 2, but this is only an example. Please note that.

  As described in the description of the flowchart of FIG. 1, in S103 and S105, when obtaining the second and subsequent representative values, the count target is reduced in S105, and the count value is recalculated in S103. The value was recalculated. However, it is not always necessary to include in the quantizer until the count value is recalculated. For example, when the first representative value is determined in S104, the count value of the surrounding ± 1 pixel range can be overwritten with 0, and the second representative value can be determined by the count value. Here, the reason why the count value of the surrounding ± 1 pixel is set to 0 is derived from the fact that the permissible range of the region of interest to be quantized with the highest accuracy in this embodiment is ± 1. As described above, even if the count value recalculation method for determining the second and subsequent representative values is changed, it is possible to use the representative value determination method using the count value, which is a feature of the present invention. is there.

Therefore, the quantization apparatus when the technical idea in the present embodiment is generalized has the following configuration. That is,
N scalar quantities S (1) to S (N) to be quantized, each having a value which can be taken by each scalar quantity, are input, and each scalar quantity is one of a plurality of representative values. Representing a quantizer that quantizes,
Obtaining means for obtaining information for specifying an allowable range of change when quantized for each scalar quantity S (n) (n is any one of 1,..., N);
An input means for inputting a scalar quantity to be quantized;
When the allowable range for the target scalar amount S (n) indicated by the information acquired by the acquisition unit is expressed as Tmin (n) to Tmax (n) for the target scalar amount S (n) input by the input unit, It is assumed that a plurality of scalar amounts in the range represented by Tmin (n) to Tmax (n) are input with respect to the target scalar amount S (n), and the number of input values of T1 to TI. Counting means for cumulatively counting,
Determining means for determining the plurality of representative values according to the count values of T1 to TI obtained by the counting means when the N scalar amounts are input by the input means;
What is necessary is just to set it as the structure which has the quantization means which quantizes said N scalar quantity using the some representative value obtained by this determination means.

[Second Embodiment]
In the second embodiment, an example of quantizing a distance image used for free viewpoint image synthesis will be described.

  First, an outline of the free viewpoint image synthesis and the distance image will be described. Free viewpoint image composition is to shoot an image from a certain viewpoint using a camera or other imaging device, and synthesize an image that can be seen from another viewpoint that is not actually photographed. Technology. A method using a distance image is known as one of free viewpoint image synthesis methods. A distance image is an image in which distance information of each pixel of a photographed viewpoint is stored. If there is a distance image, the coordinates of the photographed subject in the three-dimensional space can be specified, so it can be estimated where the subject appears in the image when viewed from another viewpoint.

  There are an active method and a passive method for obtaining a distance image. The active method is a method of acquiring a distance by irradiating a subject with a laser or the like. As a representative method, there is a time-of-flight method of estimating the distance from the time until the laser reciprocates. The passive method is a method of acquiring a distance using a photographed image, and a typical example is a stereo matching method using photographed images from two different viewpoints.

  When free viewpoint image composition is performed using an actively acquired distance image, it is necessary to encode an actively acquired distance image in addition to the captured multi-viewpoint image. When the distance image is passively acquired, it is not necessary to encode the distance image because the distance image can be acquired from the multi-viewpoint image. However, for the purpose of reducing the processing load at the time of reproduction or the like, if the distance image is passively acquired and encoded at the time of shooting and the distance image is not re-acquired at the time of reproduction, the distance image needs to be encoded.

A distance image used for free viewpoint image composition is often encoded in the form of a parallax image. The parallax is an amount indicating where a point reflected in one viewpoint is captured in a captured image at the other viewpoint when the same subject (point) is captured from two different viewpoints. In general, it is expressed by how many pixels the points (hereinafter referred to as corresponding points) in which the same subject appears in different images are shifted. More specifically, the two optical cameras are parallel to each other and arranged at an interval of b (mm) on the left and right along the horizontal axis, from the photographing surface (a plane perpendicular to the optical axis and passing through the two cameras). When a point separated by z (mm) is photographed, the parallax d (pix) of the corresponding points in the two photographed images is
d = bfW / (zC) (Formula 1)
It is. Here, f is the focal length (mm) of the camera, W is the width (pix) of the captured image, and C is the width (mm) of the imaging element. When the cameras are accurately arranged on the left and right, it can be ensured that the vertical parallax of the corresponding points is 0 (epipolar constraint). Therefore, the parallax described here indicates a horizontal shift between corresponding points.

  This completes the description of the free viewpoint image composition and the outline of the distance image. Hereinafter, a parallax image quantization method will be described on the assumption that a distance image used for free viewpoint image synthesis is encoded as a parallax image.

  Here, it supplements about the property of a parallax image. When approximating the parallax, the accuracy near the edge of the color image taken from the same viewpoint as the parallax image is particularly important. As already described, the parallax image is information indicating how much the corresponding pixels between different images are shifted. When performing free viewpoint image composition using parallax, pixels of the captured viewpoint image are shifted (warped) in accordance with the position of the viewpoint where the image is to be synthesized and the magnitude of the parallax. Therefore, the change in parallax means that in free viewpoint image synthesis, a certain pixel is warped to a position different from the original warp destination. In a region where the color change is slight, even if the warp is made to a position shifted from the original position, the apparent deterioration is small. On the other hand, if the warp destination is wrong in the vicinity of the edge of the color image, the color of a certain pixel on the free viewpoint image becomes significantly different from the color that should originally be, and thus deterioration is conspicuous. Therefore, when encoding a parallax image, higher accuracy is required near the edge of the color image.

  The system assumed in the second embodiment is shown in FIG. FIG. 6 shows the configuration of each part from the shooting of an image by the shooting apparatus to the display of the free viewpoint image synthesis result using the shot image, and the data flow.

  First, a color image is photographed by the photographing units 601, 602, and 603. Photographing is performed with three digital cameras having parallel optical axes and arranged on the left, center, and right. The left viewpoint image is indicated by I1, the central viewpoint image is indicated by I2, and the right viewpoint image is indicated by I3.

  Next, the parallax image generation unit 604 generates a parallax image S2 of the image I2. The parallax image is obtained by the stereo matching method using I1, I2, and I3. Here, it supplements about the stereo matching method using three images. As already described, the stereo matching method is basically a method for obtaining parallax using two images. However, there is known a method for obtaining the I2 parallax more accurately by performing stereo matching with combinations of I2 and I1, and I2 and I3, and finally integrating the two results. Is assumed to be used. Of course, the distance to the subject may be measured by the active method, and the parallax may be obtained according to (Equation 1).

  Next, the parallax image quantization unit 605 quantizes the parallax image. A feature in the present embodiment is a quantization process in the parallax image quantization unit 605, which will be described in detail later. As a result of the quantization, a parallax image S2 'obtained by quantizing S2 into a low bit representation and a representative value table T2 are obtained.

  Next, the encoding units C1, C2, and C3 encode I2, S2 ', and T2, respectively. For encoding, lossless compression such as ZIP may be used. These encoded data are transmitted from the transmission unit 606. In the apparatus that generates the free viewpoint image, the reception unit 607 receives the encoded data, and the decoding units D1, D2, and D3 decode I2, S2 ', and T2, respectively. Next, the inverse quantization unit 608 obtains a parallax image S2 ″ obtained by inversely quantizing S2 ′.

  Finally, the free viewpoint image composition unit 609 performs free viewpoint image composition using the color image I2 and the parallax image S2 '' to reproduce the free viewpoint image I4. In the second embodiment, it is assumed that the free viewpoint image I4 is an image of a viewpoint arranged on any one of the straight lines connecting the camera capturing the right viewpoint and the camera capturing the left viewpoint. Therefore, the parallax image quantization unit 605 also performs quantization so as to reduce the influence of image quality deterioration due to quantization of the parallax image when synthesizing an image viewed from a viewpoint within the range.

  Details of the parallax image quantization unit 605, which is a feature of the second embodiment, will be described below.

  First, the color images I1, I2, and I3 and the parallax image S2 that are inputs to the parallax image quantization unit 605 illustrated in FIG. 6 will be described. In this embodiment, the color images I1, I2, and I3 are taken by a digital camera. FIG. 7 shows the positional relationship between the subject and the camera in the color image input in this embodiment. The cameras 701 to 703 correspond to the imaging units 601 602 603 in FIG. 6, and generate color images I1, I2, and I3, respectively. The cameras 701 to 703 are arranged on the photographing surface 708, and the optical axes of the cameras are all perpendicular to the photographing surface 708. The planes 704 to 707 are photographed with these three cameras. These four planes are all parallel to the imaging plane 708, and the distance from the imaging plane 708 is converted to parallax according to (Equation 1), the plane 704 is parallax 6, the planes 705 and 706 are parallax 4, and the plane 707 is parallax 1. It is. The planes 704 to 707 are monochromatic planes, and the planes 704 to 706 are black and the plane 707 is white. The color images I1, I2, and I3 photographed under the conditions described with reference to FIG. 7 are shown in FIGS. 8A, 8B, and 8C, respectively. Naturally, planes 704, 705, 706, and 707 are shown in FIGS. 8A, 8B, and 8C, and the corresponding portions of the images are shown in FIG. The black area in FIG. 8 is made up of planes 704, 705, and 706, all of which are black and the same color. In FIG. There is no white dotted line in the color image. In addition, the white area in FIG. FIG. 9 shows the parallax image S2 of FIG. Naturally, the hatched area 901, the horizontal line area 902, the horizontal line area 903, and the white area 904 correspond to the planes 704, 705, 706, and 707, respectively, and the parallaxes are 6, 4, 4, and 1, respectively. As described above, the parallax image S2 represents which point of the corresponding point of each pixel on I2 is I1 or I3 by how many pixels are shifted to the left and right from the position of the pixel on I2. is there. It is inevitably determined from the camera arrangement that the corresponding point on I1 is always on the right side of the corresponding point on I2, and the corresponding point on I3 is always on the left side of the corresponding point on I2.

  Next, details of the processing of the parallax image quantization unit 605 will be described. The basic processing flow is the same as that shown in FIG. The main difference from the first embodiment is the allowable range calculation method in S102. In the following, with reference to FIG. 1, the description will focus on S102 and focus on the differences from the first embodiment.

  First, in S101, a parallax image S2 is set as a quantization target scalar quantity. In the present embodiment, for simplicity of explanation, as shown in FIG. 9, the parallax values of one line d1 to d22 are set as a scalar quantity to be quantized.

  Next, in S102, the allowable range is calculated. A feature of the present embodiment is that color images I1, I2, and I3 are used for calculation of the allowable range of the parallax image S2. Details of the calculation of the allowable range in S102 are shown in FIG. In S1301 of FIG. 13, one of the parallax images S2 is selected from the parallaxes d1 to d22 to be quantized. Here, it is assumed that di (i = 1,..., 22) is selected. In S1302, the allowable range of the parallax di is calculated using I1 and I2. Details of this processing will be described later with reference to FIG. Similarly, in S1303, the allowable range is calculated using I3 and I2. In step S1304, the allowable ranges calculated in steps S1302 and S1303 are integrated. At the time of integration, only the range included in both allowable ranges is set as the allowable range of di. In step S1305, it is determined whether all allowable ranges of d1 to d22 have been calculated. If there is a pixel for which the allowable range has not been calculated, the process returns to S1301, and the process is repeated until the allowable range of all pixels is obtained.

  Here, a calculation method of the allowable range using I1 and I2 in S1302 will be described with reference to FIG. As described above, the purpose of S1302 is to obtain an allowable range of the parallax di. Therefore, in S1302, dmax and dmin are obtained by setting dmax as the maximum value of the allowable range of di and dmin as the minimum value. Hereinafter, a process for obtaining the allowable range of di will be described. In the following, ci is a pixel on the color image I2, and is a pixel at the same position as the parallax di.

  First, in S1401 of FIG. 14, dmax and dmin are initialized with di. In S1402, the corresponding point on ci I1 is set to cref. Of course, cref is the point shifted to the right by the parallax di from the position of the pixel ci on I1. Next, in S1403, cref is shifted by 1 to the right. In step S1404, it is determined whether the color difference between the pixel ci and the pixel cref is equal to or less than a threshold value. For example, in this embodiment, each RGB color is represented by 256 gradations, and it is determined whether or not the sum of absolute values of differences between the colors is 20 or less. If the color difference is equal to or smaller than the threshold value in S1404, dmax is updated to the disparity of cref and ci in S1405, and S1403 to S1404 are repeated. When S1402 to S1406 are completed, a range having a pixel having a color close to ci is obtained on the right side of the corresponding point of ci on I1, and the parallax between the leftmost pixel and ci in the range is obtained as dmax. . Similarly, by repeating S1406 to S1408, a range where there is a pixel having a color close to ci on the left side of the corresponding point of ci on I1 is obtained, and the parallax between the leftmost pixel in the range and ci Is obtained as dmin. In the last S1410, dmax and dmin are output as allowable ranges using I1 and I2 in S1302 of FIG. As a specific example, FIG. 10 shows the allowable range of d7 parallax calculated using I1 and I2. The left side of FIG. 10 is the color image I1, and the right side is the color image I2. The corresponding point 1001 of I1 is a corresponding point determined by the parallax d7 of the pixel c7 of I2. The parallax between the pixel at the tip of the white arrow extending to the right from the corresponding point 1001 and c7 is dmax. The parallax between the pixel at the tip of the white arrow extending to the left from the corresponding point 1001 and c7 is dmin.

  In S1303, similar to S1302, the allowable range is calculated using I3 and I2. In step S1303, basically the same procedure as described with reference to FIG. 14 is performed. However, the process of shifting cref to the right in S1403 needs to be replaced with the process of shifting to the left, and the process of shifting cref to the left in S1408 needs to be replaced with a process of shifting to the right.

  This is the end of the description of the calculation of the allowable range in S102 in the second embodiment. When S102 ends, the allowable range of d1 to d22 is obtained as shown in FIG. Here, the role of the allowable range obtained in the second embodiment will be supplemented with reference to FIG. In FIG. 11, d6 and d17 have a narrow allowable range. This reflects that the pixels c6 and c17 corresponding to d6 and d17 are near the edge of the color image on the color image I2 shown in FIG. 8B. Conversely, when there are many similar colors around pixels on the corresponding color image, such as d11 and d12, the allowable range is widened. Thus, it can be seen that the allowable range obtained in the second embodiment reflects the characteristic of the parallax image that higher accuracy is required near the edge of the color image.

  Hereinafter, S103 to S107 may be performed in the same procedure as in the first embodiment. When all the processes of the second embodiment are finished, “1” and “4” are selected as representative values of the parallaxes d1 to d22 in FIG.

[Modification of Second Embodiment]
As a modification of the second embodiment, a method suitable for quantizing a parallax image in the quantization based on the representative value in S107 of FIG. 1 will be described. The processing in this modification is the same as that of the second embodiment except for S107 in FIG.

  Details of S107 in this modification will be described with reference to FIG. In S107, the processing shown in FIG. 12 is performed for all the parallaxes di (i = 1,..., 22) to be quantized.

  First, in S1201, it is determined whether there is a representative value within the allowable range of di. If there is a representative value, the process proceeds to S1202, and if not, the process proceeds to S1203.

  In S1202, all representative values within the allowable range are selected as representative value candidates to be assigned to di. In S1203, since there is no representative value within the allowable range, one representative value closest to di is selected from among representative values larger than di, and one representative value closest to di is selected from among representative values smaller than di. Is selected as a representative candidate.

In S1204, one is selected from the representative value candidates selected in S1202 or S1203. In this case, how to choose the representative value:
Er = diff (ci, cr1) + diff (ci, cr3)
Select according to the priority of minimizing. Here, ci is a pixel on I2, and is a point at the same coordinate as the parallax di. Cr1 and cr3 are pixels on I1 and I3, respectively, and corresponding to ci when the representative value r is regarded as parallax instead of di. diff represents the color difference between two pixels, and for example, the sum of absolute values of differences between RGB components is used.

  The method of selecting the representative value described in the modification of the second embodiment is that the color difference between pixels associated with the parallax is more important than the change in the value of the parallax when the parallax is quantized. It is a method that reflects the characteristics of.

[Third Embodiment]
In the second embodiment, an example will be described in which the reference for stopping the process of selecting the representative value is changed in S106 of FIG. In S106, the determination is made based on whether the representative value has reached a predetermined target number or the scalar quantity to be counted has been exhausted. In order to make this determination, it is necessary to determine the target number of representative values in advance. However, the third embodiment shows an example in which switching is performed according to the content of an image without determining the target number of representative values.

  A flowchart of the present embodiment is shown in FIG. The features of this embodiment are S1501 and S1502, and the other parts are the same as those described in the second embodiment with reference to FIG. Therefore, only S1501 and S1502 will be described.

  In S1501, it is determined whether the count value of the representative value determined in S104 is X% or less of the total number of pixels. Here, the count value obtained in S103 is used. X is a threshold value indicating an arbitrarily determined ratio, and for example, X = 1 or the like may be set. If the count value is less than or equal to X% of the total number of pixels, the process proceeds to S1502, and only the last selected representative value is excluded from the representative value.

  When the number of representative values is determined using the method described in the third embodiment, the number of representative values can be assigned according to the content of the image. That is, after quantization, the number of bits necessary to express the parallax of each pixel can be automatically switched according to the content of the image.

(Other examples)
The present invention can also be realized by executing the following processing. That is, software (program) that realizes the functions of the above-described embodiments is supplied to a system or apparatus via a network or various storage media, and a computer (or CPU, MPU, or the like) of the system or apparatus reads the program. It is a process to be executed.

Claims (12)

N scalar quantities S (1) to S (N) to be quantized, each having a value which can be taken by each scalar quantity, are input, and each scalar quantity is one of a plurality of representative values. Representing a quantizer that quantizes,
Obtaining means for obtaining information for specifying an allowable range of change when quantized for each scalar quantity S (n) (n is any one of 1,..., N);
Setting means for setting an allowable range according to the information for each scalar quantity to be quantized;
An input means for inputting a scalar quantity to be quantized;
The interest was entered with the input means scalar S (n), when the allowable range for the set pre-Symbol focused scalar S (n) by the setting unit expressed as Tmin (n) ~Tmax (n) , wherein It is assumed that a plurality of scalar amounts in the range represented by Tmin (n) to Tmax (n) are input with respect to the target scalar amount S (n), and the number of input values of T1 to TI is the number of input. A counting means for cumulatively counting;
Determining means for determining the plurality of representative values according to the count values of T1 to TI obtained by the counting means when the N scalar amounts are input by the input means;
And a quantization unit that quantizes the N scalar quantities using a plurality of representative values obtained by the determination unit. The scalar quantity to be quantized is a distance image of a color image captured from a predetermined viewpoint, and the acquisition unit acquires a plurality of color images captured from different viewpoints,
The quantization apparatus according to claim 1, wherein the setting unit calculates an allowable range of each distance using the color image. The quantization apparatus according to claim 2, wherein the information is information specifying whether or not there is a pixel having a small color difference in the vicinity of the target pixel for the target pixel in the color image. The setting unit sets the permissible range of the target pixel to be smaller when the pixel of interest has no small color difference near the target pixel than when the pixel of small color difference exists near the target pixel. The quantization apparatus according to claim 3, wherein: In addition, there is a counting target setting means for setting the scalar quantities S (1) to S (N) other than the scalar quantity S (n) that includes any of the representative values selected so far as the count target. ,
The counting means obtains a count value considering only the scalar quantity to be counted set by the counting object setting means,
The determination means selects a value Ti (i is one of 1,..., I) that maximizes the count value as one of the representative values,
The target number of representative values is selected in advance, or until the representative value candidates are exhausted, the counting target setting unit sets the counting target scalar value, the counting unit counts the accumulated value, and the determination quantization apparatus according to any one of claims 1 to 4, characterized in that repeatedly executing the selection of the representative value by means. 6. The quantization according to claim 5 , wherein the determination unit further stops the selection of the representative value when the count value of the selected representative value is a ratio equal to or less than a threshold value with respect to the N. apparatus. The counting means includes
Counting the focused scalar quantity S (n) in the allowable range Tmin (n) to Tmax (n) with respect to the focused scalar quantity S (n) as having a greater weight than the other scalar quantities. quantization device according to any one of claims 1 to 6, wherein. The scalar quantity to be quantized is a pixel value constituting an image, and
The acquisition means acquires information indicating a region of interest in the image as information for specifying an allowable range of change when the quantization is performed,
The quantization apparatus according to claim 1 , wherein the setting unit makes an allowable range of quantization of pixels in the region of interest smaller than an allowable range of quantization of pixels outside the region of interest. In the case where there are a plurality of representative value candidates that approximate the pixel distance, the quantization means preferentially selects and quantizes a candidate that has a smaller color difference between pixels associated with the distance. The quantization apparatus according to claim 8 . N scalar quantities S (1) to S (N) to be quantized, each having a value which can be taken by each scalar quantity, are input, and each scalar quantity is one of a plurality of representative values. Representing a method for controlling a quantizing device for quantizing,
An acquisition step in which an acquisition unit acquires information for specifying an allowable range of change when quantized for each scalar quantity S (n) (n is any one of 1,..., N);
A setting step in which a setting unit sets an allowable range corresponding to the information for each scalar quantity to be quantized;
An input step in which the input means inputs a scalar quantity to be quantized;
For the noted scalar quantity S (n) input by the counting means in the input process, an allowable range for the noted scalar quantity S (n) indicated by the information set in the setting process is Tmin (n) to Tmax (n). It is assumed that a plurality of scalar amounts in the range represented by Tmin (n) to Tmax (n) are input to the target scalar amount S (n), and each value of T1 to TI A counting process for cumulatively counting the number of input
A determining step for determining the plurality of representative values according to the count values of T1 to TI obtained in the counting step when the N scalar amounts are input in the input step;
And a quantization unit that quantizes the N scalar quantities using a plurality of representative values obtained in the determination step. By executing read into the computer, the computer program for causing to function as each unit included in the quantizer according to any one of claims 1 to 9. A computer-readable storage medium storing the program according to claim 11 .

Images