JP4937161B2 - Distance information encoding method, decoding method, encoding device, decoding device, encoding program, decoding program, and computer-readable recording medium - Google Patents

Description

  The present invention relates to a technique for encoding and decoding multi-view distance information.

  A multi-view image is a plurality of images obtained by photographing the same subject and background with a plurality of cameras, and a multi-view video (multi-view video) is a moving image. The distance information referred to here is information representing the distance from the camera to the subject for each region given to a certain image. The multi-view distance information is distance information for a multi-view image, and is a set of a plurality of normal distance information. Since the distance from the camera to the subject can be called the depth of the scene, the distance information is sometimes called depth information.

  In general, such distance information is given to a two-dimensional plane obtained as a result of being photographed by a camera, so that the distance information is represented as a distance image by mapping the distance to a pixel value of the image. Since the information for a certain point on the two-dimensional plane is only one distance, the distance image can be expressed as a gray scale image. The distance image is sometimes called a depth image or a depth map.

  One of the uses of distance information is a stereoscopic image. In general stereo image representation, a stereo image is composed of an observer's right-eye image and left-eye image, but a stereo image can be represented using an image from a camera and its distance information ( For details, see Non-Patent Document 1.)

  As a method for encoding a stereoscopic video represented by using the video at one viewpoint and the distance information, MPEG-C Part. 3 (ISO / IEC 23002-3) can be used (refer to Non-Patent Document 2 for details).

  The multi-view distance information is used to represent a stereoscopic image having a larger parallax than a stereoscopic image that can be expressed using single-view distance information (see Non-Patent Document 3 for details).

  In addition to the purpose of representing such stereoscopic images, multi-view distance information can be used as one of data for generating a free viewpoint image that allows the viewer to freely move the viewpoint without worrying about the location of the shooting camera. used. A composite image when a scene is viewed from a camera different from such a camera is sometimes referred to as a virtual viewpoint image, and its generation method is actively studied in the field of Image-based Rendering. As a representative method for generating a virtual viewpoint video from multi-view video and multi-view distance information, there is a method described in Non-Patent Document 4.

  As described above, the distance information can be regarded as a gray scale moving image, and the subject exists continuously in the real space and cannot move instantaneously. It can be said that there is a correlation. Therefore, distance information is efficiently encoded while removing spatial redundancy and temporal redundancy by an image encoding method and a moving image encoding method used for encoding a normal video signal. Actually MPEG-C Part. In No. 3, encoding is performed using an existing moving image encoding method.

  Here, a conventional general video signal encoding method will be described. In general, since an object has spatial and temporal continuity in real space, its appearance is highly correlated in space and time. In video signal encoding, such a correlation is utilized to achieve high encoding efficiency.

Specifically, the video signal of the encoding target block is predicted from the already encoded video signal, and only the prediction residual is encoded, thereby reducing the information that needs to be encoded, Achieve efficiency. As typical video signal prediction methods, in single-view video, intra-screen prediction that generates a prediction signal spatially from adjacent blocks, and estimation of subject motion from encoded frames taken at close times Then, there is motion compensated prediction that generates a prediction signal in time, and in multi-view images, in addition to these, the parallax of the subject is estimated from an encoded frame taken by another camera, and the prediction signal is transmitted between the cameras. There is a disparity compensation prediction that generates Details of each method are described in Non-Patent Document 5, Non-Patent Document 6, and the like.
C.Fehn, P.Kauff, M.Op de Beeck, F.Emst, W.IJsselsteijn, M.Pollefeys, L.Van Gool, E.Ofek and I.Sexton, “An Evolutionary and Optimised Approach on 3D-TV” , Proceedings of International Broadcast Conference, pp. 357-365, Amsterdam, The Netherlands, September 2002. WHA Bruls, C. Varekamp, R. Klein Gunnewiek, B. Barenbrug and A. Bourge, “Enabling Introduction of Stereoscopic (3D) Video: Formats and Compression Standards”, Proceedings of IEEE International Conference on Image Processing, pp.I-89 -I-92, San Antonio, USA, September 2007. A.Smolic, K.Mueller, P.Merkle, N.Atzpadin, C.Fehn, M.Mueller, O.Schreer, R.Tanger, P.Kauff and T.Wiegand, “Multi-view video plus depth (MVD) format for advanced 3D video systems ”, Joint Video Team of ISO / IEC JTC1 / SC29 / WG11 and ITU-T SG16 Q.6, Doc. JVT-W100, San Jose, USA, April 2007. CLZitnick, SBKang, M. Uyttendaele, SAJWinder, and R. Szeliski, “High-quality Video View Interpolation Using a Layered Representation”, ACM Transactions on Graphics, vol.23, no.3, pp.600-608, August 2004. ITU-T Rec.H.264 / ISO / IEC 11496-10, “Advanced Video Coding”, Final Committee Draft, Document JVT-E022, September 2002. H. Kimata and M. Kitahara, “Preliminary results on multiple view video coding (3DAV)”, document M10976 MPEG Redmond Meeting, July, 2004.

  The subject has a high spatial correlation because it is continuous in real space, and has a high temporal correlation because it cannot move instantaneously. Therefore, it is possible to efficiently encode distance information represented as a grayscale image by using an existing video encoding method that uses spatial correlation and temporal correlation.

  However, when the position and orientation of the camera are different as shown in FIG. 5, the distance from the camera to the subject is different even for the same subject. In such a case, even if the subject is the same, the distance varies from frame to frame, so it is impossible to accurately predict the distance in time or space.

  In order to respond to changes in the brightness of the entire scene, etc. There is a technique for efficiently predicting a value that changes for each frame by changing the original value of the reference destination and the value to be referred to using H.264 / AVC weighted prediction or the like.

  In such a method, since a single converted value is given for a given value, it becomes a many-to-one conversion. However, as shown in FIG. 6, even if the subject is the same distance from one camera, the subject may be a different distance from another camera. In this case, the conversion needs to handle one-to-many conversion, and the conventional processing cannot perform efficient prediction.

  This invention is made | formed in view of the situation which concerns, Comprising: It aims at encoding distance information more efficiently than before by performing appropriate conversion with respect to distance information.

  In order to solve the above-described problem, in the present invention, the distance from the camera to the subject is converted into a value representing a three-dimensional position that does not depend on the position or orientation of the camera that is the reference of the distance to be encoded. The three-dimensional position is converted from a point represented by the coordinate value to a value for a foot that is lowered to a predetermined number line in a three-dimensional space, and the value is encoded. As a result, a value indicating a certain point on the number line is obtained by decoding the encoded data on the decoding side, and the subject indicated by the value passes through the point indicated by the value on the number line, and the number It turns out that it exists on the plane orthogonal to a straight line. Therefore, it is possible to identify the three-dimensional position where the subject exists by using the position of the decoding target camera and the position information on the image plane from which the value was sampled. Knowing the three-dimensional position of the subject means that the distance from the camera to the subject indicated by the decoded value can be restored from the position and orientation of the camera.

  By performing such pre-processing, the information is not dependent on the camera position and orientation, which is the distance from the camera to the subject, but is not dependent on the camera position or orientation, which is a preset position on a number line. Information will be encoded. As a result, even when the position and orientation of the camera differ temporally and spatially, the representation of the information to be encoded and the information to be referenced will be unified, and efficient predictive encoding will be realized. Is possible.

  Note that just converting the distance to three-dimensional coordinates does not necessarily improve the encoding efficiency because the one-dimensional amount becomes a three-dimensional amount and the number of samples to be encoded increases, but in the present invention there is a distance. The number of samples to be encoded does not increase because it is converted into a one-dimensional quantity of values on the number line.

  In addition, since this conversion is performed according to the physical phenomenon when an object in real space is photographed by the camera, if the projection conversion by the camera can be sufficiently modeled, the conversion can be realized with very high accuracy. Can do.

  In general, a plane and a straight line in a three-dimensional space do not necessarily have an intersection. That is, at the time of decoding, there is no guarantee that a plane orthogonal to the number line and a ray sampled by the camera have an intersection. The fact that there is no intersection means that the value on the number line to be encoded cannot be determined, and that the three-dimensional position of the subject cannot be restored from the given value. That is, the above-described distance information cannot be encoded and decoded. However, by using a straight line that does not exist on any plane orthogonal to the straight line passing through the focal point of the camera and the projection plane as the number straight line, it is possible to set a number straight line that can be encoded and decoded.

  Any number line that satisfies the above conditions can be converted appropriately, and information to be encoded and information to be referred to when the position and orientation of the camera differ temporally and spatially. Is unified and efficient predictive coding can be realized. However, the range and distribution of values to be encoded differ depending on how the number line is selected. For this reason, it is possible to further improve the coding efficiency by selecting a number line using several samples so that the variance of the values after conversion becomes small. On the other hand, by increasing the variance after conversion, it is possible to convert to a value that is less susceptible to quantization distortion. When the number line is changed with respect to the sequence in this way, the decoding side is notified of what number line is used.

  In addition, by determining the axis that minimizes the number of values after conversion, instead of variance, distance information can be encoded using fewer representative values when lossless encoding is performed. Can be realized.

  In addition, when quantizing the value for the foot down to the number line in the above three-dimensional space, the quantum that minimizes the error due to quantization is selected for some samples used to select the number line. It is also preferable to select an encoding method and quantize and encode the value to be encoded using the quantization method.

  According to the present invention, the encoding target distance information or the reference target distance information is appropriately converted even when the reference position distance information and the encoding target distance information are based on different camera positions and orientations. Thus, distance information can be encoded more efficiently than in the past.

  Hereinafter, the present invention will be described in detail according to embodiments. First, a distance information encoding apparatus will be described as a first embodiment (hereinafter referred to as a first embodiment). FIG. 1 shows a configuration diagram of a distance information encoding apparatus according to the first embodiment.

  As shown in FIG. 1, a distance information encoding apparatus 100 includes a distance information input unit 101 that inputs distance information to be encoded, and a distance information pre-set that converts the input distance information into a predetermined display format. A conversion unit 102 and a distance information encoding unit 103 that actually encodes the pre-converted distance information are provided.

  The distance information pre-conversion unit 102 uses the camera parameters such as the position and orientation of the camera that has taken the image and the distance information to be encoded to calculate the three-dimensional coordinates of the subject indicated by the distance information. The point restoration unit 1021, the conversion distance information calculation unit 1022 that calculates a perpendicular foot drawn from a point expressed by the calculated three-dimensional coordinate value to a predetermined number line, and the calculated perpendicular foot A transform distance information quantization unit 1023 that quantizes the value by a predetermined method.

  FIG. 2 shows a processing flow executed by the distance information encoding apparatus 100 configured as described above. The processing executed by the distance information encoding apparatus 100 will be described in detail according to this processing flow.

  First, distance information to be encoded is input from the distance information input unit 101 [step A1]. In the first embodiment, it is assumed that distance information of each time and each camera is given as a gray scale image. Note that the distance from the camera to the subject is appropriately quantized and represented as a pixel value, and the mapping from the distance value to the pixel value is represented by S. Below, distance information is represented as D, and position information on an image sandwiched between symbols [] is added to represent distance information of a specific region.

  The inputted distance information D is converted into D ′ by the distance information pre-conversion unit 102 [step A2-A6]. Conversion is performed for each sample (pixel). In other words, if the index representing the sample is pix and the number of input samples is numPixs, after initializing pix to 0 [Step A2], adding 1 to pix [Step A5], until pix becomes numPixs [Step A6], the following steps A3-A4 are repeated.

  In the process performed for each sample, the three-dimensional point restoration unit 1021 first obtains the three-dimensional position gpix in the unified coordinate system of the subject observed at the position of pix on the image using the camera parameters [Step A3]. Since the subject in the three-dimensional space is projected onto a two-dimensional image plane according to the projection model of the camera, it is possible to obtain the three-dimensional position by performing the inverse transformation. Specifically, it can be calculated using the following formula.

  Since there are various ways to represent camera parameters, it is necessary to use mathematical formulas according to the definition. In the present embodiment, it is assumed that the correspondence between the image coordinate m and the world coordinate M uses a camera parameter expression obtained by the equation M * = RA-1m * + t. A, R, and t represent the internal parameter matrix, rotation matrix, and translation vector of the camera, respectively. M * and m * represent homogeneous coordinates that allow arbitrary scalar multiplication for M and m. A and R are 3 × 3 matrices, and t is a three-dimensional vector. (Upix, vpix) represents the position of pix on the image plane. S-1 represents the reverse projection of S.

  When the three-dimensional coordinate gpix of pix is obtained, the conversion distance information calculation unit 1022 projects the point onto a predetermined number line V to obtain a scalar value. Also, the transform distance information quantization unit 1023 quantizes the value by a predetermined method to obtain a transformed value D ′ [upix, vpix] [step A4].

  As long as it is not parallel to any plane orthogonal to the straight line connecting the focal point of the camera and any pixel, any number of straight lines may be V. Note that the same V and quantization method are used for all distance information.

  For example, assuming that the axis perpendicular to the projection plane of the camera is V, the origin is the point where the focus of the camera is projected, and the orientation is the same as the orientation of the camera, the projected value of gpix is S-1 (D [upix, vpix]) Thus, if quantization is performed with S, a value of D is obtained and conversion is not performed.

  As an example of simple processing, there is a method of performing transformation by quantizing a certain component of gpix with one axis of the unified coordinate system as V. If the x-axis is V, the first component is quantized, if the y-axis is V, the second component is quantized, and if the z-axis is V, the third component is quantized. At this time, the distance value can be restored by solving the following simultaneous equations.

  This is a restoration formula when the z-axis is set to V in the conversion and the value of the third component of gpix is used as it is after conversion without quantization. Note that D ′ [upix, vpix] = zpix. In Equation 2 above, there are three unknowns x, y, and d, and since three equations are given, it is possible to solve the equation for the unknown. Here, when only the distance value is restored, the distance value can be restored without wasteful calculation by solving the equation for the third row with respect to d.

  When a straight line having a direction vector m having a size of 1 with point p as zero is used as V, a value L when gpix is projected onto V is calculated by the following equation.

L = (gpix−p) m (Formula 3)
● represents the inner product of vectors. The process of restoring the distance from L can be realized by solving the following simultaneous equations.

  Here, there are four unknowns x, y, z, and d, and since four equations are given, they can always be solved. Specifically, it is possible to obtain d by expressing each of x, y, z from the first formula as a formula d and substituting it into the second formula.

  In addition, the axis may be determined using some distance information and taking into account the variance of the converted value L. This optimization problem for variance can be solved by performing principal component analysis on the sample distance information set. In this case, maximizing the variance makes the representation less susceptible to quantization, and minimizing the variance makes it possible to create a compact representation. In addition, principal component analysis may be performed on the sample distance information set to determine an axis that minimizes the number of values serving as principal component scores. By determining an axis that minimizes the number of converted values, distance information can be encoded using a small number of representative values when lossless encoding is performed.

  For the quantization, either linear quantization or nonlinear quantization may be used. Further, the L value after the conversion may be quantized, or the inverse of L may be quantized. If the distance value itself is meaningful, the L value should be quantized as it is, and if the amount of parallax caused by the subject distance is meaningful, it should be performed on the reciprocal of L. This is because L is a value proportional to the subject distance when a certain viewpoint is determined, and the parallax amount is a value proportional to the reciprocal of the subject distance.

  In quantization, for example, a quantization method setting means for setting a quantization method that minimizes an error due to quantization is provided for the sample distance information set described above, and information indicating the set quantization method is provided. And the value to be encoded may be quantized using the set quantization method.

  When the value of D ′ is obtained for all pixes, the converted distance information D ′ is encoded by the distance information encoding unit 103 [step A7]. The encoding result is the output of the distance information encoding apparatus 100.

  In addition, it is necessary to use a common axis on the encoding side and the decoding side for the axis V of the number line and the quantization method. In other words, it is necessary to always use a predetermined one in common on the encoding side and the decoding side, or encode the used value and notify the decoding side.

  By using such pre-conversion, the same value is expressed as long as the position of the subject does not change regardless of the position and orientation of the camera. In addition, since the change in the value also corresponds to the change in the position of the subject, the inter-frame correlation of the information to be encoded is increased, and efficient predictive encoding can be realized.

  In the first embodiment, the coordinate value of the unified coordinate system is obtained for each sample and D ′ is calculated. However, after obtaining the coordinate value of the unified coordinate system for all samples, D ′ is calculated. It doesn't matter.

  Next, a distance information decoding apparatus will be described as a second embodiment (hereinafter referred to as a second embodiment). FIG. 3 shows a configuration diagram of a distance information decoding apparatus according to the second embodiment.

  As shown in FIG. 3, a distance information decoding apparatus 200 includes a distance information encoded data input unit 201 that inputs encoded data of distance information to be decoded, and distance information that actually decodes the input encoded data. A decoding unit 202 and a distance information post conversion unit 203 that converts the decoding result into information representing the distance from the actual camera to the subject.

  The distance information post-conversion unit 203 includes a decoding distance information inverse quantization unit 2031 that inversely quantizes the decoded decoding distance information by a predetermined method, and a predetermined number corresponding to the dequantized value. A subject plane setting unit 2032 that sets a plane that passes through points on a straight line and is orthogonal to a number line, and the position of the focus information of the camera that has captured the distance information of the decoding target and the distance information of the decoding target on the projection plane of the camera And a subject distance setting unit 2033 that sets a distance from the camera to the intersection of the subject plane and the subject ray as a distance from the decoding target camera to the subject. .

  FIG. 4 shows a processing flow executed by the distance information decoding apparatus 200 configured as described above. The processing executed by the distance information decoding device 200 will be described in detail according to this processing flow.

  First, encoded data of distance information to be decoded is input from the encoded distance information data input unit 201 [step B1]. The input encoded data is decoded by the distance information decoding unit 202 [step B2]. In the second embodiment, it is assumed that distance information of each time and each camera is expressed as a gray scale image. That is, as a result of the decoding process here, pseudo-distance information obtained by adding some conversion to the distance information to be decoded is obtained. Hereinafter, the decoded pseudo distance information is represented as Dec ′, and the position information sandwiched between symbols [] is added to represent the pseudo distance information of a specific region.

  The decoded pseudo distance information Dec ′ is converted by the distance information post conversion unit 203 into distance information Dec representing the actual camera to the subject [step B3-B7]. Conversion is performed for each pixel. That is, assuming that the index representing the pixel is pix and the number of pixels is numPixs, after initializing pix to 0 [Step B3], while adding 1 to pix [Step B6], until pix becomes numPixs [Step B7] ], The following steps B4-B5 are repeated.

  In the processing performed for each pixel, first, the plane P in the unified coordinate system in which the three-dimensional point of the object observed at the position of pix on the image exists is obtained from the pseudo distance information Dec ′ [pix] [Step B4]. . The decoded pseudo distance information Dec ′ [pix] is obtained by quantizing a scalar value when a three-dimensional point of a subject is projected onto a certain number line V. That is, the plane P to be obtained is a plane orthogonal to the number line V at a position indicated by Dec ′ [pix]. As the number line and the quantization method, those previously set on the encoding side are used. A constant number may be used at all times, or an appropriate number line and quantization method may be designed and used for each sequence. In the latter case, information indicating them is encoded and notified to the decoding apparatus.

  Here, if a number line V is a straight line having a direction vector n with a point p0 as a zero and a magnitude of 1, and a quantization function is Q, the plane P to be obtained is expressed by the following equation. An arbitrary point on the plane is represented by g.

n. (p0 + Q-1 [Dec '[pix]]. n) = n.g (Formula 5)
Next, the distance information of the object observed at the position of pix is calculated from the plane P and the camera parameters, and the result is set to Dec [pix] [Step B5]. Specifically, a straight line where the subject exists is obtained from the information of the camera parameters and the position of pix, and the distance value is obtained assuming that the subject exists at the intersection of the straight line and the plane. The straight line on which the subject exists is expressed by the following mathematical formula.

  The camera parameter expression method is the same as that described in the first embodiment, and A, R, and t represent an internal parameter matrix, a rotation matrix, and a translation vector of the camera, respectively. d is an arbitrary scalar variable, and g represents a point on a straight line. (Upix, vpix) represents the position of pix on the image plane.

  Here, since d in Expression 6 represents the distance from the camera to the subject, the distance value can be obtained by solving the simultaneous equations of Expression 5 and Expression 6 for d. In addition, although g represents the three-dimensional coordinate value of the subject, it is not always necessary to obtain it here. As described in the first embodiment, when a straight line that is not parallel to any plane orthogonal to the straight line connecting the focal point of the camera and an arbitrary pixel is selected as the number line V, Expression 5 and Expression 6 are established. Simultaneous equations always have a unique solution.

  In this embodiment, the value of the distance from the camera to the subject is used as the output value Dec [pix], but it is necessary to output a value obtained by quantizing the distance from the camera to the subject by a predetermined method. In this case, the value of d obtained above is quantized by a given method and output.

  The processing described above can be realized by a computer and a software program, and the program can be provided by being recorded on a computer-readable recording medium or can be provided through a network.

  In the above embodiments, the distance information encoding device and the distance information decoding device have been mainly described. However, the distance information encoding method and the distance information decoding method of the present invention are performed according to steps corresponding to the operations of the respective units of these devices. Can be realized.

  The embodiments of the present invention have been described above with reference to the drawings. However, the above embodiments are merely examples of the present invention, and it is clear that the present invention is not limited to the above embodiments. . Accordingly, additions, omissions, substitutions, and other modifications of the components may be made without departing from the spirit and scope of the present invention.

It is a figure which shows the structure of the distance information encoding apparatus of Example 1. FIG. It is a distance information encoding flowchart in Example 1. It is a figure which shows the structure of the distance information decoding apparatus of Example 2. FIG. It is a distance information decoding flowchart in Example 2. It is a figure which shows that the distance with respect to the same to-be-photographed object changes with camera positions and directions. It is a figure which shows that the difference of the distance between cameras differs with a to-be-photographed object.

Explanation of symbols

DESCRIPTION OF SYMBOLS 100 Distance information encoding apparatus 101 Distance information input part 102 Distance information preconversion part 1021 Three-dimensional point decompression | restoration part 1022 Transformation distance information calculation part 1023 Transformation distance information quantization part 103 Distance information encoding part 200 Distance information decoding apparatus 201 Distance information Encoded data input unit 202 Distance information decoding unit 203 Distance information post-conversion unit 2031 Decoding distance information inverse quantization unit 2032 Subject plane setting unit 2033 Subject light beam setting unit 2034 Subject distance calculation unit

Claims (15)

In a distance information encoding method for encoding distance information representing a distance from a camera to a subject with respect to an image taken by a camera,
A three-dimensional point restoration step of calculating the three-dimensional coordinates of the subject indicated by the distance information using the parameters of the camera that has taken the image and the distance information to be encoded;
A conversion distance information calculation step for calculating a leg of a perpendicular drawn from a point represented by the calculated three-dimensional coordinate value to a predetermined number line;
A transform distance information quantization step for quantizing the calculated value of the perpendicular foot by a predetermined method;
A transform distance information encoding step for encoding the quantized value;
A distance information encoding method characterized by comprising: In the distance information encoding method according to claim 1,
Use a straight line that does not exist in any plane orthogonal to the straight line passing through the focal point of the camera and the projection plane as the number straight line used in the conversion distance information calculation step.
The distance information encoding method characterized by these. In the distance information encoding method according to claim 1,
An axis setting step for performing principal component analysis on a subset of the 3D point set obtained in the 3D point restoration step and setting an axis that maximizes the variance of the principal component scores;
An axis information encoding step for encoding information indicating the axis set in the axis setting step,
Using the axis set in the axis setting step as a number line used in the conversion distance information calculation step,
The distance information encoding method characterized by these. In the distance information encoding method according to claim 1,
An axis setting step for performing principal component analysis on a subset of the 3D point set obtained in the 3D point restoration step and setting an axis that minimizes the variance of the principal component scores;
An axis information encoding step for encoding information indicating the axis set in the axis setting step,
Using the axis set in the axis setting step as a number line used in the conversion distance information calculation step,
The distance information encoding method characterized by these. In the distance information encoding method according to claim 1,
An axis setting step for performing principal component analysis on a subset of the three-dimensional point set obtained in the three-dimensional point restoration step and setting an axis that minimizes the number of values serving as principal component scores;
An axis information encoding step for encoding information indicating the axis set in the axis setting step,
Using the axis set in the axis setting step as a number line used in the conversion distance information calculation step,
The distance information encoding method characterized by these. In the distance information encoding method according to any one of claims 3 to 5,
A sample conversion distance information calculation step for calculating a foot of a perpendicular drawn to the axis obtained in the axis setting step from each point included in the subset of the three-dimensional point set used in the axis setting step;
A quantization method setting step for setting a quantization method that minimizes an error due to quantization when the value indicated by the calculated vertical line is quantized to a predetermined number;
A quantization method encoding step for encoding information representing the set quantization method, and
In the transform distance information quantization step, the quantization method set in the quantization method setting step is used to quantize the value indicating the perpendicular foot calculated in the transform distance information calculation step,
The distance information encoding method characterized by these. In a distance information decoding method for decoding encoded data of distance information representing a distance from a camera to a subject with respect to an image taken by a camera,
An input data decoding step for decoding the input encoded data and setting decoding distance information;
A decoding distance information inverse quantization step for inversely quantizing the decoded decoding distance information by a predetermined method;
A subject plane setting step for setting a plane that passes through a point on a predetermined number line corresponding to the dequantized value and is orthogonal to the number line;
A subject ray setting step for setting, as a subject ray, a straight line connecting the focus of the camera that has captured the distance information to be decoded and the position of the distance information to be decoded on the projection plane of the camera;
Calculating the distance from the camera to the intersection of the object plane and the object ray, and calculating the distance from the camera to be decoded to the object;
A distance information decoding method characterized by comprising: The distance information decoding method according to claim 7,
An axis information decoding step for decoding encoded data of information defining an axis in a three-dimensional space,
In the subject plane setting step, the axis decoded in the axis information decoding step is used as a number line.
The distance information decoding method characterized by this. In the distance information decoding method according to claim 7 or 8,
A quantization method decoding step for decoding encoded data of information indicating a method of quantizing a value on a number line,
In the decoding distance information inverse quantization step, the inverse transformation of the quantization method decoded in the quantization method decoding step is used as an inverse quantization method,
The distance information decoding method characterized by this. In a distance information encoding device that encodes distance information representing a distance from a camera to a subject with respect to an image taken by a camera,
Three-dimensional point restoration means for calculating the three-dimensional coordinates of the subject indicated by the distance information using the parameters of the camera that has taken the image and the distance information to be encoded;
Conversion distance information calculation means for calculating a perpendicular foot drawn from a point represented by the calculated three-dimensional coordinate value to a predetermined number line;
Transform distance information quantization means for quantizing the calculated value of the perpendicular foot by a predetermined method;
Transform distance information encoding means for encoding the quantized value;
A distance information encoding device comprising: In a distance information decoding device that decodes encoded data of distance information representing a distance from a camera to a subject with respect to an image photographed by the camera,
Input data decoding means for decoding input encoded data and setting decoding distance information;
Decoding distance information inverse quantization means for inversely quantizing the decoded decoding distance information by a predetermined method;
Subject plane setting means for setting a plane that passes through a point on a predetermined number line corresponding to the dequantized value and is orthogonal to the number line;
Subject light beam setting means for setting, as a subject light beam, a straight line connecting the focal point of the camera that has captured the distance information of the decoding target and the position of the distance information of the decoding target on the projection plane of the camera;
Subject distance calculating means for calculating a distance from the camera to the intersection of the subject plane and the subject light beam, and setting the distance from the camera to be decoded to the subject;
A distance information decoding apparatus comprising:   A distance information encoding program for causing a computer to execute the distance information encoding method according to any one of claims 1 to 6.   The computer-readable recording medium which recorded the distance information encoding program of Claim 12.   A distance information decoding program for causing a computer to execute the distance information decoding method according to any one of claims 7 to 9.   The computer-readable recording medium which recorded the distance information decoding program of Claim 14.

Images