WO2013155814A1

WO2013155814A1 - View synthesis method and apparatus when encoding depth information and encoder

Info

Publication number: WO2013155814A1
Application number: PCT/CN2012/080100
Authority: WO
Inventors: 贾杰; 刘鸿彬; 马思伟; 王诗淇
Original assignee: 乐金电子(中国)研究开发中心有限公司; 北京大学
Priority date: 2012-04-20
Filing date: 2012-08-14
Publication date: 2013-10-24
Also published as: CN103379348A; CN103379348B

Abstract

A view synthesis method and apparatus when encoding depth information and an encoder are provided. The method includes: judging whether each line of pixels of a depth image encoding unit leads to variations of the pixel value of the synthesized view during the view synthesis when encoding the depth information; if judging that a certain line of pixels of the depth image encoding unit does not lead to the variations of the pixel value of the synthesized view, skipping the line during the view synthesis and proceeding the view synthesis without the line of pixels. The apparatus includes: a judging unit for judging whether each line of pixels of the depth image encoding unit leads to the variations of the pixel value of the synthesized view during the view synthesis when encoding the depth information; a view synthesis processing unit for, if judging that a certain line of pixels of the depth image encoding unit does not lead to the variations of the pixel value of the synthesized view, skipping the line during the view synthesis and proceeding the view synthesis without the line of pixels. The technical solutions mentioned above can reduce the time complexity of the encoding end while ensuring the encoding performance.

Description

The present invention claims to be submitted to the Chinese Patent Office on April 20, 2012, and the application number is 201210119222.6. The invention name is "a method for synthesizing viewpoints when encoding depth information, The priority of the Chinese Patent Application for Apparatus and Encoder is hereby incorporated by reference in its entirety. Technical field

The present invention relates to the field of multimedia technologies, and in particular, to a method, an apparatus, and an encoder for synthesizing a viewpoint when encoding depth information.

Background technique

In the 98th MPEG (Moving Pictures Experts Group) conference, HHI (Heinrich Hertz Institute, Rhein-Hertz Institute) proposed a rate-distortion optimization based on synthetic viewpoint distortion information in depth image compression. algorithm. In this algorithm, the distortion size of the depth image is measured by the distortion variation of the synthesized viewpoint, which can be expressed as:

M) = - D = [Ρ _Γ (χ, y) - s (χ, y)f - [ ⁵ ( y) ~ ^s

y)f (丄) where represents a virtual viewpoint synthesized using the original texture image and the original depth information. At the same time, when encoding the depth information, a depth image is divided into three types: encoded, current encoded and uncoded. For and , both select the texture image to which the distortion is applied for synthesis. The difference is that ( ^X , W applies the reconstructed coded depth information, the original depth information of the current coding unit and the original other pixel depth information are synthesized, and the reconstructed coded depth information is applied, the current coding unit depth of the distortion The information is synthesized with the original other pixel depth information. It can be seen that ΔΙ) = 0 if the depth information of the current distortion has no effect on the view synthesis. The existing technical solution is to estimate the synthesized viewpoint distortion caused by the current depth information distortion based on the change of the synthesized viewpoint distortion, so the following operations are required in the actual encoding:

1. Before encoding the current depth image, perform the viewpoint synthesis on the original depth image and the original texture image, that is, the composite '". Perform the viewpoint synthesis on the original depth image and the distorted texture image, that is, the synthesis and before the encoding process starts = g .

2. When encoding the current depth image coding unit, rate distortion calculation is required. Let the original value of the current depth image coding unit be B, the distortion block be B', the distortion depth information of the coded area of the depth image is Ρ', and the original depth information of the uncoded area of the depth image is Η, then the current ( ^X , W The sum is drawn by the depth information of P, B, H, and does not need to be updated at this time ( ^x , (updated in 2.3). In order to calculate ΔΖ) in formula (1), the embodiment of the present invention needs to apply B. 'Update g( ^x , . In this process, only the composite viewpoint of B' (the pixel in ^χ , is redrawn, so that the synthetic viewpoint drawn by the depth information of Ρ', Β', H can be obtained, Thus, ΔΟ can be obtained according to the formula (1), and it can be used for rate distortion calculation.

3. After encoding the current depth image coding unit, the synthesized view information is updated with the reconstructed current depth image coding unit ( ^X , W. Suppose the current depth block is B, the reconstructed block is B", and the depth image has been coded. The distortion depth information is Ρ', and the original depth information of the uncoded area of the depth image is

Hey. In the process of encoding the current depth image coding unit, is it? The depth information of ', 8, 11 is obtained. After encoding the current depth image coding unit, it is necessary to apply the "update to the update. The update process only re-draws the pixels in the composite view ( ^X , W) corresponding to Β" Do not repaint the entire image, so that it is drawn with Ρ', Β", Η ( ^X , W.

As can be seen from the above process, the existing technical solution requires continuous drawing and updating of the viewpoint in the encoding process, which requires a large time complexity (time overhead). Therefore, it is necessary to design a fast technical solution to reduce the time complexity of the encoding end while ensuring the encoding performance. Summary of the invention

Embodiments of the present invention provide a method, an apparatus, and an encoder for synthesizing a viewpoint when encoding depth information, so as to reduce the time complexity of the encoding end while ensuring encoding performance.

In one aspect, the embodiment of the present invention provides a method for synthesizing a view when encoding depth information, where the method for synthesizing view when encoding depth information includes:

In the process of view synthesis in encoding the depth information, it is determined whether each line of pixels of the depth image coding unit causes a change in the pixel value of the synthesized view;

If it is determined that a certain row of pixels of the depth image coding unit does not cause a change in the composite view pixel value, the line is skipped during the view synthesis process, and the line synthesis is not performed using the line pixel.

Preferably, in an embodiment of the present invention, determining whether each row of pixels of the depth image encoding unit causes a change in the synthesized viewpoint pixel value comprises: determining a depth image encoding unit according to the relationship between the depth information of the pixel and the disparity information The depth information distortion of a certain row of pixels does not cause parallax distortion, and it is determined that the row of pixels does not cause a change in the synthesized viewpoint pixel value.

Preferably, in an embodiment of the present invention, determining whether each row of pixels of the depth image encoding unit causes a change in the pixel value of the synthesized viewpoint comprises: determining a depth image according to a relationship between a gradient characteristic of the texture image and a synthetic viewpoint distortion; The distortion of the parallax of a certain row of pixels of the coding unit does not affect the synthesized viewpoint distortion, and it is determined that the row of pixels does not cause a change in the composite viewpoint pixel value.

Preferably, in an embodiment of the present invention, determining whether each row of pixels of the depth image encoding unit causes a change in the synthesized viewpoint pixel value comprises: determining whether a pixel in the synthesized viewpoint corresponding to a certain row of pixels of the depth image encoding unit is If it is determined that the pixels in the synthesized viewpoint corresponding to a certain row of pixels of the depth image encoding unit are occluded, further determining whether the pixel in the corresponding synthesized viewpoint after the pixel of the depth image encoding unit is distorted is still If the pixel in the corresponding synthesized viewpoint is still occluded after the row pixel of the depth image encoding unit is determined to be occluded, it is determined that the row of pixels does not cause a change in the synthesized viewpoint pixel value.

On the other hand, an embodiment of the present invention provides a view point synthesizing apparatus for encoding depth information, and the view point synthesizing apparatus for encoding depth information includes: a determining unit, configured to determine, in a process of synthesizing the depth information when encoding the depth information, whether each row of pixels of the depth image encoding unit causes a change in the pixel value of the synthesized viewpoint;

The view synthesis processing unit is configured to skip the line in the view synthesis process if it is determined that a certain row of pixels of the depth image coding unit does not cause a change in the synthesized view pixel value, and does not use the line pixel for view synthesis.

Preferably, in an embodiment of the present invention, the determining unit includes: a first determining module, configured to determine, according to the relationship between the depth information of the pixel and the disparity information, if the depth information of a certain row of pixels of the depth image encoding unit is not distorted As a result of parallax distortion, it is determined that the row of pixels does not cause a change in the synthesized viewpoint pixel value.

Preferably, in an embodiment of the present invention, the determining unit includes: a second determining module, configured to determine a parallax of a row of pixels of the depth image encoding unit according to a relationship between a gradient characteristic of the texture image and a synthetic viewpoint distortion If the distortion does not affect the synthetic viewpoint distortion, it is determined that the row of pixels does not cause a change in the composite viewpoint pixel value.

Preferably, in an embodiment of the present invention, the determining unit includes: a third determining module, determining whether a pixel in a synthesized viewpoint corresponding to a certain row of pixels of the depth image encoding unit is occluded; if determining a certain depth image encoding unit If the pixels in the synthesized view corresponding to one line of pixels are occluded, it is further determined whether the pixels in the corresponding synthesized view are still occluded after the line pixel of the depth image coding unit is distorted; if it is determined that the depth image coding unit is After the row of pixels is distorted, the pixels in the corresponding synthesized viewpoint are still occluded, and it is determined that the row of pixels does not cause a change in the synthesized viewpoint pixel value.

In still another aspect, an embodiment of the present invention provides an encoder, where the encoder includes the above-described viewpoint synthesis device when encoding depth information.

The above technical solution has the following beneficial effects: It is determined whether each row of pixels of the depth image encoding unit causes a change of the synthesized viewpoint pixel value in the viewpoint synthesis process when encoding the depth information; if it is determined that a certain row of pixels of the depth image encoding unit is not Resulting in a change in the pixel value of the synthetic viewpoint In the process of view synthesis, the line is skipped, and the line pixels are not used for the technique of view synthesis. Therefore, the time complexity of the code end can be reduced while ensuring the coding performance.

DRAWINGS

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the embodiments or the description of the prior art will be briefly described below. Obviously, the drawings in the following description are only It is a certain embodiment of the present invention, and other drawings can be obtained from those skilled in the art without any inventive labor.

1 is a flowchart of a method for synthesizing a viewpoint when encoding depth information according to an embodiment of the present invention; FIG. 2 is a schematic structural diagram of a device for synthesizing a viewpoint when encoding depth information according to an embodiment of the present invention; FIG. 3 is a structural diagram of a judging unit according to an embodiment of the present invention; Schematic diagram

4 is a schematic diagram of the original parallax and the disparity of the disparity corresponding to the same 1/4 pixel precision parallax in the application example of the present invention;

FIG. 5(a) is a schematic diagram of a synthetic viewpoint corresponding to an original depth of an application example of the present invention; FIG.

FIG. 5(b) is a schematic diagram of a synthetic viewpoint corresponding to a distortion depth of an application example of the present invention; FIG.

Figure 5 (c) is a schematic diagram showing the difference in pixel values of the synthesized viewpoints in Figures 5(a) and 5(b) of the application example of the present invention;

6 is a schematic diagram of an occlusion pixel of an application example of the present invention;

FIG. 7 is a flowchart of a method for synthesizing a viewpoint when encoding depth information according to an application example of the present invention. detailed description

The technical solutions in the embodiments of the present invention are clearly and completely described in the following with reference to the accompanying drawings in the embodiments of the present invention. It is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative efforts are within the scope of the present invention.

As shown in FIG. 1 , it is a flowchart of a method for synthesizing a view when encoding depth information according to an embodiment of the present invention, where the method for synthesizing a view when encoding depth information includes: 101. In a process of synthesizing the depth information when encoding the depth information, determining whether each row of pixels of the depth image encoding unit causes a change in the pixel value of the synthesized viewpoint;

102. If it is determined that a certain row of pixels of the depth image coding unit does not cause a change in the composite view pixel value, the row is skipped during the view synthesis process, and the view pixel synthesis is not performed by using the row of pixels.

Preferably, determining whether each row of pixels of the depth image encoding unit causes a change of the synthesized viewpoint pixel value comprises: determining, according to the relationship between the depth information of the pixel and the disparity information, determining the depth information distortion of a certain row of pixels of the depth image encoding unit If parallax distortion is not caused, it is determined that the row of pixels does not cause a change in the synthesized viewpoint pixel value.

Or, preferably, determining whether each row of pixels of the depth image encoding unit causes a change of the synthesized viewpoint pixel value comprises: determining a row of pixels of the depth image encoding unit according to a relationship between a gradient characteristic of the texture image and the synthesized viewpoint distortion The distortion of the parallax does not affect the synthetic viewpoint distortion, and it is determined that the row of pixels does not cause a change in the composite viewpoint pixel value.

Or, the determining whether the pixel of each line of the depth image encoding unit causes the change of the composite view pixel value comprises: determining whether the pixel in the synthesized view corresponding to a certain row of pixels of the depth image encoding unit is occluded; If the pixel in the synthesized view corresponding to a certain row of pixels of the image coding unit is occluded, it is further determined whether the pixel in the corresponding synthesized view is still occluded after the line pixel of the depth image coding unit is distorted; If the pixels in the corresponding synthesized viewpoint are still occluded after the row pixel of the depth image encoding unit is distorted, it is determined that the row of pixels does not cause a change in the synthesized viewpoint pixel value.

Corresponding to the foregoing method embodiment, as shown in FIG. 2, it is a schematic structural diagram of a view synthesizing device when encoding depth information according to an embodiment of the present invention, and the view synthesizing device for encoding depth information includes:

The determining unit 21 is configured to determine, during the view synthesis process when the depth information is encoded, whether each row of pixels of the depth image encoding unit causes a change of the synthesized view pixel value; The view synthesis processing unit 22 is configured to skip the line in the view synthesis process if it is determined that a certain row of pixels of the depth image coding unit does not cause a change in the composite view pixel value, and does not use the line pixel to perform view synthesis.

Preferably, as shown in FIG. 3, which is a schematic structural diagram of a determining unit according to an embodiment of the present invention, the determining unit 21 includes: a first determining module 211, configured to determine a depth image coding according to a relationship between depth information of a pixel and disparity information. If the depth information distortion of a certain row of pixels of the cell does not cause parallax distortion, it is determined that the row of pixels does not cause a change in the composite viewpoint pixel value. Alternatively, the determining unit includes: a second determining module 212, configured to determine, according to the relationship between the gradient characteristic of the texture image and the synthesized viewpoint distortion, if the distortion of the parallax of a certain row of pixels of the depth image encoding unit is not affected by the synthesized viewpoint distortion Then, it is determined that the row of pixels does not cause a change in the composite viewpoint pixel value. Alternatively, the determining unit includes: a third determining module 213, configured to determine whether a pixel in a synthesized view corresponding to a certain row of pixels of the depth image encoding unit is occluded; and if determining a synthesized view point corresponding to a certain row of pixels of the depth image encoding unit If the pixel in the pixel is occluded, it is further determined whether the pixel in the corresponding synthesized viewpoint is still occluded after the line pixel of the depth image encoding unit is distorted; if it is determined that the line pixel of the depth image encoding unit is distorted If the pixels in the corresponding synthesized viewpoint are still occluded, it is determined that the row of pixels does not cause a change in the composite viewpoint pixel value.

The foregoing method, apparatus, or encoder technical solution has the following beneficial effects: because in the process of view synthesis in encoding depth information, it is determined whether each row of pixels of the depth image coding unit causes a change in the pixel value of the synthesized view; Determining that a certain row of pixels of the depth image coding unit does not cause a change in the pixel value of the synthesized viewpoint, skipping the line in the process of view synthesis, and not using the line pixel to perform the technique of view synthesis, can be reduced while ensuring the coding performance. The time complexity of the small coding end. Embodiments of the Invention The above described method, apparatus or encoder solution can be applied to the processes described in Backgrounds 2 and 3. Different from the process of encoding the depth image coding unit and the coded depth image coding unit, the process of the process of encoding the depth image coding unit is different, and the embodiment of the present invention uses the depth information and the disparity information of the pixel before updating the synthesized view point. The relationship is analyzed according to the relationship between the gradient characteristics of the texture image and the synthetic viewpoint distortion, or the occlusion characteristics of the synthetic viewpoint, to determine whether the synthetic viewpoint needs to be updated. The embodiment of the present invention can effectively reduce the synthesized view update operation by reducing the encoder complexity by performing the update operation of the synthesized view only for some of the depth image coding units or some of the depth image coding units. (It should be noted that the technical solution has no influence on the decoding end.) The specific technical solution of the application example of the present invention is as follows:

1. In the case where the cameras are horizontally arranged in parallel, the relationship between the true depth information z and the parallax information ί can be expressed as:

D-l±

z (2)

Where / represents the focal length of the camera. / Represents the reference distance between two viewpoints.

If the quantization depth corresponding to the current depth z is ν=ρ(ζ), the relationship between the disparity information and the quantization depth can be expressed as:

As can be seen from equation (3), different depth information leads to different horizontal parallaxes. However, in practical applications, horizontal parallax is generally rounded. For example, in the existing 3DV-HEVC, the horizontal parallax is rounded by 1/4 pixel precision, as shown in Fig. 4, which is a schematic diagram of the original parallax and distortion parallax corresponding to the same 1/4 pixel precision parallax for the application example of the present invention. After rounding, it can be seen that the original depth and the distortion depth correspond to the horizontal disparity of the same 1/4 pixel precision. Therefore, the distortion of the depth information does not necessarily cause distortion of the horizontal parallax.

Let the position of the current pixel be p, and the rounding operation of the parallax corresponding to the pixel depth value at 1/N precision is RA<d:). Assume that ^ is the original parallax and 4 _P is the distortion parallax. Then the parallax distortion can be expressed as

D _N (d _op , d _sp ) = R _N (d _op ) -R _N (d _sp ) _(A , Then, when the distortion depth v 'is a condition of the set d of the following conditions, the current depth image distortion does not affect the view synthesis.

ς = { Ί 3⁄4(^(ν)^( ')) = ο} ₍₅₎

In the existing 3DV-HEVC, the horizontal parallax is rounded with 1/4 pixel precision, then Ν=4.

2. When the parallax distortion is not 0, the distortion of the synthesized viewpoint has a great relationship with the texture image characteristics:

As shown in FIG. 5( a ), it is a schematic diagram of a synthetic viewpoint corresponding to the original depth of the application example of the present invention; as shown in FIG. 5( b ), a schematic view of a synthetic viewpoint corresponding to the distortion depth of the application example of the present invention; FIG. 5( c ) Shown is a schematic diagram showing the difference in pixel values of the synthesized viewpoints in FIGS. 5(a) and 5(b) of the application example of the present invention. The pixel values of positions 2 to 5 in the texture image are the same. The gradient of the texture image at these positions is very small or 0. In this case, the pixel difference in the synthesized viewpoint obtained by using the original depth information and the distortion depth information is small, as shown in FIG. 5. (a) and FIG. 5(b), the pixel values of the synthesized viewpoint positions 1 to 3; on the other hand, the pixel values of the positions 5 to 9 in the texture image vary greatly, and the gradient of the texture image at these positions is large, in this case The pixels in the synthesized viewpoint obtained by using the original depth information and the distortion depth information are largely different, and the pixel values of the synthesized viewpoint positions 4 to 7 are as shown in FIGS. 5(a) and 5(b). That is, in the smooth region, the distortion of the parallax does not affect the synthetic viewpoint distortion. Therefore, the embodiment of the present invention defines that when the distortion depth ν ' satisfies the condition of the set C ₂ as follows, it can be considered that the current depth image distortion does not affect the view synthesis.

C ₂ = {v' l fe(v),g(v')) = 0 and = 0} ₍₆₎

Where G(p) is the gradient of the distorted texture image at position p. It can be expressed as:

G(P) =| i(p) - i(p - i) \ + \ I(P) - i(P + 1) I (7) 3. The pixels in the synthesized viewpoint corresponding to the current depth pixel are occluded This depth information does not work in view synthesis. As shown in FIG. 6, the application example of the present invention obscures the pixel, and the depth information of the c, d pixels does not affect the result of the viewpoint synthesis. Therefore, if the pixel in the corresponding synthesized viewpoint is still occluded after the current depth pixel is distorted, the current depth pixel distortion is considered not to be Will have an impact on view synthesis. Embodiments of the present invention define that when the distortion depth V' satisfies the conditions of the following set, it can be considered that the current depth image distortion does not affect the view synthesis:

Q ={V | A(g(v), g(v')) = 0 and ve O} ₍₈₎ where Ο is the depth pixel of the occlusion (ie, the pixels in its corresponding composite view are occluded) set. It is assumed that the set of the _/th row pixels of the current depth image coding unit is Sj, and from the condition d, c ₂ , c ₃ defined above, when all the pixels of the current line satisfy any of the conditions of the set c _h c ₂ , which is

C = ! uc ₂ uc ₃ (9) It can be considered that the distortion of the current line depth information pixel has no effect on the synthesized viewpoint distortion.

As shown in FIG. 7, the flow chart of the method for synthesizing the viewpoint when encoding the depth information of the application example of the present invention includes the following steps:

701, initialize the household 0;

702, the first _ / row pixel in the current block

703, determine whether ^ _£ ^, _{£ £} (:, if yes, can be considered that the distortion of the current line depth information pixel has no effect on the synthetic viewpoint distortion, then step 704, if not, then step 706;

704. Skip the _/th line, and perform the view synthesis without using the _/th line pixel;

705, assign '+l to the newest', §Ρ · = · ₊ 1, and then transfer to step 702;

706, using the _/_ pixel pixel view synthesis;

707. Assign +1 to the newest one, ie, == + 1, and then go to step 702.

It can be seen that when the encoder needs to update the synthesized viewpoint with the information of a certain depth image encoding unit, first, the condition C is determined for each row of the depth image encoding unit, and if all the depth information of a certain row satisfies the condition C, The line is not updated with the synthetic view point, otherwise the line needs to be used to update the composite view. The technical scope (domain) to which the present technical solution is applied is a viewpoint synthesis process in the process of encoding depth information. In the process of encoding depth information to calculate distortion, existing methods use each line of pixels of each depth image coding unit for view synthesis. In the proposed scheme, it is first determined whether each row of the depth image coding unit has an influence on the view synthesis, and if there is no influence, the row is skipped during the view synthesis process to reduce the coding time complexity.

Experiments show that this technical solution can reduce the total time complexity of the coding end by more than 10% for the sequence of 1024x768 without losing coding performance.

Those skilled in the art can also appreciate that the various illustrative logical blocks, units, and steps listed in the embodiments of the present invention can be implemented by electronic hardware, computer software, or a combination of the two. To clearly illustrate the interchangeability of hardware and software, the above-described illustrative components, units, and steps have generally described their functions. Whether such functionality is implemented by hardware or software depends on the design requirements of the particular application and the overall system. A person skilled in the art can implement the described functions using various methods for each specific application, but such implementation should not be construed as being beyond the scope of the embodiments of the present invention.

The various illustrative logic blocks, or units described in the embodiments of the invention may be implemented by a general purpose processor, a digital signal processor, an application specific integrated circuit (ASIC), a field programmable gate array.

(FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination of the above, is designed to implement or operate the described functionality. The general purpose processor may be a microprocessor, which may alternatively be any conventional processor, controller, microcontroller or state machine. The processor may also be implemented by a combination of computing devices, such as a digital signal processor and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a digital signal processor core, or any other similar configuration. achieve.

The steps of the method or algorithm described in the embodiments of the present invention may be directly embedded in hardware, a software module executed by a processor, or a combination of the two. Software modules can be stored in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, hard disk, Mobile disk, CD-ROM or any other form of storage medium in the field. Illustratively, the storage medium can be coupled to the processor such that the processor can read information from the storage medium and can write information to the storage medium. Alternatively, the storage medium can also be integrated into the processor. The processor and the storage medium may be disposed in an ASIC, and the ASIC may be disposed in the user terminal. Alternatively, the processor and the storage medium may also be disposed in different components in the user terminal.

In one or more exemplary designs, the above-described functions described in the embodiments of the present invention may be implemented in hardware, software, firmware, or any combination of the three. If implemented in software, these functions can be stored on a computer readable medium or transmitted in a form of one or more instructions or code to a computer readable medium. Computer readable media includes computer storage media and communication media that facilitates the transfer of computer programs from one place to another. The storage medium can be any available media that can be accessed by any general or special computer. For example, such computer-readable media can include, but is not limited to, RAM, ROM, EEPROM, CD-ROM or other optical disk storage, disk storage or other magnetic storage device, or any other device or data structure that can be used for carrying or storing Other media that can be read by a general purpose or special computer, or a general purpose or special processor. In addition, any connection can be appropriately defined as a computer readable medium, for example, if the software is from a website site, server or other remote resource through a coaxial cable, fiber optic computer, twisted pair, digital subscriber line (DSL) Or wirelessly transmitted in, for example, infrared, wireless, and microwave, is also included in a defined computer readable medium. The disks and discs include compact disks, laser disks, optical disks, DVDs, floppy disks, and Blu-ray disks. Disks typically replicate data magnetically, while disks typically optically replicate data with a laser. Combinations of the above may also be included in a computer readable medium.

The specific embodiments of the present invention have been described in detail with reference to the preferred embodiments of the present invention. The scope of the invention, any modifications, equivalents, improvements, etc., made within the spirit and scope of the invention are intended to be included within the scope of the invention.

Claims

Claim

A method for synthesizing a view when encoding depth information, wherein the method for synthesizing view when encoding depth information includes:

2. The method of synthesizing the depth of the information according to claim 1, wherein the determining whether each row of pixels of the depth image encoding unit causes a change in the pixel value of the synthesized viewpoint comprises:

According to the relationship between the depth information of the pixel and the disparity information, if it is determined that the depth information distortion of a certain line of pixels of the depth image encoding unit does not cause parallax distortion, it is determined that the line pixel does not cause a change in the synthesized view pixel value.

The method of synthesizing the depth information when the depth information is encoded according to claim 1, wherein the determining whether each row of pixels of the depth image encoding unit causes a change in the pixel value of the synthesized viewpoint includes:

According to the relationship between the gradient characteristics of the texture image and the synthesized viewpoint distortion, if it is determined that the distortion of the parallax of a certain row of pixels of the depth image encoding unit does not affect the synthetic viewpoint distortion, it is determined that the line pixel does not cause a change in the synthesized viewpoint pixel value.

Determining whether a pixel in a composite view corresponding to a certain row of pixels of the depth image coding unit is occluded; if it is determined that a pixel in a composite view corresponding to a certain row of pixels of the depth image coding unit is occluded, determining the depth image coding unit further Whether the pixel in the corresponding synthesized viewpoint is still occluded after the row pixel is distorted; if it is determined that the row of the depth image encoding unit is missing If the pixels in the corresponding synthesized viewpoint are still occluded, it is determined that the row of pixels does not cause a change in the composite viewpoint pixel value.

5. A viewpoint synthesis apparatus for encoding depth information, wherein the viewpoint synthesis apparatus for encoding depth information comprises:

a determining unit, configured to determine, during the process of synthesizing the depth information, whether each row of pixels of the depth image encoding unit causes a change in the pixel value of the synthesized viewpoint;

6. The apparatus according to claim 5, wherein the determining unit comprises:

a first determining module, configured to determine, according to the relationship between the depth information of the pixel and the disparity information, if it is determined that the depth information distortion of a certain row of pixels of the depth image encoding unit does not cause parallax distortion, determining that the row of pixels does not cause a change in the synthesized viewpoint pixel value .

7. The apparatus according to claim 5, wherein the determining unit comprises:

a second determining module, configured to determine, according to the relationship between the gradient characteristic of the texture image and the synthetic viewpoint distortion, if it is determined that the distortion of the parallax of a certain row of pixels of the depth image encoding unit does not affect the synthetic viewpoint distortion, determining that the row of pixels does not cause synthesis The change in the pixel value of the viewpoint.

8. The apparatus according to claim 5, wherein the determining unit comprises:

a third determining module, configured to determine whether a pixel in a synthesized view corresponding to a certain row of pixels of the depth image encoding unit is occluded; if it is determined that a pixel in a synthesized view corresponding to a certain row of pixels of the depth image encoding unit is occluded, Determining whether a pixel in the corresponding synthesized viewpoint is still occluded after the row pixel of the depth image encoding unit is distorted; if the depth image encoding sheet is determined After the pixel of the element is distorted, the pixel in the corresponding synthesized viewpoint is still occluded, and it is determined that the row of pixels does not cause a change in the synthesized viewpoint pixel value.

An encoder, characterized in that the encoder comprises a viewpoint synthesizing device for encoding depth information as claimed in any one of claims 5-8.