CN102714742A - Method for generating, transmitting and receiving stereoscopic images, and related devices - Google Patents

Abstract

The present invention relates to a method for generating a stereoscopic video stream (101) comprising composite images (C) which comprise information about a right image (R) and a left image (L). According to the method, pixels are selected from the right image (R) and from the left image (L), and then the selected pixels are entered into a composite image (C) of the stereoscopic video stream. The method also provides for entering all the pixels of the right image (R) and all the pixels of the left image (L) into the composite image (C) by leaving one of said two images unchanged and breaking up the other one into regions (Rl, R2, R3) comprising a plurality of pixels. Said regions are subsequently entered into the composite image (C). The invention also relates to a method for reconstructing the right and left images starting from a composite image, as well as to devices allowing said methods to be implemented.

Description

Be used to generate, transmit and receive the method and the relevant device of stereo-picture

Technical field

The present invention relates to generation, storage, transmission, reception and the reproduction of stereo video streaming (that is, when in visualization device, suitably handling, producing the video flowing that is perceived by a viewer as three-dimensional image sequence).

As know, can be through reproducing two images, right eye and another left eye to the beholder to the beholder obtains three-dimensional perception.

Therefore, stereo video streaming transmits the information about two image sequences, and it is corresponding to the right side and the LOOK LEFT of object or scene.

The present invention relates to the method and apparatus of two images (after this being called right image and left image) that are used for the interior right side of multiplexing combination picture and LOOK LEFT especially, and combination picture is represented the stereo video streaming frame of (after this being called the container frame).

In addition, the invention still further relates to and be used for the said combination picture of demultiplexing, promptly be used for extracting the method and apparatus of the right and left image of multiplexing equipment input from it.

Background technology

In order to prevent that TV signal from transmitting and radio network (no matter land or satellite network) is transshipped, the known single combination picture that right and left image is multiplexed with stereo video streaming in this area.

First example is that so-called (side-by-side) side by side is multiplexing, and its right-of-center in political views's image and left image are owed sampling (undersample) by level and be arranged side by side in the same number of frames of stereo video streaming.

The type multiplexing has following shortcoming: horizontal resolution reduces by half, and vertically resolution remains unchanged.

Another example is that (top-bottom) is multiplexing at the bottom of the so-called top, and its right-of-center in political views's image and left image are vertically owed to sample and with in the same number of frames that on another top, is arranged in stereo video streaming.

This type of is multiplexing to have a following shortcoming, and vertically resolution reduces by half and horizontal resolution remains unchanged.

Also there are other more complicated methods, such as disclosed a kind of method among the patent application WO03/088682 for example.This application has been described and has been used the chessboard sampling so that confirm to form the pixel quantity of right and left image.Will to the right pixel of selecting with the frame of left image " on how much " boil down to side by side form (through remove the pixel filling of blank that respective pixel produces in row 1, by that analogy) by row 2.Be used for during the decoding step that presents image on the screen, the frame of right and left image is being brought back to their unprocessed form, and through using the pixel that suitable interpolation technique comes reconstructing lost.This method allows level and vertically the maintenance of the ratio between the resolution is constant, but it has reduced diagonal resolution and has also changed the correlation in the image pixel through the high frequency spatial spectral component of introducing otherwise will lacking.This can reduce the efficient of compression step (for example, MPEG2 or MPEG4 or H.264 compression) subsequently, and also increases the bit rate of compressing video frequency flow.

Become known for the additive method of the multiplexing right side and left image according to patent application W02008/153863.

One of these methods provide 70% convergent-divergent of carrying out right and left image; Picture breakdown with convergent-divergent is the piece of 8 * 8 pixels then.

The piece of each zoomed image can equal the zone of half combination picture approximately by boil down to.

This method has following shortcoming: the redistribution of piece has been revised the spatial coherence in the piece of composition diagram picture through introducing the high frequency spatial spectral component, thereby has reduced compression efficiency.

And the zoom operations of each image and each image segmentation are that a large amount of pieces relates to that height assesses the cost and therefore increased the complexity of multiplexing reconciliation multiplexing equipment.

In these methods another is applied to each right and left image with the diagonal convergent-divergent, makes original image is deformed into parallelogram.Two parallelogram are broken down into triangle subregion (region) then, and form the rectangle combination picture, wherein discern and arrange again through decomposing the triangle subregion that two parallelogram obtain.Organize the triangle subregion of right and left image as follows, the feasible diagonal through combination picture of this mode separates them.

As the end, top and scheme side by side, this scheme has also received following shortcoming: the ratio (balance) between change level and the vertical resolution.In addition; Being subdivided into a large amount of triangle subregions that are arranged in again in the three-dimensional frame makes subsequently compression step (for example, MPEG2, MPEG4 or H.264) in the borderline region between the triangle subregion before the transmission on the communication channel, generate counterfeit picture (artifact).Said counterfeit picture for example can be produced by the motion estimation process according to the H.264 compression process execution of standard.

Other shortcomings of this scheme relate to the operation that is used for the convergent-divergent right side and left image and are used for segmentation of diabolo subregion and the required computational complexity of operating afterwards of rotating translation (rototranslate).

The purpose of this invention is to provide allow to overcome the prior art shortcoming, be used for multiplexing and demultiplexing is right and the multiplexing method of left image and Deplexing method (and relevant device).

Especially, one of the object of the invention provide the balance that allows between reservation level and the vertical resolution, be used for multiplexing and demultiplexing is right and the multiplexing method of left image and Deplexing method (and relevant device).

Another object of the present invention provides and allows subsequently the applying high voltage shrinkage and multiplexing method generation, that be used for the multiplexing right side and left image (and relevant device) of minimal distortion or counterfeit picture.

Other purposes of the present invention provide multiplexing method that assesses the cost and the Deplexing method (and relevant device) that is characterised in that minimizing.

Characteristic through merging record in the appended claims (being intended to intact part) as this specification, be used for multiplexing and demultiplexing is right and the multiplexing method of left image and Deplexing method (and relevant device) realize of the present invention these with other purposes.

Summary of the invention

The general thoughts on the present invention basis is that two images are got in the combination picture of its pixel quantity more than or equal to the pixel sum of treating two multiplexing images (for example, right image and left image).

Make the pixel of first image (for example, left image) get into combination picture and without successive what change, and second image is subdivided into the subregion of its pixel arrangement in the free space of combination picture.

This scheme provides one of two images to keep immovable advantage, and this has caused rebuilding the more good quality of figure.

Advantageously, be the subregion of minimum possibility quantity then with second picture breakdown, thus the generation of the counterfeit picture during spatial coherence in the maximization pixel and the minimizing compression stage.

In advantageous embodiments, only make the subregion of second image get into combination picture, so the ratio between maintenance level and the vertical resolution does not change by means of translation or rotation translation.

In other embodiments; At least one experience mirror-inverted (specular inversion) operation of the subregion that second image has been broken down into (promptly; (particularly on one side) puts upside down it with respect to an axle) and be arranged in such a way in combination picture; This mode make since corresponding (homologous) pixel that is present in two right sides and left image (promptly; Be positioned at the pixel of two images in the identical row and column) between strong correlation, one side one of its limit with bordering on (the border on) of another image of having identical or similar pixel on the limit.

This solution provides and in borderline region, has reduced the advantage that counterfeit picture produces.The ground that more has superiority, the subregion that second image is subdivided into has rectangular shape; The solution that is furnished with the triangle subregion of the borderline region that on diagonal, passes combination picture with use is compared; This selection provides and has reduced the counterfeit picture that subsequent compression produces; If especially the latter acts on the square (for example, to standard H.264 16 * 16) of pixel.

According to particularly advantageous embodiment, redundant through in combination picture, introducing, promptly through duplicating that some pixel groups several times further reduces or even eliminating the formation of counterfeit picture fully.Especially, be decomposed into the subregion with following such size and reach this point through waiting to get into primary image in the combination picture, this size makes the sum of all pixels of these subregions surpass the pixel quantity of image to be decomposed.In other words, image is broken down into wherein at least two subregions that comprise the image common ground.Common images partly is the borderline region between the subregion adjacent one another are in the disassembly diagram picture.The size of this common ground preferably depends on the follow-up compression type that will be applied to combination picture, and can serve as rebuild disassembly diagram as the time buffer area that will partially or even wholly remove.Since compression maybe through eliminate the buffer area or at least its outermost portion in the borderline region of said subregion, introduce counterfeit picture, so the image of eliminating any counterfeit picture and rebuilding faithful to original image is possible.

According to the following description of some embodiment of the present invention that provides through non-limiting example, other purposes of the present invention and advantage will become clearer.

Description of drawings

To said embodiment be described with reference to accompanying drawing, wherein:

Fig. 1 shows the block diagram that right image and left image is multiplexed with the equipment of combination picture;

Fig. 2 is the flow chart of the method carried out of the equipment of Fig. 1;

Fig. 3 shows first form of disassembling that gets into the image in the combination picture.

Fig. 4 shows the phase I of constructing combination picture according to one embodiment of present invention.

Fig. 5 shows the complete combination picture of Fig. 4.

Fig. 6 shows second form of disassembling with the image that gets into combination picture.

Fig. 7 shows the combination picture of the image that comprises Fig. 6.

Fig. 8 shows the 3rd form of disassembling with the image that gets into combination picture.

Fig. 9 shows the combination picture of the image that comprises Fig. 8.

Figure 10 shows the block diagram of the receiver that is used to receive the combination picture that generates according to the method for the invention.

Figure 11 shows that reconstruction is disassembled according to the method for Fig. 8 and gets into some stage of the image of the combination picture that the receiver by Figure 10 receives.

Figure 12 is the flow chart of method that is used for rebuilding the right side and the left image of the combination picture that is multiplexed into type shown in Fig. 9.

Figure 13 shows the combination picture according to fourth embodiment of the invention.

Figure 14 a shows at right image and the left image of different disposal in the stage of carrying out to right image and left image are got in the combination picture of Figure 13 to 14f.

Wherein by means of similarly, similarly structure, parts, material and/or element suitable with reference to indicating.

Embodiment

Fig. 1 shows the block diagram of the equipment 100 that is used to generate stereo video streaming 101.

In Fig. 1, equipment 100 receives two image sequences 102 and 103 that expection is used for left eye (L) and is used for right eye (R), for example two video flowings respectively.

Equipment 100 allows realization to be used for the method for two images of multiplexing two sequences 102 and 103.

In order to realize being used for the method for the multiplexing right side and left image; Equipment 100 comprises and is used for disassembling device module 104 with what input picture (the right image of the example of Fig. 1) was decomposed into a plurality of subimages; Each subimage is corresponding to a subregion that receives image, and can make the pixel that receives image get into assembler (assembler) module 105 of the single combination picture that will provide in its output place.

An example of the multiplexing method of equipment 100 realizations is described referring now to Fig. 2.

Method begins in step 200.(step 201) subsequently is decomposed into a plurality of subregions with one of two input pictures (a right or left side), as shown in Figure 3.In the example of Fig. 3, disassembly diagram similarly is the frame R of video flowing 720p, promptly has the progressive format (progressive format) of 1280 * 720 pixel resolutions, 25/30fps (frame per second).

The frame R of Fig. 3 is from carrying the video flowing 103 of image that expection is used for right eye, and to be disassembled be three subregion R1, R2 and R3.

Two parts through image R being divided into identical size and two parts that subsequently one of these parts are subdivided into identical size obtain disassembling image R.Subregion R1 has the size of 640 * 720 pixels and obtains subregion R1 through all preceding 640 pixels that obtain every row.Subregion R2 has the size of 640 * 360 pixels and obtains subregion R2 through from 641 to 720 the pixel that obtains preceding 360 row.Subregion R3 has the size of 640 * 360 pixels and obtains subregion R3 through the rest of pixels (that is from 641 to 720 of last 360 row pixel) that obtains image R.

In the example of Fig. 1, module 104 is carried out the operation of disassembly diagram as R, and module 104 receives input picture R (being frame R) and exports three number of sub images (that is three pixel groups) corresponding to three subregion R1, R2 and R3 in this situation.(step 202 and 203) subsequently, structure combination picture C, it comprises the information that relates to right and left input picture; In the example described here, said combination picture C is the frame of output stereo video streaming, and therefore it is called as the container frame.

At first (step 202); That equipment 100 receives and can't help the input picture (the left image L in the example of Fig. 1) that equipment 105 disassembles and do not enter into the container frame with changing, to come container frame sizing (size) such as the mode of all pixels that comprise two input pictures.For example; If input picture has the size of 1280 * 720 pixels; The container frame that is fit to comprise two images so will be the frame of 1920 * 1080 pixels, for example, and the frame of the video flowing of 1080p type (progressive format) with 1920 * 1080 pixels, 25/30 frame per second.

In the example of Fig. 4, left image L enters into container frame C and is positioned at the upper left corner.This obtains in zone C 1 through 1280 * 720 copied pixels with image L, and zone C 1 is made up of preceding 1280 pixels of preceding 720 row of container frame C.

When in the following description to image is got in the frame or from a frame to another frame transmit or copy pixel carry out with reference to the time, should be appreciated that to this means to carry out and generate (through using hardware and/or software service) and comprise the process with the new frame of source images same pixel.

Be used for (software and/or hardware) technology that source images (or pixel groups of source images) is reproduced as target image is regarded as inessential and will no longer further discuss for the object of the invention at this because their this know as those skilled in the art.

In next step 203, the image that module 104 is disassembled in step 201 gets in the container frame.This is through with realizing that by module 105 this zone is not taken by image L in the copied pixels of the image of disassembling its zone in the container frame C, that is, and and the zone that zone C 1 is outside.

In order to realize that the best possibly compressed and when decompressed video flows, to reduce the generation of counterfeit picture, come the pixel of the subimage of replication module 104 outputs through keeping corresponding spatial relationship.In other words, by means of translation and/or rotary manipulation copy to subregion R1, R2 and R3 in the respective regions of frame C exclusively without successive what distortion.

The example of the container frame C of module 105 outputs is shown in Fig. 5.

Subregion R1 is copied in last 640 pixels (zone C 2) of preceding 720 row the image L next door of promptly duplicating before.

Zone C 1 time, promptly in zone C 3 and C4, duplicate subregion R2 and R3 respectively, zone C 3 comprises preceding 640 pixels of last 360 row and 640 pixels afterwards respectively with C4.

As the preferred of solution shown in Figure 5 substituted, can subregion R2 and R3 be copied in the separation region of being separated by pixel groups of container frame C (that is, neither overlapping also non-conterminous), thus minimizing border subregion.

The operation that is used to make image L and R get into the container frame does not mean that level and vertically any change of balance between the resolution.

In one embodiment, distribute identical rgb value to the rest of pixels of frame C; For example, said rest of pixels can all be a black.

In other embodiments, remaining available space can be used to get into the signal that is used for and any kind that left image required right in the reconstruction of demodulation multiplexer level in combination picture, and for example how indication forms combination picture.

The subregion of the container frame that is not taken by right or left image or its part in this embodiment, is used to receive signal.The pixel of this signal subregion is for example with two kinds of colors (for example, black and white) dyeing, thereby produces the bar code of any kind of that carries signal message, and is promptly linear or two dimension.

In case two input pictures (with also being signal) are sent in the container frame and accomplish; The method that equipment 100 is realized just finishes and can compress and transmission container frame and it is recorded on the suitable media (for example, CD, DVD, blue light, mass storage etc.) on communication channel.

Because the multiplexing operation of top explanation does not change the spatial relationship among the pixel of a subregion or image, keep very faithful to image transmitted so the video flowing of equipment 100 outputs may be compressed to suitable degree and do not produce the failure-free probability of remarkable counterfeit picture ground reconstructed image.Before describing other embodiment, must be pointed out that frame R is divided into three subregion R1, R2 and R3 and is divided into minimum subregion that maybe quantity corresponding to frame, this has considered space and the left image occupation space that changes entering container frame available in the combination picture.

In other words, said minimum number is to take the minimum number that left image is stayed the required subregion in space available among the container frame C.

Generally speaking, therefore, image must by disassemble for the minimum number of subregion be defined as the function of the form of source images (right and left image) and target combination picture (container frame C).

Preferably, through consideration the needs that image (for example, the R in the last example) is decomposed into the rectangular area of minimum number are disassembled the image that gets into frame.

In other embodiments, that kind as shown in Figure 6 is disassembled right image R.

Subregion R1 ' is corresponding to the subregion R1 of Fig. 3, and therefore comprises preceding 640 pixels of all 720 row of image.

Subregion R2 ' comprises the 320 row pixels adjacent with subregion R1 ', and subregion R3 ' comprises last 320 row pixels.

Therefore that kind structure container frame C as shown in Figure 7, the subregion R2 ' and the R3 ' that rotate 90 ° will be arranged in the zone C 3 ' and C4 ' under image L and the subregion R1 '.

Therefore the subregion R2 ' of rotation and R3 ' take 720 pixels of 320 row; Therefore, zone C 3 ' is separated with C2 with zone C 1 with C4 ', and zone C 1 and C2 comprise from the image L and the pixel of duplicating from subregion R1 '.

Preferably, zone C 3 ' is separated at least one protective wire with C4 ' with other zone C 1 with C2.Especially, be favourable and preferred with the copied pixels of subregion R2 ' and R3 ' to the last multirow (last rows) of container frame C.

Because in this situation, the container frame is made up of 1080 row, so in the embodiment of Fig. 7, the subregion R2 ' of rotation has separated 40 pixel height of ceiling with R3 ' with top image L with subregion R1 '.

In the example of Fig. 7, subregion R2 ' and R3 ' are separated from one another, make they by be not from the pixel of the predefine color (for example, white or black) of right and left image around.In this way, reduced the borderline region between the subregion that comprises the pixel that comes from the right side and left image, and also reduced any counterfeit picture that causes by image compression and maximized compression ratio.

As substituting of the last multirow that R2 ' and R3 ' is navigated to container frame C (like what describing) with reference to figure 7; In a preferred embodiment, so that the mode that 32 pixel column height of ceiling are stayed between the top edge of lower limb and R2 ' and R3 ' of L is located R2 ' and R3 '.This provides 8 the pixel column height of second ceiling between the lower limb of lower limb and C of R2 ' and R3 '.Through further utilizing the width of container frame, it is possible locating R2 ' and R3 ' as follows, this mode make they fully by neither from right image also not from the pixel of left image around.

In with reference to figure 8 and Fig. 9 other embodiment described here, module 104 is extracted three number of sub images R1s ' ', R2 ' ' and the R3 ' ' of their pixel summations above the pixel summation of disassembly diagram picture.

Subregion R1 ' ' is corresponding to the subregion R1 ' of Fig. 6, and R2 ' ' and R3 ' ' comprise that the area of subregion R2 ' and R3 ' adds additional area (Ra2 and Ra3), and this allows to be minimized in the generation of counterfeit picture during the image compression stage.

Therefore, section R1 ' ' is the subregion that has 640 * 720 pixel sizes and take first row of frame R to be disassembled.

Section R3 ' ' takies the last multiple row (last columns) of frame R to be disassembled, and borders on central subregion R2 ' '.(side of bordering on R2 ' ') comprises the buffer bar Ra3 that comprises with the common pixel of subregion R2 ' ' to R3 ' ' in the left side.In other words, the preceding multiple row of the last multiple row of R2 ' ' and R3 ' ' (forming buffer bar Ra3) is overlapping.

Preferably, select the size of buffer bar Ra3 according to the compression type that is applied to container frame C subsequently and generally is applied to comprise its video flowing.Especially, said has following size, and this size is the twice of the size of the basic processing unit that in compression process, uses.For example, H.264 standard provides that image is disassembled is the macro block of 16 * 16 pixels, and each macro block is represented the basic processing unit of this standard.Based on this hypothesis, bar Ra3 has the width of 32 pixels.Section R3 ' ' therefore has the size of 352 (320+32) * 720 pixels, and comprises the pixel of last 352 row of image R.

Section R2 ' ' takies the middle body of image R to be disassembled and comprises the buffer bar Ra2 that has with the identical size of bar Ra3 in its left side.In considering the example of compression standard H.264, therefore bar Ra2 is that 32 pixels are wide and comprise the pixel common with subregion R1 ' '.Therefore section R2 ' ' has 352 * 720 pixel sizes and comprises the pixel of (R1 ' ' 640-32) row to 978 from 608 from frame R.

Three subimages (visible among Fig. 8) that relate to subregion R1 ' ', R2 ' ' and the R3 ' ' of module 104 outputs get into container frame C as shown in Figure 9 then.Subregion R2 ' ' and R3 ' ' rotate 90 ° and through providing with zone C 3 ' ' the protection pixel of certain quantity of separating with C2 with C4 ' ' and zone C 1 is with the last multirow (be indicated as being the zone of C3 ' ' and C4 ' ') of copied pixels to frame C, zone C 1 and C2 comprise the pixel of image L and R1 ' '.In situation shown in Figure 9, this ceiling is that 8 pixels are wide.

Therefore, condensed frame C and with its transmission or be saved in storage medium (for example, DVD) subsequently.For this purpose, provide be suitable for compressed image or vision signal compression set together with the device that is used to write down and/or transmit compressed image or vision signal.

Figure 10 shows the block diagram of receiver 1100, and the container frame of 1100 pairs of receptions of this receiver decompresses (if compression), rebuilds two right sides and left image and makes them available for visual device (for example, television set), and this allows the realization of 3D content.Receiver 1100 can be STB or the receiver that is built in television set.

The identical comment of carrying out to receiver 1100 (for example also is applicable to reader; The DVD reader); It reads container frame (possibly be compressed) and handles it, so that the right side in the container frame (possibly be compressed) that obtains to read corresponding to the entering reader and a pair of frame of left image.

With reference to returning Figure 10, receiver receives (via cable or antenna) compression stereo video streaming 1101 and by means of decompression module 1102 it is decompressed, thereby obtains to comprise the video flowing corresponding to the frame C ' sequence of frame C.If if having ideal communication channel or read the container frame from mass storage or data medium (blue light, CD, DVD), then frame C ' is corresponding to the container frame C that carries about the information of right and left image, except any counterfeit picture of being introduced by compression process.

To rebuilding module 1103 these frames of supply C ', rebuilding module 1103 is carried out the image rebuilding method that is described below as with reference to Figure 11 and 12 then.

Be clear that,, then can omit decompression module 1102 and can directly flow to rebuilding module 1103 supply videos if video flowing is not compressed.

When receiving the container frame C ' of decompression, process of reconstruction begins in step 1300.Rebuilding module 1103 is through extracting (step 1301) left image L with preceding 720 * 1080 copied pixels (for example, frame of 720p stream) in the new frame less than the container frame of decompressed frame.Therefore, the image L of reconstruction outputs to receiver 1100 (step 1302).

Subsequently, method provides from container frame C ' and extracts right image R.

The stage of extracting right image is included in the region R 1 ' among the frame C ' through duplicating (step 1303) ' a part begin.In more detail, the copied pixels of preceding 624 (640-16) row of R1 ' ' is in preceding 624 row of the correspondence of the new frame of expression reconstructed image R out, shown in figure 11.In fact, this for example the effect through the motion estimation process carried out by compression standard H.264 removed 16 row of R1 ' ' from phase of regeneration, their major parts have received the generation of counterfeit picture.

Then, extract the middle body (step 1304) of R2 ' '.According to the frame C ' that decompresses (as stated; It is corresponding to the frame C of Fig. 9); Select zone C 3 ' ' pixel (corresponding to source partition R2 ' ') and carry out with multiplexer 100 in 90 ° of opposite rotations of rotation of carrying out; This takes back former beginning/row condition with them, i.e. shown in Fig. 8.At this point, eliminated preceding and back ten six (16) of R2 ' ' and be listed as and all the other 352-32=320 pixel columns are copied to freely be listed as adjacent with those pixels of only duplicating from R1 ' '.

Through 16 outermost row of cutting subregion R2 ' ', eliminate those row, the formation of counterfeit picture takes place probably in those row.The width of cutting zone (being 16 row in this situation) depends on the compression type of use.Said zone preferably equals the basic processing unit that compression process is used; In the situation described here, H.264 standard is to the block operations of 16 * 16 pixels, and therefore will cut 16 and be listed as.

About R3 ' ' (step 1305), extract the pixel of subregion C4 ' ' and subimage R3 ' ' is brought back to former beginning/column format (referring to Fig. 8) from frame C '.Subsequently, eliminate preceding 16 pixel columns (corresponding to region R a3 half the) and all the other 352-16=336 pixel column is copied to the last a plurality of of the reconstruction frames left side freely be listed as.Such as R2 ' ', also in R3 ' ', cutting zone equals the basic processing unit that compression process is used.

Certainly, for subregion R2 ' ' and R3 ' ' both, can carry out rotary manipulation with virtual mode; Promptly; Identical result aspect the extraction of interested pixel can be through (if R2 ' ' the then is) zone C 3 ' in the row that in reconstruction frames, duplicate new frame Rout ' pixel of the row of (if R3 ' ' then is C4 ' ') obtains; Except (if R2 ' ' corresponding to 16 row to be cut; Then be) zone C 3 ' ' (if R3 ' ' then is C4 ' ') last 16 the row, shown in Fig. 8.

In this, right image Rout has been rebuild fully and can be by output (step 1306).

Be used for reconstruction and be included in the right side of container frame C ' and the therefore completion (step 1307) of process of left image.Each frame of the video flowing that receives to receiver 1100 repeats said process, makes output to form with two video flowings 1104 and 1105 that are used for left image by being respectively applied for right image.

The process of rebuilding the right side and left image in the above with reference to Figure 10,11 and 12 being used to of describing is based on following hypothesis: demodulation multiplexer 1100 know container frame C be how to make up and therefore can extract right and left image.

Certainly, standardization multiplexing method how, then this is possible.

For consider can with in the said method any or in a word according to utilizing the fact that generates the container frame as any method of the solution of appended claims theme; Demodulation multiplexer use in the predefine zone be included in combination picture signaling information (for example; Aforesaid bar code), must be how the content of combination picture be unpacked and how to rebuild right and left image so that know.

After the said signal of decoding; Demodulation multiplexer will be known and (for example not change image; Left image in the above-mentioned example) position, and other images (for example, the right image in the above-mentioned example) by disassemble into position and any conversion (rotation, translation etc.) of subregion.

Utilize this information, therefore demodulation multiplexer can extract unaltered image (for example left image) and rebuild the image of disassembling (for example, right image).

Although preferably show the present invention everywhere with advantageous embodiment with reference to some; But should be clear, the invention is not restricted to the technical staff that two images that said embodiment and hope will be referred to two different visual angles (right and a left side) of object or scene merge in the field of combination picture and can make a lot of changes it.

For example, provide the electronic module of the said equipment, particularly equipment 100 and receiver 1100 can be by various segmentations and distribution; In addition, can they be provided with the form of hardware module or as the software algorithm that realizes by processor (particularly being equipped with the video processor in the suitable memory zone of the incoming frame that is used for interim storage reception).Therefore these modules can walk abreast or serial execution image according to the present invention is multiplexing and one or more Video processing steps of demultiplexing.Represent two 720p video flowings are multiplexed with a 1080p video flowing although also will be clear that preferred embodiment, also can use extended formatting.

The present invention also is not limited to the specific arrangements type of combination picture, can have concrete advantage because be used to generate the different solutions of combination picture.

For example, referring to figs. 1 to 12 embodiment that describe following advantage is provided in the above: they only carry out translation or rotation translation, therefore only need little rated output.

Alternatively, can expect except said rotation and/or translation that image also receives the mirror-inverted operation, so that obtain the combination picture type shown in Figure 13.

Comprise the border girth between the subregion of respective pixel from maximization, thereby adopt the purpose that is present in the strong correlation among them and minimizes the counterfeit picture that subsequent compression introduces, carry out these additional operations.In the example of Figure 13 and 14, supposed that for clear two right sides are identical with left image, even they are generally different slightly.

In the figure, left image L (shown in Figure 14 a) is positioned at the upper right corner of container frame C, thereby takies last 1280 pixels of preceding 720 row.As in the example of describing in front, therefore image L is not copied among the container frame C with changing.

Instead, disassemble right image R according to the example of Fig. 3; Figure 14 b shows the image R that is broken down into three subregion R1, R2 and R3.

Subsequently, some subregion (subregion R1 and R3 in the example of Figure 14) has experienced minute surface and has operated anyway; Counter-rotating can take place with respect to vertical axes (row that promptly are parallel to image) or with respect to trunnion axis (row that promptly is parallel to image).

In the situation with respect to the counter-rotating of vertical axes, the pixel of row N (wherein N is the integer between 1 and 1080, the 1080th, the columns of image) is copied to and is listed as among the 1080+1-N.

In situation with respect to the counter-rotating of trunnion axis, and the pixel of row M (wherein M is the integer between 1 and 720, the 720th, and the line number of image) be copied among the capable 720+1-N.

Figure 14 c and 14d show extract from image R and with respect to the vertical axes counter-rotating, particularly with respect to the subregion R1 of vertical edge counter-rotating (R1rot).

Preceding 640 pixels that make the subregion R1inv of counter-rotating get into preceding 640 pixel columns.

As can be seeing in the example of Figure 13, when R1inv rotated among the container frame C, the pixel of the R1inv of bordering on L be similar to the pixel of the L of bordering on R1inv very much.Spatial coherence among these pixels has the advantage that reduces counterfeit picture formation.

Figure 14 and 14f show extract from the image R of Figure 14 b and then with respect to trunnion axis, particularly with respect to the subregion R3 of horizontal sides counter-rotating (R3inv).

Subregion R3inv gets into last 640 pixels of last 360 row.This has reduced the generation of counterfeit picture, because the pixel of the border subregion between R3inv and the L is the pixel with high spatial correlation.In fact, the pixel in this border subregion has been reproduced the similar or same section of image.

Through being got into, subregion R2 accomplishes container frame C then.

In this example, not counter-rotating and/or rotation R2 are because the border subregion that in any situation, can not make R2 is complementary with the border subregion of being made up of the respective pixel of another subregion of R or L.

At last, will be clear that also to the present invention relates to any Deplexing method that this any Deplexing method allows to extract right image and left image from combination picture through putting upside down one of above-mentioned multiplex process of dropping in the protection range of the present invention.

Therefore the present invention also relates to the method that is used for beginning to generate from combination picture a pair of image, and it may further comprise the steps:

-duplicate single group of first image (for example, left image) that generates the said right side and left image through subregion in abutting connection with (contiguous) pixel from said combination picture,

-other groups of duplicating adjacent pixels through the different subregions from combination picture generate second image (for example, right image).

According to an embodiment, be used to generate the extracted region of the information of said second image from said combination picture.Preferably said information is encoded according to bar code.

At an embodiment of the method that is used for generating right and left image, the generation of the image of in combination picture, disassembling comprises at least one stage of mirror-inverted of the pixel groups of one of said different subregions.

At an embodiment of the method that is used for generating right and left image, the generation of the image of in combination picture, disassembling comprises at least one stage that removes pixel from one of subregion of the combination picture that comprises the pixel of waiting to rebuild this image.Especially, the borderline region from this subregion removes pixel.

In one embodiment, through making the pixel partitions that comprises the pixel of treating reconstructed image only receive translation and/or selection operation is rebuild the image of being disassembled to the different subregions of combination picture.

Claims (25)

1. method that is used to generate stereo video streaming (101), said stereo video streaming (101) comprises combination picture (C), said combination picture (C) comprises the information about right image (R) and left image (L), wherein

Select the pixel of said right image (R) and the pixel of said left image (L), and

Make said selected pixel enter into the combination picture of said stereo video streaming (C),

Said method is characterized in that through one of said two images not being changed and another being decomposed into the subregion (R1 that comprises a plurality of pixels; R2, R3) and all pixels that said subregion got into make said right image (R) in the said combination picture (C) and all pixels of said left image (L) enter into said combination picture (C).

2. method according to claim 1 is wherein through considering space available in said combination picture (C) and be the subregion of minimum possible quantity with said another picture breakdown by keeping a unaltered said image (L) occupation space.

3. method according to claim 2, wherein said minimum number are to take the said required minimum number of free space that image stays that do not change.

4. according to each described method in the aforementioned claim, wherein obtain said subregion through following steps:

Said another image (R) is divided into the part of two equal sizes,

One of said two parts are divided into the part of two equal sizes.

5. according to each described method in the aforementioned claim, (R1, R2 R3) get in the said combination picture wherein only to make said subregion by means of translation.

6. according to each described method in the claim 1 to 4, (R1, R2 R3) get in the said combination picture (C) wherein to make said subregion by means of translation and/or rotary manipulation.

7. according to each described method in the aforementioned claim, wherein in said combination picture, keep freely at least a portion in space to be used to be used in and rebuild the said right side and the required signal of left image gets in the demodulation multiplexer level.

8. according to each described method in the aforementioned claim, wherein, carry out the mirror-inverted operation along one side of a said subregion making before one of said subregion gets in the said combination picture.

9. method according to claim 8, one side a wherein said subregion gets in the said combination picture and on one side with the bordering on of another image or subregion, making is arranged side by side relates to the pixel in same space zone.

10. according to each described method in the aforementioned claim, wherein said subregion has rectangular shape.

11. according to each described method in the aforementioned claim, wherein said subregion comprises the contiguous set of row of the pixel of said image.

12. according to each described method in the aforementioned claim, at least two of wherein said subregion have at least one common pixel groups, said pixel groups is arranged in the borderline region between said two said subregions at least.

13. according to each described method in the aforementioned claim, at least one that wherein gets in the said subregion in the said combination picture leaves with comprising other division of said combination picture from the pixel of said right image or said left copying image.

14. according to each described method in the aforementioned claim, wherein

Receive the sequence of right image and the sequence of left image,

Begin to generate the sequence of combination picture through said sequence from right and left image,

The said sequence of compression combined image.

15. one kind through beginning to rebuild the method for a pair of image from combination picture, may further comprise the steps:

-duplicate single group of first image that generates the said right side (R) and a left side (L) image of adjacent pixels through subregion from said combination picture,

-through (other groups of R3) duplicating adjacent pixels generate second image of the said right side (R) and left (L) image for R1, R2 from the different subregions of said combination picture.

16. method according to claim 15, wherein the extracted region from said combination picture is used to generate said information to image.

17. method according to claim 20 is wherein encoded to said information according to bar code.

18. according to claim 15 or 17 described methods, the generation of wherein said second image comprises at least one at least one stage of mirror-inverted of pixel groups in the said different subregion.

19. according to each described method in the claim 15 to 18, the generation of wherein said second image comprises that from said subregion at least one removes at least one stage of pixel.

20. method according to claim 19, wherein the borderline region from said at least one subregion removes pixel.

21., wherein generate said second image through making said pixel partitions only receive translation according to each described method in the claim 15 to 20.

22. according to each described method in the claim 15 to 21, wherein through said pixel partitions being rotated and/or translation generates said second image.

23. equipment (100) that is used to generate combination picture (C); Comprise the device (104) that is used to receive right image and left image and be used to generate the device (105) that comprises about the combination picture (C) of the information of said right image and said left image, it is characterized in that comprising being suitable for realizing device according to each described method in the claim 1 to 14.

24. one kind is used for through begin to rebuild the equipment (1100) of a pair of image from combination picture, it is characterized in that realizing according to each described method in the claim 15 to 22.

25. a stereo video streaming (1101) is characterized in that comprising at least one combination picture (C) that generates by means of according to each described method in the claim 1 to 14.

Images