Classifications

• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
  • H04N13/10—Processing, recording or transmission of stereoscopic or multi-view image signals
  • H04N13/106—Processing image signals
  • H04N13/128—Adjusting depth or disparity
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
  • H04N13/10—Processing, recording or transmission of stereoscopic or multi-view image signals
  • H04N13/106—Processing image signals
  • H04N13/111—Transformation of image signals corresponding to virtual viewpoints, e.g. spatial image interpolation
• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T7/00—Image analysis
  • G06T7/50—Depth or shape recovery
  • G06T7/55—Depth or shape recovery from multiple images
  • G06T7/593—Depth or shape recovery from multiple images from stereo images
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
  • H04N13/10—Processing, recording or transmission of stereoscopic or multi-view image signals
  • H04N13/106—Processing image signals
  • H04N13/122—Improving the 3D impression of stereoscopic images by modifying image signal contents, e.g. by filtering or adding monoscopic depth cues
• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T2207/00—Indexing scheme for image analysis or image enhancement
  • G06T2207/10—Image acquisition modality
  • G06T2207/10004—Still image; Photographic image
  • G06T2207/10012—Stereo images
• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T2207/00—Indexing scheme for image analysis or image enhancement
  • G06T2207/10—Image acquisition modality
  • G06T2207/10028—Range image; Depth image; 3D point clouds
• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T2207/00—Indexing scheme for image analysis or image enhancement
  • G06T2207/20—Special algorithmic details
  • G06T2207/20024—Filtering details

Definitions

• the present invention relates to a stereoscopic image and more particularly to a method and a device for filtering the disparity map associated with such an image.
• 3D view synthesis or 3D view interpolation is to create a 3D rendering other than that represented by the source stereoscopic image.
• the 3D rendering then adjusts to the chosen application or the user.
• the views needed to address multi-view screens are mostly recreated from a single stereoscopic image.
• this synthesis of view is based on the source stereoscopic image and on a map of disparity.
• the disparity map is generally associated with a right or left source image in the case of a stereoscopic image and provides for each pixel a datum representing the distance (in number of pixels) between the position of this image element in the right image and in the left image.
• This view synthesis can also be based on a depth map because it is possible to return to a depth value from the disparity value, or vice versa, by introducing certain data, such as the size of the screen or the distance between the observer and the screen using an algorithm known to those skilled in the art.
• this disparity map is projected at the level of the view you want to generate. After this projection, we therefore have a modified disparity map corresponding to the desired view, each pixel of this disparity map modified indicates a disparity value which will then be used to adapt the information corresponding to the right or left image to regenerate.
• the disparity vector of each pixel to be interpolated must point to the same points of objects in the right and left images.
• Document US2010 / 0215251 describes a method and a device for processing a depth map.
• the method consists in obtaining a depth map based on a compressed depth map including the depth information of a captured scene.
• the scene has an object.
• Occlusion information including information occluded by the object in the depth map is provided. At least a portion of the occlusion information is then used to reduce the distortions of the artifacts in the depth map.
• This document therefore relates to an information processing method for the correction of a depth map related to an area obscured by an object.
• our invention aims to overcome a different problem which concerns so-called transition pixels located on the edges of objects. If, for example, a scene has two flat objects located at fixed and distinct depths, the disparity map associated with the elements of the right view or the left view should contain only two values, the value of the disparity of the first object. and that distinct from the disparity of the second object since there are only two possibilities in the scene with two different disparities.
• the disparity map containing only two disparity values it is in this case easy to project the disparity map on an intermediate view between the left view and the right view and thus to interpolate one of the two images.
• transitions results from a mixture between the disparity value of the pixel of the first object and that of the second object.
• This problem of erroneous or outlier disparity value is present in the disparity maps obtained by a disparity map estimator and in disparity maps of the CGI contents.
• a possible solution to the above-mentioned problem would be to use unfiltered disparity maps (that is, crenellated at the contours of the objects). Unfortunately if it is possible to consider this kind of map for CGI content, disparity estimators will most often naturally generate intermediate values between different objects.
• the present invention provides a method for overcoming the disadvantages mentioned.
• the invention consists of a filtering method of a disparity map associated with one of the views of a stereoscopic image comprising at least one object partially covering an area obscured by this object.
• the method comprises the steps of determining at least one transition zone around the object, identifying the pixels of this transition zone whose disparity value is considered as erroneous compared with the disparity values of the neighboring pixels.
• This method therefore allows intelligent filtering that only filters out doubtful disparity values and thus preserves the majority of the disparity map.
• the step of identifying pixels having an erroneous disparity value consists in identifying as erroneous any pixel whose disparity value is not between the disparity values of the neighboring pixels (x n- i, x n + i) ⁇
• the step of identifying pixels having an erroneous disparity value consists in identifying as erroneous any pixel whose disparity value is different from the disparity values of the neighboring pixels (x n- i, x n + i) of a determined threshold value.
• the step of correcting the erroneous disparity values of the pixels consists in replacing the disparity value of the pixels identified by a value calculated as a function of the disparities of the neighboring pixels.
• the step of correcting the erroneous pixel disparity values consists in eliminating the disparity value of the identified pixels.
• the step of correcting the disparity values of the pixels consists in determining which disparity value of the neighboring pixels ( ⁇ ⁇ - ⁇ , ⁇ ⁇ + ⁇ ), the disparity value of the identified pixel is the closest, then to replace the pixel disparity value identified by a replacement value corresponding to the value of one of the neighboring pixels (x i n, x n + i), the disparity value is closest.
• the method comprises the additional step, after the
• the invention also consists in a device for generating three-dimensional image display signals comprising at least one object partially covering a zone obscured by this object comprising a device for transmitting a first flow of data corresponding to one of the right or left images to be transmitted, and a second data stream corresponding to the disparity map associated with one of the right or left images to be transmitted and a device for receiving data streams for displaying three-dimensional images.
• It comprises a device for filtering the disparity map comprising means for determining at least one transition zone around the object, means for identifying the pixels (x n ) of this transition zone whose value of disparity (d (x n )) is considered erroneous by comparison with disparity values of neighboring pixels (x n- i, x n + i)
• FIG. 1 represents a system for delivering 3D content and containing a device for filtering the elements of a disparity card according to the invention, either at the level of the signal generation device or at the level of the receiver.
• FIG. 2 represents a block diagram explaining a filtering method according to the invention.
• Figure 3 shows the disparity values of a disparity map of a system without a filtering device.
• FIG. 4 represents the disparity values of a disparity card of a system with filtering device according to the invention.
• the system as represented by FIG. 1 is a device for generating display signals for at least one three-dimensional image (3D). It comprises in the transmission device a generation means 2 of two data streams representing the pixels, or picture elements (in English "picture elements" or "pixels") of the images. left and right respectively, a means 3 for generating a data stream corresponding to the difference information of the pixels of the right or left image represented by a disparity map for each image, a selection means 4 of one of the streams of data of the right or left image to be transmitted, a transmission means 5 of one of the data streams corresponding to the selected right or left image and the data flow corresponding to the disparity map.
• a generation means 2 of two data streams representing the pixels, or picture elements (in English "picture elements" or "pixels" of the images. left and right respectively, a means 3 for generating a data stream corresponding to the difference information of the pixels of the right or left image represented by a disparity map for each image, a selection means 4 of one of the streams of data of the
• a filtering device 6 disparity values of the various elements of the disparity map for the detection of erroneous values and the replacement of these values by other disparity values depending on the disparities of neighboring elements of the disparity map associated with the means of generating the data of the disparity map.
• a data receiving device receives the two data streams corresponding to one of the right or left images and the disparity map respectively and provides an interface 7 with the 3D image display signals to a screen or display device 8
• the filtering device 6 of the disparity card is situated at the level of the signal receiver device and not at the level of the transmission device.
• the method according to the invention consists in filtering the disparity map 6 so as to determine, eliminate or replace the "dubious" disparity values.
• the term "dubious" disparity value is defined as the pixel disparity values of the transition zones that do not correspond to one of the disparity values of the neighboring pixels.
• This method allows intelligent filtering that only filters out questionable disparity values and thus preserves the majority of the disparity map.
• the disparity map represents sampled information. We must therefore solve the sampling problems. This is why the method according to the invention consists in filtering the disparity map to improve the contours of objects that suffer from this lack of sampling. This filtering method must first correctly identify the erroneous disparity values, and secondly correct these values.
• the erroneous disparity values are intermediate values between actual disparity values in that these disparity values correspond to depths where an object is located. But the disparity values can be locally increasing or decreasing in the presence of a surface not parallel to the plane of the screen. The definition of the detection algorithm must take into account these different cases.
• the method proposed here is associated with a study of pixels belonging to a horizontal line along the x-axis, but it would be possible to use vertical information as well.
• the disparity values d (x n ) of pixels x n are determined as erroneous if these values differ from the disparity values d (x n- i) and d (x n + i) of the two neighboring pixels horizontally direct x n- i and x n + i of a value greater than the difference
• Another criteria can be used to define what will be considered as erroneous disparity value.
• One method consists in comparing the disparity value of a pixel with respect to a determined threshold value as a function of the disparities of the neighboring pixels. Threshold values can thus be added. For example, it is possible to determine as erroneous pixels whose disparity values (expressed in pixels) different from more than 2 pixels with its two neighbors.
• the determination of the erroneous disparity values is therefore done by comparing these values d (x n ) with the disparity values d (x n- 2), d (x n- i), d ( x n + i) or d (x n- i), d (Xn + i), d (x n + 2) of four other neighboring pixels x n- 2 x n -i n + ix n + 2 ⁇
• the first step 100 corresponds to identifying the pixels N pixels having erroneous disparity values by comparison with the disparity values of the neighboring pixels N-1 and N + 1:
• Steps 101 and 102 then make it possible to determine whether the disparity d (x n ) of the pixel x n lies between those d (x n- i) and d (x n + 1) of the pixels x n- i and x n + i and, in step 101 and step 102, comparing the disparities of pixels x n- i and x n + i : d (x n- i) ⁇ d (Xn) ⁇ d (x n + i) or (x n- i)> d (Xn)> d (x n + i)
• Step 103 corresponds to the case where the value of the disparity is not between those of the neighboring pixels. A correction of this value is necessary. This value can be replaced by the nearest disparity value or deleted.
• steps 104 and 105 the disparity gap between the pixels x n and x n- i is compared with that between x n and x n + i. These steps correspond to a comparison study determining if the disparity value of x n is closer to that of x n + i or x n- i. In the case of a pixel located on the border between two objects, this amounts to determining which object this pixel is closest to:
• Step 106 makes it possible to take into account a variable disparity (increasing or
• This step therefore makes it possible to determine a replacement disparity value of (x n ) corresponding to the maximum value of the sum between the disparity d (x n- i) of the pixel x n- i and the difference between the disparities d (x n- 2) and d (x n- i) pixesl x n- 2 and x n- i.
• the disparity of replacement values (x n) are determined taking into account a variable disparity for objects corresponding to the adjacent pixels n x 2 and x n i or x n + i and x n + 2- Maximum values are determined if d (x n ) is closer to the larger value of the two disparity values. And minimal values are determined if d (x n ) is closer to the smaller value of the two disparity values.
• step 1 if the erroneous disparity value of the pixel is not greater than this maximum value determined in step 106, the erroneous disparity value of the pixel x n is replaced by the disparity value of the pixel.
• the erroneous disparity value of the pixel x n is replaced by the minimum or maximum value determined during the preceding step or by the disparity value of the next pixel x n + 1 or preceding x n i.
• the process described by this mimic can be refined by taking into account a large change of disparity or particular cases.
• the last steps 1 10 - 1 16 therefore preferably assign a new value to the disparity value according to the preceding criteria. It is alternatively possible to replace this step with a deletion of the disparity value.
• the filtering method proposed by the invention is symmetrical and therefore treats the right and left edges of the objects in the same way, although the problems related to these edges are different.
• the examples concern the interpolation of the right edge of objects in the foreground from the disparity map associated with the left view or the interpolation of the left edge of the objects in the foreground from the map of disparity associated with the right view because artifact problems are the most important.
• the method provides processing on the right edge of the objects in the foreground for the interpolation from the disparity map associated with the left view. Similarly and according to another variant of the invention, the method provides processing on the left edge of the objects in the foreground for the interpolation from the disparity map associated with the right view.
• This dissociation taking place in the first step (d (xn-1) ⁇ d (xn) ⁇ d (xn + 1) or d (xn-1)> d (xn)> d (xn + 1)), It would suffice to retain only one of these conditions to pre-filter only one side.
• FIG. 3 represents the disparity values of a disparity map of an image comprising two elements of disparities different from a system without a filtering device. These two values are represented by a light gray and a dark gray
• FIG. 4 represents the disparity values of a disparity card of a system with filtering device according to the invention.
• the erroneous disparity values have been corrected and replaced by one of the two existing values according to the method according to the invention.
• the disparity map has only two values.

Landscapes

• Engineering & Computer Science (AREA)
• Multimedia (AREA)
• Signal Processing (AREA)
• Computer Vision & Pattern Recognition (AREA)
• Physics & Mathematics (AREA)
• General Physics & Mathematics (AREA)
• Theoretical Computer Science (AREA)
• Image Processing (AREA)
• Processing Or Creating Images (AREA)
• Testing, Inspecting, Measuring Of Stereoscopic Televisions And Televisions (AREA)

Applications Claiming Priority (2)

Publications (1)

Family

ID=47071367

Family Applications (1)

Country Status (7)

Families Citing this family (2)

Citations (3)

Family Cites Families (16)

• 2012
  • 2012-09-27 US US14/347,045 patent/US9299154B2/en not_active Expired - Fee Related
  • 2012-09-27 CN CN201280046617.6A patent/CN103828355B/zh not_active Expired - Fee Related
  • 2012-09-27 KR KR1020147011116A patent/KR20140069266A/ko active IP Right Grant
  • 2012-09-27 BR BR112014007263A patent/BR112014007263A2/pt not_active Application Discontinuation
  • 2012-09-27 EP EP12775782.1A patent/EP2761876A1/de not_active Withdrawn
  • 2012-09-27 JP JP2014532456A patent/JP2014534665A/ja active Pending
  • 2012-09-27 WO PCT/FR2012/052195 patent/WO2013045853A1/fr active Application Filing

Patent Citations (3)

Non-Patent Citations (2)

Also Published As

Similar Documents

Legal Events