CN106780383B - The depth image enhancement method of TOF camera - Google Patents
The depth image enhancement method of TOF camera Download PDFInfo
- Publication number
- CN106780383B CN106780383B CN201611149210.2A CN201611149210A CN106780383B CN 106780383 B CN106780383 B CN 106780383B CN 201611149210 A CN201611149210 A CN 201611149210A CN 106780383 B CN106780383 B CN 106780383B
- Authority
- CN
- China
- Prior art keywords
- interpolation
- pixel
- depth image
- image
- edge
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 40
- 230000003321 amplification Effects 0.000 claims abstract description 44
- 238000003199 nucleic acid amplification method Methods 0.000 claims abstract description 44
- 238000003708 edge detection Methods 0.000 claims description 11
- 238000012935 Averaging Methods 0.000 claims description 6
- 239000011159 matrix material Substances 0.000 claims description 6
- 238000001514 detection method Methods 0.000 claims description 5
- NAWXUBYGYWOOIX-SFHVURJKSA-N (2s)-2-[[4-[2-(2,4-diaminoquinazolin-6-yl)ethyl]benzoyl]amino]-4-methylidenepentanedioic acid Chemical compound C1=CC2=NC(N)=NC(N)=C2C=C1CCC1=CC=C(C(=O)N[C@@H](CC(=C)C(O)=O)C(O)=O)C=C1 NAWXUBYGYWOOIX-SFHVURJKSA-N 0.000 claims description 4
- 238000005070 sampling Methods 0.000 claims description 4
- 230000000007 visual effect Effects 0.000 abstract description 3
- 230000002708 enhancing effect Effects 0.000 abstract description 2
- 238000003672 processing method Methods 0.000 abstract 1
- 230000000694 effects Effects 0.000 description 6
- HUTDUHSNJYTCAR-UHFFFAOYSA-N ancymidol Chemical compound C1=CC(OC)=CC=C1C(O)(C=1C=NC=NC=1)C1CC1 HUTDUHSNJYTCAR-UHFFFAOYSA-N 0.000 description 3
- 241000208340 Araliaceae Species 0.000 description 2
- 235000005035 Panax pseudoginseng ssp. pseudoginseng Nutrition 0.000 description 2
- 235000003140 Panax quinquefolius Nutrition 0.000 description 2
- 230000007547 defect Effects 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 238000002474 experimental method Methods 0.000 description 2
- 235000008434 ginseng Nutrition 0.000 description 2
- 230000003993 interaction Effects 0.000 description 2
- 238000011161 development Methods 0.000 description 1
- 235000013399 edible fruits Nutrition 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 238000011835 investigation Methods 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 230000003014 reinforcing effect Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/70—Denoising; Smoothing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T3/4007—Scaling of whole images or parts thereof, e.g. expanding or contracting based on interpolation, e.g. bilinear interpolation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T3/4023—Scaling of whole images or parts thereof, e.g. expanding or contracting based on decimating pixels or lines of pixels; based on inserting pixels or lines of pixels
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T3/403—Edge-driven scaling; Edge-based scaling
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Image Processing (AREA)
Abstract
The present invention relates to a kind of depth image enhancement methods of TOF camera, belong to depth image processing method.Depth image is subjected to noise suppressed processing first, then using the distance between pixels and edge directional information in image, interpolation amplification is carried out to the TOF depth image of low resolution.The present invention improves depth image resolution ratio, improves depth image visual quality, can be applied to the depth image enhancing occasion of the acquisitions such as TOF camera, and depth image interpolation algorithm therein can also be applied to the interpolation amplification occasion of other images.
Description
Technical field
The present invention relates to the method that the depth image of a kind of pair of TOF camera acquisition is enhanced, in particular to a kind of utilizations
The method that distance between pixels and edge directional information carry out TOF depth image interpolation amplification in image.
Background technique
Each of depth image pixel indicates that some point in scene arrives the distance of camera imaging plane, it describes three
The geological information for tieing up scene, is always the emphasis and hot spot of the research of computer vision field.
Based on the three-dimensional ranging camera of flight time (TOF) principle, after being emitted to body surface by measurement active light
The reflected time difference calculates the range information of body surface, can collect the three-dimensional geometry letter of body surface in real time
Breath, is a kind of breakthrough to conventional three-dimensional information acquisition method, certainly will promote the development of related fields.With stereovision technique phase
Than not limited by baseline length, image taking speed is fast, small by texture effects；Compared with scanning laser radar, image taking speed
Fastly, small in size.The depth information that TOF three-dimensional camera obtains can be applied to human-computer interaction, 3D reconstruction, navigation, ranging, 3D freedom
The numerous areas such as video.
It can be seen that the depth information that TOF three-dimensional camera obtains plays very important effect, the quality pair of depth image
It is had a very big impact in the treatment effect of follow-up work.The depth map of high quality is that base is established in finer human-computer interaction
Plinth.
But the obtained depth image of TOF camera has some defects below: the resolution ratio of depth image is far away from common coloured silk
The resolution ratio of chromatic graph picture；The depth value of depth image is by significant noise jamming；Depth of the depth image in the edge of object
Easily there is error in angle value.These defects bring some limitations to the marketing and application of TOF camera.Therefore, for TOF phase
The problems such as machine depth image resolution ratio is lower, and depth value is significant by noise jamming, the enhancing technology of the depth of investigation image improve
TOF depth image quality is of great significance.
Summary of the invention
The present invention proposes a kind of depth image enhancement method of TOF camera, it is therefore an objective to improve depth image resolution ratio, improve
Depth image visual quality.
The technical solution adopted by the present invention is that, including the following steps:
(1), noise suppressed is carried out to the depth image that TOF camera acquires using image averaging method, improves depth map image quality
Amount；
(2) using the distance between pixels and edge directional information in image, the TOF depth image of low resolution is carried out
Interpolation amplification improves depth image resolution ratio, if depth image interpolation amplification is N × N times of original resolution ratio, carries out 2 first
Times interpolation amplification；
This process includes two parts: first is that judging that each pixel belongs to fringe region still in low resolution depth image
Smooth region；Second is that, using different interpolation methods, steps are as follows according to judging result:
(1) judges whether each pixel belongs to fringe region in low resolution depth image
3 × 3 neighborhoods using centered on low-resolution image pixel (i, j) utilize four sides as edge detection window
To template detection, whether the pixel belongs to fringe region, and detects edge direction；W1、W2、W3、W4Be respectively detect 0 °, 45 °,
90 °, extrorse 3 × 3 template in 135 ° of sides, concrete form are as follows:
The convolution value R of detection template and 3 × 3 edge detection windows are as follows:
Rm=f (i-1, j-1) Wm(1,1)+f(i-1,j)·Wm(1,2)+f(i-1,j+1)·Wm(1,3)+f(i,j-
1)·Wm(2,1)+f(i,j)·Wm(2,2)+f(i,j+1)·Wm(2,3)+f(i+1,j-1)·Wm(3,1)+f(i+1,j)·Wm
(3,2)+f(i+1,j+1)·Wm(3,3)m∈{1,2,3,4} (1)
Edge detection threshold T is set, will | Rm| maximum value be compared with T: when max (| Rm|) > T when, pixel (i,
J) belong to fringe region, | Rm| the corresponding direction of maximum value is edge direction；When max (| Rm|)≤T when, which belongs to
Smooth region；
(2) if pixel (i, j) belongs to smooth region, tri- interpolation pixels of A, B, C are calculated using following formula
Value:
(3) if pixel (i, j) belongs to fringe region, according to edge direction, A, B, C tri- is calculated using following formula
The value of a interpolation pixel:
Wherein, αKThe weighting coefficient of operation is participated in for 3 × 3 white pixel points in Interpolation Process.The present invention utilizes interpolation
Edge direction in the range information and 3 × 3 neighborhood of pixels of pixel and 3 × 3 pixel of surrounding determines weighting coefficient；
Wherein, αK_DisFor the weighting coefficient based on distance,For the weighting coefficient based on edge direction；
1)αK_Dis:
Interpolation pixel in 3 × 3 windows at a distance from white pixel point it is closer, the weight of pixel value is bigger, calculate to
The Euclidean distance square value of white pixel in interpolating pixel and 3 × 3 windows uses it reciprocal as weight coefficient, it may be assumed that
Wherein, DeFor Euclidean distance, for pixel p and q, coordinate is respectively (x, y) and (s, t), European between p and q
Distance definition is as follows:
De(p, q)=[(x-s)2+(y-t)2]1/2 (8)
2)
In order to keep the edge clear of interpolation image, interpolation arithmetic is participated according to edge direction selected section neighborhood territory pixel,
The location of pixels for participating in interpolation arithmetic is indicated using 3 × 3 two-value mask matrix, i.e., " 1 " corresponding pixel ginseng in mask matrix
With interpolation arithmetic, under four kinds of edge directions, calculate A, B, C interpolation pixel value used byForm difference is as follows:
(4) repeats above step to all pixels in low resolution depth image, completes corresponding to each pixel
Interpolation pixel A, the interpolation of B, C, can be obtained 2 times of interpolation target image of amplification；
(3) as the integral number power that amplification factor N is 2, i.e. N=2k, k is positive integer, by iteration k times in step (2), i.e.,
It can get corresponding interpolation image；When amplification factor is not 2 integral number power, i.e., 2k-1<N<2k, then will first change in step (2)
For k times, obtaining amplification factor is 2kImage, according still further to formula (10) carry out it is down-sampled can be obtained amplification factor be N figure
Picture；
Wherein, g (i, j) is down-sampled image, and interpolation amplification multiple is N times of original TOF camera sampling depth image.
It is proposed by the present invention to carry out TOF depth image interpolation side using distance between pixels in image and edge directional information
Method, compared with other depth image interpolation methods, depth image interpolation method of the invention has significant technical effect:
(1) in conventional depth image interpolation algorithm, arest neighbors interpolation is to carry out interpolation using 1 closest pixel,
Algorithm idea is simple, and speed is quite fast, but is easy to produce mosaic phenomenon；Bilinear interpolation is inserted using 4 neighborhood pixels
Value, computation complexity is slightly larger, overcomes mosaic phenomenon, but will cause depth edge excess smoothness；Bicubic interpolation
It is to have preferably kept marginal texture using 16 neighborhood pixels interpolation of surrounding, but calculated more complex.Side proposed by the present invention
Method utilizes 9 neighborhood pixels interpolation of surrounding, and computation complexity is reduced than bicubic interpolation, but can obtain better interpolation effect
Fruit.
(2) operand both can be reduced using different interpolation algorithms for the fringe region of depth image and smooth region
It can guarantee interpolation quality again.In the Interpolation Process of edge region pixel, while considering known to interpolation pixel and surrounding 3 × 3
Two factors of distance and edge direction of pixel, adaptively changing participate in the pixel and weight coefficient of interpolation, can preferably protect
Interpolation image edge feature is held, good interpolation is obtained.
The present invention improves depth image resolution ratio, improves depth image visual quality, can be applied to TOF camera etc.
The depth image of acquisition enhances occasion, and depth image interpolation algorithm therein can also be applied to the interpolation amplification of other images
Occasion.
Detailed description of the invention
Fig. 1 is the target image schematic diagram of 2 times of amplifications；
Fig. 2 is the target image rough schematic view of 2 times of amplifications；
Fig. 3 is experiment scene schematic diagram；
Fig. 4 is the depth image of TOF camera acquisition；
Fig. 5 is the depth image after image averaging；
Fig. 6 is the depth image after 2 times of interpolation of the method for the present invention；
Fig. 7 is the depth image of 8 times of different interpolation methods amplifications；(a) interpolation method of the present invention in figure, (b) arest neighbors is inserted
Value method, (c) bilinear interpolation method, (d) bicubic interpolation method, the small figure on each part side are that the part in big figure is put
Big figure.
Specific embodiment
Include the following steps:
(1), noise suppressed is carried out to the depth image that TOF camera acquires using image averaging method, improves depth map image quality
Amount；
(2) using the distance between pixels and edge directional information in image, the TOF depth image of low resolution is carried out
Interpolation amplification improves depth image resolution ratio, if depth image interpolation amplification is N × N times of original resolution ratio, carries out 2 first
Times interpolation amplification；
This process includes two parts: first is that judging that each pixel belongs to fringe region still in low resolution depth image
Smooth region；Second is that according to judging result, using different interpolation methods；
Fig. 1 is 2 times of amplified target images, and white pixel point comes from low resolution source images, and black pixel point is height
Interpolation pixel in resolution goals image, this method are exactly 9 whites in 3 × 3 windows utilized centered on (i, j)
Pixel value finds out tri- interpolation pixel values of A, B, C of the right pixel (i, j), lower section, and so on, source figure can be completed
2 times of interpolation amplifications of picture；
For clarity, Fig. 1 is simplified, only draws known 3 × 3 white pixel points and (is derived from lower-resolution source
Image) and interpolation tri- black pixel points of A, B, C, simplification figure is as shown in Figure 2；
Detailed step is as follows:
(1) judges whether each pixel belongs to fringe region in low resolution depth image
3 × 3 neighborhoods centered on low-resolution image pixel (i, j) utilize four sides as edge detection window
To template detection, whether the pixel belongs to fringe region, and detects edge direction；W1、W2、W3、W4Be respectively detect 0 °, 45 °,
90 °, extrorse 3 × 3 template in 135 ° of sides, concrete form are as follows:
The convolution value R of detection template and 3 × 3 edge detection windows are as follows:
Rm=f (i-1, j-1) Wm(1,1)+f (i-1, j) Wm(1,2)+f (i-1, j+1) Wm(1,3)+f (i, j-
1)·Wm(2,1)+f(i,j)·Wm(2,2)+f(i,j+1)·Wm(2,3)+f(i+1,j-1)·Wm(3,1)+f(i+1,j)·Wm
(3,2)+f(i+1,j+1)·Wm(3,3)m∈{1,2,3,4} (1)
Edge detection threshold T is set, will | Rm| maximum value be compared with T: when max (| Rm|) > T when, pixel (i,
J) belong to fringe region, | Rm| the corresponding direction of maximum value is edge direction；When max (| Rm|)≤T when, which belongs to
Smooth region；
(2) if pixel (i, j) belongs to smooth region, tri- interpolation pixels of A, B, C are calculated using following formula
Value:
(3) if pixel (i, j) belongs to fringe region, according to edge direction, A, B, C tri- is calculated using following formula
The value of a interpolation pixel:
Wherein, αKThe weighting coefficient of operation is participated in for 3 × 3 white pixel points in Interpolation Process.The present invention utilizes interpolation
Edge direction in the range information and 3 × 3 neighborhood of pixels of pixel and 3 × 3 pixel of surrounding determines weighting coefficient；
Wherein, αK_DisFor the weighting coefficient based on distance,For the weighting coefficient based on edge direction；
1)αK_Dis:
Interpolation pixel in 3 × 3 windows at a distance from white pixel point it is closer, the weight of pixel value is bigger, calculate to
The Euclidean distance square value of white pixel in interpolating pixel and 3 × 3 windows uses it reciprocal as weight coefficient, it may be assumed that
Wherein, DeFor Euclidean distance, for pixel p and q, coordinate is respectively (x, y) and (s, t), European between p and q
Distance definition is as follows:
De(p, q)=[(x-s)2+(y-t)2]1/2 (8)
2)
In order to keep the edge clear of interpolation image, this patent method is not to participate in interpolation fortune using whole neighborhood territory pixels
It calculates, but interpolation arithmetic is participated according to edge direction selected section neighborhood territory pixel, ginseng is indicated using 3 × 3 two-value mask matrix
With the location of pixels of interpolation arithmetic, i.e., " 1 " corresponding pixel participates in interpolation arithmetic in mask matrix, under four kinds of edge directions,
Used by calculating A, B, C interpolation pixel valueForm difference is as follows:
(4) repeats above step to all pixels in low resolution depth image, completes corresponding to each pixel
Interpolation pixel A, the interpolation of B, C, can be obtained 2 times of interpolation target image of amplification；
(3) as the integral number power that amplification factor N is 2, i.e. N=2k, k is positive integer, by iteration k times in step (2), i.e.,
It can get corresponding interpolation image；When amplification factor is not 2 integral number power, i.e., 2k-1<N<2k, then will first change in step (2)
For k times, obtaining amplification factor is 2kImage, according still further to formula (10) carry out it is down-sampled can be obtained amplification factor be N figure
Picture；
Wherein, g (i, j) is down-sampled image, and interpolation amplification multiple is N times of original TOF camera sampling depth image.
Further illustrate the present invention in conjunction with specific embodiments below.
TOF depth camera SR4000 is used in the embodiment of the present invention, to scene sampling depth image in Fig. 3, such as Fig. 4 institute
Show, resolution ratio 176*144.
Embodiment 1: the interpolation amplification multiple N=8 of depth image is set
(1), noise suppressed is carried out using depth image of the image averaging method to acquisition, after 10 amplitude deepness images are average
As a result as shown in Figure 5；
(2), each pixel in low resolution depth image is operated according to below step, completes depth image
2 times of interpolation amplifications.
1. judging whether pixel (i, j) belongs to fringe region.
According to formula (1), W is utilized1、W2、W3、W4Four direction template carries out operation respectively.This example edge detection threshold T
Be set as 50, when max (| R1|,|R2|,|R3|,|R4|) > 50 when, which belongs to fringe region, and the corresponding direction of maximum value is
Edge direction；Otherwise, which belongs to smooth region.
2. if pixel (i, j) belongs to smooth region, using formula (2), (3), (4) calculate separately A, B, C tri- to
Pixel value interpolation.
3. if pixel (i, j) belongs to fringe region, according to the edge direction that step 1 records,
Selection calculates A, B, C interpolation pixel value in formula (9)Form.In addition, according to formula (7), (8)
Calculate αK_Dis, it is as a result as follows:
Then formula (6) are utilized, calculates weighting coefficient αA、αB、αC；
Finally according to formula (5), the value of tri- interpolation pixels of A, B, C is calculated；
It completes in (two) after operation, the depth image after obtaining 2 times of interpolation, as shown in Figure 6；
(3), step in (two) is had altogether iteration 3 times, can be obtained 23=8 times of interpolation amplification image, such as Fig. 7 (a) institute
Show；
In order to compare interpolation, emulation experiment, Fig. 7 (b) (c) are carried out using several traditional interpolation algorithms to Fig. 5 below
It (d) is respectively the image for amplifying 8 times using arest neighbors interpolation, bilinear interpolation, bicubic interpolation.Divide in four width interpolation images
Not Xuan Qu toy ear location one piece of region, provide the interpolation algorithm amplification effect in the region, it can be seen that the bright interpolation of we
Algorithm has sharpened the edge of image, and image is made to seem apparent, can obtain good TOF depth image reinforcing effect.
Embodiment 2: the interpolation amplification multiple N=10 of depth image is set
(1), noise suppressed is carried out using depth image of the image averaging method to acquisition, this example uses 10 amplitude deepness images
It is averaged；
(2), each pixel in low resolution depth image is operated according to below step, completes depth image
2 times of interpolation amplifications；
1. judging whether pixel (i, j) belongs to fringe region；
According to formula (1), operation is carried out respectively using four direction template in Fig. 3.This example edge detection threshold T is set as
50, when max (| R1|,|R2|,|R3|,|R4|) > 50 when, which belongs to fringe region, and the corresponding direction of maximum value is edge side
To；Otherwise, which belongs to smooth region；
2. if pixel (i, j) belongs to smooth region, using formula (2), (3), (4) calculate separately A, B, C tri- to
Pixel value interpolation；
3. if, according to the edge direction that step 1 records, selected in formula (9) pixel (i, j) belongs to fringe region
Calculate A, B, C interpolation pixel valueForm.In addition, calculating α according to formula (7), (8)K_Dis, it is as a result as follows:
Then formula (6) are utilized, calculates weighting coefficient αA、αB、αC；
Finally according to formula (5), the value of tri- interpolation pixels of A, B, C is calculated；
It completes in (two) after operation, the depth image after obtaining 2 times of interpolation；
(3), it will be had altogether iteration 4 times in step (2), can be obtained 24Interpolation amplification image again, according to formula (10)
It is down-sampled to the depth image progress of 16 times of interpolation amplifications, it can be obtained the depth image of 10 times of interpolation amplifications, wherein k=4, N
=10.
Claims (3)
1. a kind of depth image enhancement method of TOF camera, it is characterised in that include the following steps:
(1), noise suppressed is carried out to the depth image that TOF camera acquires using image averaging method, improves depth image quality；
(2) using the distance between pixels and edge directional information in image, interpolation is carried out to the TOF depth image of low resolution
Amplification improves depth image resolution ratio, if depth image interpolation amplification is N × N times of original resolution ratio, carries out 2 times first and inserts
Value amplification；
This process includes two parts: first is that judging in low resolution depth image that each pixel belongs to fringe region or smooth
Region；Second is that according to judging result, using different interpolation methods,
(3) as the integral number power that amplification factor N is 2, i.e. N=2k, k is positive integer, and iteration k times in step (2) can obtain
Obtain corresponding interpolation image；When amplification factor is not 2 integral number power, i.e., 2k-1<N<2k, then first by iteration k in step (2)
Secondary, obtaining amplification factor is 2kImage, according still further to formula (10) carry out it is down-sampled can be obtained amplification factor be N image；
Wherein, g (i, j) is down-sampled image, and interpolation amplification multiple is N times of original TOF camera sampling depth image.
2. a kind of depth image enhancement method of TOF camera according to claim 1, it is characterised in that in step (2):
(1) judge that the step of whether each pixel belongs to fringe region in low resolution depth image is as follows:
3 × 3 neighborhoods using centered on low-resolution image pixel (i, j) utilize four direction mould as edge detection window
Plate detects whether the pixel belongs to fringe region, and detects edge direction；W1、W2、W3、W4Be respectively detect 0 °, 45 °, 90 °,
Extrorse 3 × 3 template in 135 ° of sides, concrete form are as follows:
The convolution value R of detection template and 3 × 3 edge detection windows are as follows:
Rm=f (i-1, j-1) Wm(1,1)+f(i-1,j)·Wm(1,2)+f(i-1,j+1)·Wm(1,3)+f(i,j-1)·Wm
(2,1)+f(i,j)·Wm(2,2)+f(i,j+1)·Wm(2,3)+f(i+1,j-1)·Wm(3,1)+f(i+1,j)·Wm(3,2)+
f(i+1,j+1)·Wm(3,3)m∈{1,2,3,4} (1)
Edge detection threshold T is set, will | Rm| maximum value be compared with T: when max (| Rm|) > T when, pixel (i, j) belong to
In fringe region, | Rm| the corresponding direction of maximum value is edge direction；When max (| Rm|)≤T when, which belongs to smoothly
Region.
3. a kind of depth image enhancement method of TOF camera according to claim 1, it is characterised in that root in step (2)
It is judged that as a result, steps are as follows using different interpolation methods:
(2) if pixel (i, j) belongs to smooth region, tri- interpolation pixel values of A, B, C are calculated using following formula:
(3) if pixel (i, j) belongs to fringe region, according to edge direction, using following formula calculate A, B, C tri- to
The value of interpolating pixel point:
Wherein, αKThe weighting coefficient that operation is participated in for 3 × 3 white pixel points in Interpolation Process, utilizes interpolation pixel and surrounding 3
Edge direction in the range information of × 3 pixels and 3 × 3 neighborhood of pixels determines weighting coefficient；
Wherein, αK_DisFor the weighting coefficient based on distance,For the weighting coefficient based on edge direction；
1)αK_Dis:
Interpolation pixel in 3 × 3 windows at a distance from white pixel point it is closer, the weight of pixel value is bigger, calculate interpolation
The Euclidean distance square value of white pixel in pixel and 3 × 3 windows uses it reciprocal as weight coefficient, it may be assumed that
Wherein, DeFor Euclidean distance, for pixel p and q, coordinate is respectively (x, y) and (s, t), and the Euclidean distance between p and q is fixed
Justice is as follows:
De(p, q)=[(x-s)2+(y-t)2]1/2 (8)
2)
In order to keep the edge clear of interpolation image, interpolation arithmetic is participated according to edge direction selected section neighborhood territory pixel, using 3
× 3 two-value mask matrix indicates the location of pixels for participating in interpolation arithmetic, i.e., " 1 " corresponding pixel participates in interpolation in mask matrix
Operation, under four kinds of edge directions, used by calculating A, B, C interpolation pixel valueForm difference is as follows:
(4) repeats above step to all pixels in low resolution depth image, complete corresponding to each pixel to
The interpolation of interpolating pixel A, B, C can be obtained the interpolation target image of 2 times of amplification.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201611149210.2A CN106780383B (en) | 2016-12-13 | 2016-12-13 | The depth image enhancement method of TOF camera |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201611149210.2A CN106780383B (en) | 2016-12-13 | 2016-12-13 | The depth image enhancement method of TOF camera |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN106780383A CN106780383A (en) | 2017-05-31 |
| CN106780383B true CN106780383B (en) | 2019-05-24 |
Family
ID=58876717
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201611149210.2A Active CN106780383B (en) | 2016-12-13 | 2016-12-13 | The depth image enhancement method of TOF camera |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106780383B (en) |
Families Citing this family (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109919847B (en) * | 2017-12-13 | 2023-03-14 | 彩优微电子(昆山)有限公司 | Method for improving quality of amplified image |
| CN111091592B (en) | 2018-10-24 | 2023-08-15 | Oppo广东移动通信有限公司 | Image processing method, image processing apparatus, electronic device, and readable storage medium |
| CN111199520B (en) * | 2018-11-19 | 2023-07-14 | 北京华航无线电测量研究所 | FPGA implementation method for color image scale expansion based on three-time convolution algorithm |
| CN112040214A (en) * | 2019-06-04 | 2020-12-04 | 万维科研有限公司 | Double-camera three-dimensional imaging system and processing method |
| CN111862183A (en) * | 2020-07-02 | 2020-10-30 | Oppo广东移动通信有限公司 | Depth image processing method and system, electronic device and storage medium |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104200426A (en) * | 2014-08-25 | 2014-12-10 | 北京京东方视讯科技有限公司 | Image interpolation method and device |
| US9208537B1 (en) * | 2014-07-10 | 2015-12-08 | Shenzhen China Star Optoelectronics Technology Co., Ltd | Super-resolution reconstructing method for enhancing smoothness and sharpness of video image |
| CN105139355A (en) * | 2015-08-18 | 2015-12-09 | 山东中金融仕文化科技股份有限公司 | Method for enhancing depth images |
| CN105447819A (en) * | 2015-12-04 | 2016-03-30 | 腾讯科技（深圳）有限公司 | Picture processing method and apparatus |
| WO2016129702A1 (en) * | 2015-02-12 | 2016-08-18 | Mitsubishi Electric Corporation | Method for image denoising |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP5448617B2 (en) * | 2008-08-19 | 2014-03-19 | パナソニック株式会社 | Distance estimation device, distance estimation method, program, integrated circuit, and camera |
| US20130077858A1 (en) * | 2011-09-22 | 2013-03-28 | Himax Imaging Limited | Image processing module and image processing method |
-
2016
- 2016-12-13 CN CN201611149210.2A patent/CN106780383B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9208537B1 (en) * | 2014-07-10 | 2015-12-08 | Shenzhen China Star Optoelectronics Technology Co., Ltd | Super-resolution reconstructing method for enhancing smoothness and sharpness of video image |
| CN104200426A (en) * | 2014-08-25 | 2014-12-10 | 北京京东方视讯科技有限公司 | Image interpolation method and device |
| WO2016129702A1 (en) * | 2015-02-12 | 2016-08-18 | Mitsubishi Electric Corporation | Method for image denoising |
| CN105139355A (en) * | 2015-08-18 | 2015-12-09 | 山东中金融仕文化科技股份有限公司 | Method for enhancing depth images |
| CN105447819A (en) * | 2015-12-04 | 2016-03-30 | 腾讯科技（深圳）有限公司 | Picture processing method and apparatus |
Also Published As
| Publication number | Publication date |
|---|---|
| CN106780383A (en) | 2017-05-31 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106780383B (en) | The depth image enhancement method of TOF camera | |
| Park et al. | High-quality depth map upsampling and completion for RGB-D cameras | |
| Yang et al. | Color-guided depth recovery from RGB-D data using an adaptive autoregressive model | |
| CN104318548B (en) | Rapid image registration implementation method based on space sparsity and SIFT feature extraction | |
| CN104680496B (en) | A kind of Kinect depth map restorative procedures based on color images | |
| CN106408513B (en) | Depth map super resolution ratio reconstruction method | |
| Lindner et al. | Sub-pixel data fusion and edge-enhanced distance refinement for 2d/3d images | |
| CN109903328B (en) | Object volume measuring device and method applied to smart phone | |
| Hua et al. | Extended guided filtering for depth map upsampling | |
| CN106408596B (en) | Sectional perspective matching process based on edge | |
| KR20130120730A (en) | Method for processing disparity space image | |
| JP2021168143A (en) | System and method for efficiently scoring probe in image by vision system | |
| JP4631973B2 (en) | Image processing apparatus, image processing apparatus control method, and image processing apparatus control program | |
| Cherian et al. | Accurate 3D ground plane estimation from a single image | |
| CN104574312A (en) | Method and device of calculating center of circle for target image | |
| CN111105452A (en) | High-low resolution fusion stereo matching method based on binocular vision | |
| Li et al. | 3d reconstruction and texture optimization using a sparse set of rgb-d cameras | |
| CN114549669B (en) | Color three-dimensional point cloud acquisition method based on image fusion technology | |
| Chen et al. | A color-guided, region-adaptive and depth-selective unified framework for Kinect depth recovery | |
| JP2018036898A (en) | Image processing device and control method of the same | |
| US8340399B2 (en) | Method for determining a depth map from images, device for determining a depth map | |
| CN109345570B (en) | Multi-channel three-dimensional color point cloud registration method based on geometric shape | |
| CN109064473B (en) | 2.5D ultrasonic panoramic image segmentation method | |
| KR101766431B1 (en) | Method and apparatus for detecting disparity by using hierarchical stereo matching | |
| CN111126418A (en) | Oblique image matching method based on planar perspective projection |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |