WO2016006901A1 - Procédé et appareil d'extraction d'informations de profondeur d'une image - Google Patents

Procédé et appareil d'extraction d'informations de profondeur d'une image Download PDF

Info

Publication number
WO2016006901A1
WO2016006901A1 PCT/KR2015/006977 KR2015006977W WO2016006901A1 WO 2016006901 A1 WO2016006901 A1 WO 2016006901A1 KR 2015006977 W KR2015006977 W KR 2015006977W WO 2016006901 A1 WO2016006901 A1 WO 2016006901A1
Authority
WO
WIPO (PCT)
Prior art keywords
data
patch data
psf
image
dimensional
Prior art date
Application number
PCT/KR2015/006977
Other languages
English (en)
Korean (ko)
Inventor
박현상
Original Assignee
재단법인 다차원 스마트 아이티 융합시스템 연구단
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 재단법인 다차원 스마트 아이티 융합시스템 연구단 filed Critical 재단법인 다차원 스마트 아이티 융합시스템 연구단
Publication of WO2016006901A1 publication Critical patent/WO2016006901A1/fr

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/10—Processing, recording or transmission of stereoscopic or multi-view image signals
    • H04N13/106—Processing image signals
    • H04N13/128—Adjusting depth or disparity
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/20—Image signal generators

Definitions

  • the present invention relates to an apparatus and method for extracting depth information from an image, and more particularly, to a technique for extracting depth information from an image by applying a one-dimensional point spread function (PSF).
  • PSF point spread function
  • the apparatus for extracting depth information may correspond to a plurality of preset areas corresponding to a predetermined pixel position of the first image 110.
  • the plurality of filters 120 is a filter applying the 2D PSF
  • a result value of applying the 2D PSF to the first patch data may mean a brightness value of the pixel.
  • the apparatus for extracting the existing depth information extracts first patch data corresponding to the MxM region based on a specific pixel position (x, y) of the first image 110 and then extracts the first extracted data.
  • a plurality of blur patches 130 including MxM pixels may be generated.
  • the first patch data is data corresponding to each of the plurality of pixels corresponding to the MxM area around the specific pixel position (x, y), and the plurality of pixels are (x + i, y + i),- M / 2 ⁇ i, j ⁇ M / 2.
  • the plurality of blur patches 130 since the plurality of blur patches 130 are generated by the N plurality of filters 120 corresponding to the N depths, the plurality of blur patches 130 may be configured of N MxM pixels.
  • the apparatus for extracting the existing depth information includes a second data including data of each of a plurality of pixels corresponding to a predetermined area centered on a specific pixel position of the second image 140 corresponding to the first patch data.
  • the apparatus for extracting the existing depth information extracts the second patch data corresponding to the MxM area around the specific pixel position (x, y) of the second image 140 so as to correspond to the first patch data.
  • the depth at a specific pixel position may be determined.
  • Embodiments of the present invention provide a method, apparatus, and system that reduce complexity of computation and reduce computation time by using a one-dimensional PSF in the process of generating a plurality of blur patches.
  • embodiments of the present invention by applying the one-dimensional PSF in any one of the order of the one-dimensional horizontal PSF from the one-dimensional vertical PSF or the order of the one-dimensional vertical PSF from the one-dimensional horizontal PSF, Provided are a method, an apparatus, and a system for substituting data of a plurality of pixels corresponding to a predetermined area around a specific pixel position of a single image with brightness values.
  • the method of extracting depth information from an image extracts first patch data including data of each of a plurality of pixels corresponding to a predetermined area around a specific pixel position of a first image. step; Receiving the first patch data in a raster scan order; The first patch input to the raster scan order using a plurality of filters applying a one-dimensional horizontal point spread function (PSF) and a one-dimensional vertical PSF to generate a plurality of blurred patches Processing the data; Extracting second patch data including data of each of a plurality of pixels corresponding to the preset area around the specific pixel position of the second image, which is different from the first image, corresponding to the first patch data Doing; Comparing the similarity between each of the second patch data and the plurality of blur patches; And determining a depth at the specific pixel position based on the comparison result.
  • PSF horizontal point spread function
  • the receiving of the first patch data in the raster scan order may include receiving data of each of the plurality of pixels included in the first patch data in the order from the horizontal direction to the vertical direction.
  • the processing of the first patch data may include applying the one-dimensional horizontal PSF to data of each of a plurality of pixels included in the first patch data input in the horizontal direction, for each of the plurality of filters. step; Storing data of each of the plurality of pixels included in the first patch data to which the one-dimensional horizontal PSF is applied to a line memory included in each of the plurality of filters; And for each of the plurality of filters, applying the one-dimensional vertical PSF to data of each of the plurality of pixels included in the first patch data to which the one-dimensional horizontal PSF applied to the line memory is applied. It may include.
  • the receiving of the first patch data in the raster scan order may include receiving data of each of the plurality of pixels included in the first patch data in the order from the vertical direction to the horizontal direction.
  • the processing of the first patch data may include applying the one-dimensional vertical PSF to data of each of a plurality of pixels included in the first patch data input in the vertical direction, for each of the plurality of filters. step; Storing data of each of a plurality of pixels included in the first patch data to which the one-dimensional vertical PSF is applied to a line memory included in each of the plurality of filters; And for each of the plurality of filters, applying the one-dimensional horizontal PSF to data of each of the plurality of pixels included in the first patch data to which the one-dimensional vertical PSF applied to the line memory is applied. It may include.
  • Comparing the similarity between the second patch data and each of the plurality of blur patches may include a Sum of Absolute Difference (SAD) technique, a Sum of Squared Difference (SSD) technique, a Mean of Absolute Difference (MAD) technique, or a SASE (SASE) technique. And comparing the similarity between each of the second patch data and each of the plurality of blur patches using at least one of a Sum of Squared Error technique.
  • SAD Sum of Absolute Difference
  • SSD Sum of Squared Difference
  • MAD Mean of Absolute Difference
  • SASE SASE
  • Comparing the similarity between each of the second patch data and the plurality of blur patches includes obtaining a difference value between each of the second patch data and each of the plurality of blur patches. Based on the above method, determining a depth at the specific pixel position may include selecting a smallest difference value among difference values between the second patch data and each of the plurality of blur patches; And determining the smallest difference value as a depth at the specific pixel position.
  • An apparatus for extracting depth information from an image extracts first patch data including data of each of a plurality of pixels corresponding to a predetermined area around a specific pixel position of a first image.
  • a first patch data extracting unit An input unit configured to receive the first patch data in a raster scan order; The first patch input to the raster scan order using a plurality of filters applying a one-dimensional horizontal point spread function (PSF) and a one-dimensional vertical PSF to generate a plurality of blurred patches
  • a processing unit for processing data Extracting second patch data including data of each of a plurality of pixels corresponding to the preset area around the specific pixel position of the second image, which is different from the first image, corresponding to the first patch data
  • a second patch data extracting unit A comparison unit comparing the similarity between each of the second patch data and the plurality of blur patches; And a determination unit that determines a depth at the specific pixel position based on the comparison result.
  • the input unit may receive data of each of the plurality of pixels included in the first patch data in the order from the horizontal direction to the vertical direction.
  • the processing unit applies the one-dimensional horizontal PSF to each of the plurality of pixels included in the first patch data input in the horizontal direction, for each of the plurality of filters, and the one-dimensional horizontal PSF. Is stored in a line memory included in each of the plurality of filters, and the data of each of the plurality of pixels included in the first patch data to which the first patch data is applied is stored in each of the plurality of filters.
  • the one-dimensional vertical PSF may be applied to data of a plurality of pixels included in the first patch data to which the stored one-dimensional horizontal PSF is applied.
  • the input unit may receive data of each of the plurality of pixels included in the first patch data in the order from the vertical direction to the horizontal direction.
  • the processing unit applies the one-dimensional vertical direction PSF to each of the plurality of pixels included in the first patch data input in the vertical direction, for each of the plurality of filters, and applies the one-dimensional vertical direction to the data.
  • the data stored in each of the plurality of pixels included in the first patch data to which the PSF is applied is stored in a line memory included in each of the plurality of filters, and for each of the plurality of filters, the 1 stored in the line memory.
  • the one-dimensional horizontal PSF may be applied to data of a plurality of pixels included in the first patch data to which the dimensional vertical PSF is applied.
  • the comparison unit uses the second patch data using at least one of a sum of absolute difference (SAD) technique, a sum of squared difference (SSD) technique, a mean of absolute difference (MAD) technique, or a sum of squared error (SASE) technique. And similarity between each of the plurality of blur patches.
  • SAD sum of absolute difference
  • SSD sum of squared difference
  • MAD mean of absolute difference
  • SASE sum of squared error
  • Embodiments of the present invention can provide a method, apparatus and system for reducing the complexity of computation and reducing the computation time by using a one-dimensional PSF in the process of generating a plurality of blur patches.
  • embodiments of the present invention by applying the one-dimensional PSF in any one of the order of the one-dimensional horizontal PSF from the one-dimensional vertical PSF or the order of the one-dimensional vertical PSF from the one-dimensional horizontal PSF, It is possible to provide a method, an apparatus, and a system for replacing data of a plurality of pixels corresponding to a predetermined area centering on a specific pixel position of a single image with brightness values.
  • 1 is a conceptual diagram illustrating a method of extracting depth information from an existing image.
  • FIG. 2 is a conceptual diagram illustrating a method of extracting depth information from an image according to an embodiment of the present invention.
  • FIG. 3 is a diagram illustrating a first patch data input process and a one-dimensional PSF application process according to an embodiment of the present invention.
  • FIG. 4 is a diagram illustrating a first patch data input process and a 1D PSF application process according to another embodiment of the present invention.
  • FIG. 5 is a diagram illustrating a one-dimensional horizontal PSF and a one-dimensional vertical PSF according to an embodiment of the present invention.
  • FIG. 6 is a flowchart illustrating a method of extracting depth information from an image according to an embodiment of the present invention.
  • FIG. 7 is a block diagram illustrating an apparatus for extracting depth information from an image according to an embodiment of the present invention.
  • FIG. 2 is a conceptual diagram illustrating a method of extracting depth information from an image according to an embodiment of the present invention.
  • an apparatus for extracting depth information from an image uses a one-dimensional PSF based on a first image 210 and a second image 220 distinguished from the first image.
  • the first image 210 may be an image clearly captured at all depths using a small aperture
  • the second image 220 may be a small aperture used in the process of acquiring the first image 210.
  • the image may be sharper as the distance from the depth is in focus.
  • the one-dimensional PSF is a one-dimensional function indicating how the data of the pixel, which can be represented by the brightness value, has a spatial distribution, and may mean a one-dimensional vertical PSF and a one-dimensional horizontal PSF.
  • the apparatus for extracting depth information from an image extracts first patch data including data of each of a plurality of pixels corresponding to a predetermined area around a specific pixel position of the first image 210 and then extracts the first patch data.
  • Received first patch data in a raster scan order.
  • the raster scan order refers to a method in which data of each of the plurality of pixels included in the first patch data is sequentially input from left to right and from top to bottom.
  • the apparatus for extracting depth information from the image uses a plurality of filters 240 that apply a one-dimensional horizontal PSF and a one-dimensional vertical PSF to generate a plurality of blur patches 230.
  • the first patch data input to the master scan order is processed.
  • the apparatus for extracting depth information from the image receives data of each of the plurality of pixels included in the first patch data in the order from the horizontal direction to the vertical direction, for each of the plurality of filters 240, One-dimensional horizontal PSF may be applied to data of each of the plurality of pixels included in the first patch data input in the horizontal direction, and may be stored in the line memory 250 included in each of the plurality of filters 240.
  • the apparatus for extracting depth information from the image reads data of each of the plurality of pixels included in the first patch data to which the one-dimensional horizontal PSF applied to the one-dimensional horizontal PSF stored in the line memory 250 is applied in the vertical direction, thereby the one-dimensional vertical PSF. Can be applied.
  • an apparatus for extracting depth information from an image may include data of each of a plurality of pixels included in first patch data corresponding to an MxM region, based on a specific pixel position (x, y) of the first image 210.
  • the line is applied by applying a one-dimensional horizontal PSF to each of k-1 rows of data of each of the plurality of pixels included in the first patch data input in the horizontal direction. May be stored in the memory 250.
  • the apparatus for extracting the depth information from the image applies the one-dimensional horizontal PSF to the last row of the data of each of the plurality of pixels included in the first patch data currently input in the horizontal direction, and the line memory 250
  • the one-dimensional vertical PSF may be applied by combining with k-1 rows of data of each of the plurality of pixels included in the first patch data to which the one-dimensional horizontal PSF to be stored is applied.
  • the apparatus for extracting depth information from the image may receive data of each of the plurality of pixels included in the first patch data in the order from the vertical direction to the horizontal direction, For example, one-dimensional vertical PSF may be applied to data of each of the plurality of pixels included in the first patch data input in the vertical direction, and stored in the line memory 250 included in each of the plurality of filters 240. .
  • the apparatus for extracting depth information from the image reads horizontally the data of each of the plurality of pixels included in the first patch data to which the one-dimensional vertical PSF to which the one-dimensional vertical PSF is stored in the line memory 250 is applied. You can also apply
  • an apparatus for extracting depth information from an image may include data of each of a plurality of pixels included in first patch data corresponding to an MxM region, based on a specific pixel position (x, y) of the first image 210.
  • the line is applied by applying a one-dimensional vertical PSF to each of the k-1 columns of data of each of the plurality of pixels included in the first patch data input in the vertical direction. May be stored in the memory 250.
  • the apparatus for extracting depth information from the image applies the one-dimensional vertical PSF to the last column of each of the plurality of pixels included in the first patch data currently input in the vertical direction, and applies the line memory 250 to the line memory 250.
  • the one-dimensional horizontal PSF may be applied by combining the data of k-1 columns of the plurality of pixels included in the first patch data to which the stored one-dimensional vertical PSF is applied.
  • the one-dimensional PSF application process is performed in each of the plurality of filters 240, a plurality of blur patches 230 corresponding to the number of the plurality of filters 240 may be generated.
  • the plurality of filters 240 are filters applying one-dimensional vertical PSF and one-dimensional horizontal PSF, and a result value of applying the one-dimensional PSF to the first patch data may mean a brightness value of the pixel. Detailed description thereof will be described with reference to FIGS. 3 and 4.
  • the apparatus for extracting depth information from the image includes a plurality of regions corresponding to a predetermined area of a specific pixel position of the second image 220, which is distinguished from the first image 210, corresponding to the first patch data.
  • the apparatus for extracting the depth information from the image may use various techniques in the process of comparing the similarity between the second patch data and each of the plurality of blur patches 230.
  • an apparatus for extracting depth information from an image may include at least one of a Sum of Absolute Difference (SAD) technique, a Sum of Squared Difference (SSD) technique, a Mean of Absolute Difference (MAD) technique, or a Sum of Squared Error (SASE) technique.
  • SAD Sum of Absolute Difference
  • SSD Sum of Squared Difference
  • MAD Mean of Absolute Difference
  • SASE Sum of Squared Error
  • the apparatus for extracting depth information from an image obtains a difference value between each of the second patch data and the plurality of blur patches 230, and the second patch data and the plurality of blur patches 230. By selecting the smallest difference value among the difference values between each, the smallest difference value can be determined as the depth at a specific pixel position.
  • the apparatus for extracting depth information from an image may generate one-dimensional PSFs (one-dimensional horizontal PSF and one-dimensional vertical PSF) in the process of generating a plurality of blur patches 230.
  • one-dimensional PSFs one-dimensional horizontal PSF and one-dimensional vertical PSF
  • FIG 3 is a diagram illustrating a first patch data input process and a one-dimensional PSF application process according to an embodiment of the present invention.
  • the first patch data input process and the 1D PSF application process according to an embodiment of the present invention receive data of each of the plurality of pixels included in the first patch data in the order from the horizontal direction to the vertical direction, 1 After applying the dimensional horizontal PSF, a description will be given of the case of applying the one-dimensional vertical PSF.
  • an apparatus for extracting depth information from an image inputs data of each of a plurality of pixels included in the first patch data 310 in a horizontal direction to a vertical direction. I can receive it.
  • an apparatus for extracting depth information from an image may configure a plurality of rows using data of each of a plurality of pixels included in the first patch data 310, and when input of data constituting one row is completed, You can receive input from the data that makes up the next row.
  • the apparatus for extracting depth information from an image receives data included in the first row 320 in the order of left to right, and when input of data included in the first row 320 is completed, The data included in the second row 330 may be input in the order of left to right.
  • the data of each of the pixels included in the first patch data 310 input as described above may be stored in the line memory included in each of the plurality of filters by applying a one-dimensional horizontal PSF to each of the plurality of filters.
  • the data of each of the pixels included in the first patch data 310 configured by configuring the plurality of rows may be stored in the line memory by applying one-dimensional horizontal PSF to each row.
  • an apparatus for extracting depth information from an image may be applied to data included in the first row 320 and stored in the line memory by applying the one-dimensional horizontal PSF to the second row 330.
  • One-dimensional horizontal PSF is applied to the data to be stored in the line memory.
  • the apparatus for extracting depth information from the image may apply the one-dimensional vertical PSF to the data of each of the plurality of pixels included in the first patch data 310 to which the one-dimensional horizontal PSF to which the one-dimensional horizontal PSF is stored is applied.
  • the one-dimensional horizontal PSF may be applied to each of the plurality of rows, and thus, the one-dimensional vertical PSF may be applied to the data of each of the pixels included in the first patch data 310 stored in the line memory.
  • the apparatus for extracting depth information from an image may include data included in the first row 320 to which the one-dimensional horizontal PSF is applied and second row 330 to which the one-dimensional horizontal PSF is applied.
  • One-dimensional vertical PSF may be applied to the data to generate a blur patch including brightness values corresponding to data of each of the pixels included in the first patch data 310.
  • the apparatus for extracting depth information from an image applies the one-dimensional PSF in the order of the one-dimensional vertical PSF to the one-dimensional vertical PSF, thereby focusing on the specific pixel position of the first image.
  • Data of each of the plurality of pixels corresponding to the preset area may be replaced with a brightness value without omission.
  • the one-dimensional PSF application process is performed in each of the plurality of filters, a plurality of blur patches corresponding to the number of the plurality of filters may be generated.
  • FIG. 4 is a diagram illustrating a first patch data input process and a 1D PSF application process according to another embodiment of the present invention.
  • the first patch data input process and the 1D PSF application process according to another embodiment of the present invention receive data of each of the plurality of pixels included in the first patch data in the order from the vertical direction to the horizontal direction, A case in which one-dimensional vertical PSF is applied and then one-dimensional horizontal PSF is applied will be described.
  • an apparatus for extracting depth information from an image may include data of each of a plurality of pixels included in the first patch data 410 in a vertical direction from a horizontal direction. You can receive input.
  • the apparatus for extracting depth information from an image may configure a plurality of columns of data of each of the plurality of pixels included in the first patch data 410, and when input of data constituting one column is completed, In this case, you can receive input of the data that makes up the next column.
  • the apparatus for extracting depth information from an image receives data included in the first column 420 in the order of the top to the bottom, and receives data included in the second column 430 from the top to the bottom. In this order, the data included in the third column 440 may be input in the order of the top to the bottom.
  • Data of each of the pixels included in the first patch data 410 input as described above may be stored in the line memory included in each of the plurality of filters by applying a one-dimensional vertical PSF to each of the plurality of filters.
  • data of each of the plurality of pixels included in the first patch data 410 input by configuring a plurality of columns may be stored in the line memory by applying one-dimensional vertical PSF to each column.
  • the apparatus for extracting depth information from an image may be applied to the data included in the first column 420 and stored in the line memory by applying a one-dimensional vertical PSF to the data included in the second column 430. After applying the one-dimensional vertical PSF to the data and storing it in the line memory, the one-dimensional vertical PSF may be applied to the data included in the third column 440 and stored in the line memory.
  • the apparatus for extracting the depth information from the image may apply the one-dimensional horizontal PSF to the data of each of the plurality of pixels included in the first patch data 410 to which the one-dimensional vertical PSF applied in the line memory is applied.
  • the one-dimensional vertical PSF may be applied to each of the plurality of columns so that the one-dimensional horizontal PSF may be applied to the data of each of the pixels included in the first patch data 410 stored in the line memory.
  • the apparatus for extracting depth information from an image includes data included in the first column 420 to which the one-dimensional vertical PSF is applied and data included in the second column 430 to which the one-dimensional vertical PSF is applied.
  • a blur patch consisting of brightness values can be generated.
  • the apparatus for extracting depth information from an image applies the one-dimensional PSF in the order of the one-dimensional vertical PSF to the one-dimensional horizontal PSF, thereby centering the specific pixel position of the first image.
  • data of each of the pixels corresponding to the preset area may be replaced with a brightness value without omission.
  • the one-dimensional PSF application process is performed in each of the plurality of filters, a plurality of blur patches corresponding to the number of the plurality of filters may be generated.
  • FIG. 5 is a diagram illustrating a one-dimensional horizontal PSF and a one-dimensional vertical PSF according to an embodiment of the present invention.
  • the one-dimensional horizontal PSF 510 and the one-dimensional vertical PSF 520 may determine the brightness weight of the horizontal component and the brightness weight of the vertical component from the separable two-dimensional PSF. By separating each, it can be obtained.
  • the one-dimensional horizontal PSF 510 and the one-dimensional vertical PSF 520 separate the brightness weights of the horizontal component and the brightness weights of the vertical component from the Gaussian filter 530 which is the two-dimensional PSF, respectively. Can be obtained.
  • the one-dimensional horizontal PSF 510 and the one-dimensional vertical PSF 520 according to an embodiment of the present invention are not limited thereto, but are not limited thereto. It can be obtained by separating the brightness weights and the brightness weights of the vertical components, respectively.
  • FIG. 6 is a flowchart illustrating a method of extracting depth information from an image according to an embodiment of the present invention.
  • an apparatus for extracting depth information from an image may include a first data including data of each of a plurality of pixels corresponding to a predetermined area around a specific pixel position of a first image.
  • One patch data is extracted (610).
  • the apparatus for extracting depth information from the image receives the first patch data in a raster scan order.
  • the apparatus for extracting the depth information from the image may receive the data of each of the plurality of pixels included in the first patch data in the order from the horizontal direction to the vertical direction.
  • the apparatus for extracting depth information from the image may receive data of each of the plurality of pixels included in the first patch data in the order from the vertical direction to the horizontal direction.
  • the apparatus for extracting depth information from an image uses a plurality of filters to apply a one-dimensional horizontal point spread function (PSF) and a one-dimensional vertical PSF to generate a plurality of blurred patches.
  • PSF horizontal point spread function
  • the first patch data input to the raster scan order is processed (630).
  • the apparatus for extracting depth information from the image may be horizontally applied to each of the plurality of filters.
  • One-dimensional horizontal PSF is applied to data of each of the plurality of pixels included in the first patch data input, and data of each of the plurality of pixels included in the first patch data to which the one-dimensional horizontal PSF is applied is plural.
  • the apparatus for extracting depth information from the image is input in the vertical direction with respect to each of the plurality of filters.
  • One-dimensional vertical PSF is applied to data of each of the plurality of pixels included in the first patch data
  • data of each of the plurality of pixels included in the first patch data to which the one-dimensional vertical PSF is applied is a plurality of filters.
  • the one-dimensional horizontal PSF in the data of each of the plurality of pixels included in the first patch data to which the one-dimensional vertical PSF stored in the line memory is applied.
  • the apparatus for extracting depth information from the image may extract data of each of a plurality of pixels corresponding to a predetermined area around a specific pixel position of a second image, which is different from the first image, corresponding to the first patch data.
  • the second patch data including the extracted second patch data is included.
  • the apparatus for extracting depth information from the image compares the similarity between each of the second patch data and the plurality of blur patches (650).
  • the apparatus for extracting the depth information from the image is at least any one of a Sum of Absolute Difference (SAD) technique, a Sum of Squared Difference (SSD) technique, a Mean of Absolute Difference (MAD) technique, or a Sum of Squared Error (SASE) technique.
  • SAD Sum of Absolute Difference
  • SSD Sum of Squared Difference
  • MAD Mean of Absolute Difference
  • SASE Sum of Squared Error
  • the apparatus for extracting depth information from the image determines 660 the depth at a particular pixel location based on the comparison result.
  • an apparatus for extracting depth information from an image obtains a difference value between each of the second patch data and the plurality of blur patches, and obtains a smallest value of the difference value between each of the second patch data and the plurality of blur patches.
  • the difference value By selecting the difference value, the smallest difference value selected can be determined as the depth at a specific pixel position.
  • FIG. 7 is a block diagram illustrating an apparatus for extracting depth information from an image according to an embodiment of the present invention.
  • an apparatus for extracting depth information from an image may include a first patch data extractor 710, an input unit 720, a processor 730, and a second patch data extractor ( 740, a comparator 750, and a determiner 760.
  • the first patch data extractor 710 extracts first patch data including data of each of a plurality of pixels corresponding to a predetermined area centering on a specific pixel position of the first image.
  • the input unit 720 receives the first patch data in a raster scan order.
  • the input unit 720 receives data of each of the plurality of pixels included in the first patch data in the order from the horizontal direction to the vertical direction.
  • the input unit 720 may receive data of each of the plurality of pixels included in the first patch data in the order from the vertical direction to the horizontal direction.
  • the processor 730 is input to the raster scan order using a plurality of filters applying one-dimensional horizontal point spread function (PSF) and one-dimensional vertical PSF to generate a plurality of blurred patches. Process the first patch data.
  • PSF horizontal point spread function
  • the processor 730 may receive the horizontal direction with respect to each of the plurality of filters.
  • One-dimensional horizontal PSF is applied to data of each of the plurality of pixels included in the first patch data input to the first patch data, and data of each of the plurality of pixels included in the first patch data to which the one-dimensional horizontal PSF is applied is plural.
  • Data of each of the plurality of pixels stored in a line memory included in each of the filters of the plurality of filters and included in the first patch data to which the one-dimensional horizontal PSF stored in the line memory is applied to each of the plurality of filters.
  • the processing unit 730 may be configured in the vertical direction with respect to each of the plurality of filters.
  • a one-dimensional vertical PSF is applied to data of each of the plurality of pixels included in the input first patch data, and the data of each of the plurality of pixels included in the first patch data to which the one-dimensional vertical PSF is applied is plural.
  • One-dimensional horizontal direction is stored in the line memory included in each of the filters, and for each of the plurality of filters, data of each of the plurality of pixels included in the first patch data to which the one-dimensional vertical direction PSF stored in the line memory is applied.
  • the second patch data extractor 740 includes data of each of a plurality of pixels corresponding to a predetermined area centered on a specific pixel position of a second image, which is different from the first image, corresponding to the first patch data. Extract the second patch data.
  • the comparison unit 750 compares the similarity between each of the second patch data and the plurality of blur patches.
  • the comparator 750 uses at least one of a Sum of Absolute Difference (SAD) technique, a Sum of Squared Difference (SSD) technique, a Mean of Absolute Difference (MAD) technique, or a Sum of Squared Error (SASE) technique.
  • SAD Sum of Absolute Difference
  • SSD Sum of Squared Difference
  • MAD Mean of Absolute Difference
  • SASE Sum of Squared Error
  • the determination unit 760 determines the depth at the specific pixel position based on the result.
  • the comparison unit 750 obtains a difference value between each of the second patch data and the plurality of blur patches, and the determination unit 760 determines a difference value between each of the second patch data and the plurality of blur patches. By selecting the smallest difference value among these, the smallest difference value selected can be determined as the depth at a specific pixel position.
  • the apparatus described above may be implemented as a hardware component, a software component, and / or a combination of hardware components and software components.
  • the devices and components described in the embodiments may be, for example, processors, controllers, arithmetic logic units (ALUs), digital signal processors, microcomputers, field programmable arrays (FPAs), It may be implemented using one or more general purpose or special purpose computers, such as a programmable logic unit (PLU), microprocessor, or any other device capable of executing and responding to instructions.
  • the processing device may execute an operating system (OS) and one or more software applications running on the operating system.
  • the processing device may also access, store, manipulate, process, and generate data in response to the execution of the software.
  • OS operating system
  • the processing device may also access, store, manipulate, process, and generate data in response to the execution of the software.
  • processing device includes a plurality of processing elements and / or a plurality of types of processing elements. It can be seen that it may include.
  • the processing device may include a plurality of processors or one processor and one controller.
  • other processing configurations are possible, such as parallel processors.
  • the software may include a computer program, code, instructions, or a combination of one or more of the above, and configure the processing device to operate as desired, or process it independently or collectively. You can command the device.
  • Software and / or data may be any type of machine, component, physical device, virtual equipment, computer storage medium or device in order to be interpreted by or to provide instructions or data to the processing device. Or may be permanently or temporarily embodied in a signal wave to be transmitted.
  • the software may be distributed over networked computer systems so that they may be stored or executed in a distributed manner.
  • Software and data may be stored on one or more computer readable recording media.
  • the method according to the embodiment may be embodied in the form of program instructions that can be executed by various computer means and recorded in a computer readable medium.
  • the computer readable medium may include program instructions, data files, data structures, etc. alone or in combination.
  • the program instructions recorded on the media may be those specially designed and constructed for the purposes of the embodiments, or they may be of the kind well-known and available to those having skill in the computer software arts.
  • Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tape, optical media such as CD-ROMs, DVDs, and magnetic disks, such as floppy disks.
  • Examples of program instructions include not only machine code generated by a compiler, but also high-level language code that can be executed by a computer using an interpreter or the like.
  • the hardware device described above may be configured to operate as one or more software modules to perform the operations of the embodiments, and vice versa.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Image Processing (AREA)

Abstract

L'invention concerne un procédé d'extraction d'informations de profondeur d'une image comprenant les étapes consistant à: extraire des premières données de correction comprenant des données de chaque pixel d'une pluralité de pixels correspondant à une région prédéterminée sur la base d'une certaine position des pixels d'une première image; recevoir une entrée des premières données de correction dans un ordre de balayage tramé; traiter les premières données de correction qui sont entrées dans l'ordre de balayage tramé à l'aide d'une fonction d'étalement de point horizontale unidimensionnelle (PSF) et d'une pluralité de filtres, auxquels est appliquée la fonction d'étalement de point horizontale unidimensionnelle (PSF), afin de générer une pluralité de plages floues; extraire des secondes données de correction qui correspondent aux premières données de correction et comprennent des données de chaque pixel d'une pluralité de pixels correspondant à la région prédéterminée sur la base d'une position donnée des pixels d'une seconde image qui se distingue de la première image; comparer des similarités entre les secondes données de correction et chaque plage de la pluralité de plages floues; et déterminer la profondeur de ladite position donnée des pixels sur la base du résultat de la comparaison.
PCT/KR2015/006977 2014-07-08 2015-07-07 Procédé et appareil d'extraction d'informations de profondeur d'une image WO2016006901A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
KR1020140085203A KR101600183B1 (ko) 2014-07-08 2014-07-08 이미지로부터 깊이 정보를 추출하는 방법 및 장치
KR10-2014-0085203 2014-07-08

Publications (1)

Publication Number Publication Date
WO2016006901A1 true WO2016006901A1 (fr) 2016-01-14

Family

ID=55064461

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/KR2015/006977 WO2016006901A1 (fr) 2014-07-08 2015-07-07 Procédé et appareil d'extraction d'informations de profondeur d'une image

Country Status (2)

Country Link
KR (1) KR101600183B1 (fr)
WO (1) WO2016006901A1 (fr)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109195435A (zh) * 2018-10-29 2019-01-11 珠海奇川精密设备有限公司 贴片机防重贴贴装机构

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR101829534B1 (ko) 2016-05-25 2018-02-19 재단법인 다차원 스마트 아이티 융합시스템 연구단 멀티 초점 이미지를 이용하는 깊이 추출 카메라 시스템 및 그 동작 방법

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2001208524A (ja) * 2000-01-26 2001-08-03 Sony Corp 画像処理装置及び画像処理方法
KR101086274B1 (ko) * 2008-12-05 2011-11-24 한국전자통신연구원 깊이정보 추출 장치 및 추출 방법
KR101088347B1 (ko) * 2010-06-11 2011-11-30 주식회사 실리콘웍스 1차원 내용기반 영상 보간 장치, 이를 위한 데이터베이스 구축방법 및 1차원 내용기반 영상 보간 방법

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH11234703A (ja) * 1998-02-09 1999-08-27 Toshiba Corp 立体表示装置

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2001208524A (ja) * 2000-01-26 2001-08-03 Sony Corp 画像処理装置及び画像処理方法
KR101086274B1 (ko) * 2008-12-05 2011-11-24 한국전자통신연구원 깊이정보 추출 장치 및 추출 방법
KR101088347B1 (ko) * 2010-06-11 2011-11-30 주식회사 실리콘웍스 1차원 내용기반 영상 보간 장치, 이를 위한 데이터베이스 구축방법 및 1차원 내용기반 영상 보간 방법

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
SAEED MAHMOUDPOUR ET AL.: "Depth Map Generation Using Infocused and Defocused Images", JOURNAL OF BROADCAST ENGINEERING, vol. 19, no. 3, May 2014 (2014-05-01), pages 363 - 367 *

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109195435A (zh) * 2018-10-29 2019-01-11 珠海奇川精密设备有限公司 贴片机防重贴贴装机构
CN109195435B (zh) * 2018-10-29 2024-02-13 珠海奇川精密设备有限公司 贴片机防重贴贴装机构

Also Published As

Publication number Publication date
KR101600183B1 (ko) 2016-03-04
KR20160005984A (ko) 2016-01-18

Similar Documents

Publication Publication Date Title
WO2015005577A1 (fr) Appareil et procédé d'estimation de pose d'appareil photo
WO2020071701A1 (fr) Procédé et dispositif de détection d'un objet en temps réel au moyen d'un modèle de réseau d'apprentissage profond
WO2020027607A1 (fr) Dispositif de détection d'objets et procédé de commande
WO2021095991A1 (fr) Dispositif et procédé de génération d'une image de défaut
WO2011087289A2 (fr) Procédé et système pour réaliser le rendu de vues tridimensionnelles d'une scène
WO2020139009A1 (fr) Dispositif d'apprentissage de maladies cérébrovasculaires, dispositif de détection de maladies cérébrovasculaires, procédé d'apprentissage de maladies cérébrovasculaires et procédé de détection de maladies cérébrovasculaires
US11386934B2 (en) Method and device for displaying target object
WO2017099510A1 (fr) Procédé permettant de segmenter une scène statique sur la base d'informations statistiques d'image et procédé s'y rapportant
WO2017195984A1 (fr) Dispositif et procédé de numérisation 3d
AU2020289853A1 (en) Matching method and apparatus, electronic device, computer-readable storage medium, and computer program
WO2016006901A1 (fr) Procédé et appareil d'extraction d'informations de profondeur d'une image
WO2020231005A1 (fr) Dispositif de traitement d'image et son procédé de fonctionnement
WO2014010820A1 (fr) Procédé et appareil d'estimation de mouvement d'image à l'aide d'informations de disparité d'une image multivue
WO2016190698A1 (fr) Procédé de filtrage médian pondéré et dispositif de suppression de bruit
WO2012134238A2 (fr) Procédé et appareil de génération d'un modèle numérique d'élévation
WO2021084797A1 (fr) Dispositif de détection d'objet, procédé de détection d'objet, programme de détection d'objet et dispositif d'apprentissage
WO2018101607A1 (fr) Processeur vectoriel et son procédé de commande
WO2017195985A1 (fr) Dispositif portable et procédé de balayage de document 3d
WO2016159613A1 (fr) Procédé et système de suivi d'objet pour synthèse d'image
WO2016098943A1 (fr) Procédé et système de traitement d'images pour améliorer la capacité de détection de visages
JP6982194B2 (ja) ビデオ特徴の抽出方法および装置
WO2017007047A1 (fr) Procédé et dispositif de compensation de la non-uniformité de la profondeur spatiale en utilisant une comparaison avec gigue
WO2019225799A1 (fr) Procédé et dispositif de suppression d'informations d'utilisateur à l'aide d'un modèle génératif d'apprentissage profond
WO2017073823A1 (fr) Dispositif et procédé de dérivation de valeur seuil adaptative et de distinction entre une fourrure de langue, une texture de langue, et une zone mixte associée
WO2021071258A1 (fr) Dispositif et procédé d'apprentissage d'image de sécurité mobile basés sur l'intelligence artificielle

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 15819549

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 15819549

Country of ref document: EP

Kind code of ref document: A1