WO2011087083A1 - Procédé de traitement de données, dispositif de traitement de données et programme de traitement de données - Google Patents

Procédé de traitement de données, dispositif de traitement de données et programme de traitement de données Download PDF

Info

Publication number
WO2011087083A1
WO2011087083A1 PCT/JP2011/050534 JP2011050534W WO2011087083A1 WO 2011087083 A1 WO2011087083 A1 WO 2011087083A1 JP 2011050534 W JP2011050534 W JP 2011050534W WO 2011087083 A1 WO2011087083 A1 WO 2011087083A1
Authority
WO
WIPO (PCT)
Prior art keywords
data
frequency
image data
data processing
spectrum
Prior art date
Application number
PCT/JP2011/050534
Other languages
English (en)
Japanese (ja)
Inventor
茂樹 広林
貴博 加古
Original Assignee
国立大学法人富山大学
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from JP2010006234A external-priority patent/JP2013084019A/ja
Application filed by 国立大学法人富山大学 filed Critical 国立大学法人富山大学
Publication of WO2011087083A1 publication Critical patent/WO2011087083A1/fr

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
    • G06T3/4007—Scaling of whole images or parts thereof, e.g. expanding or contracting based on interpolation, e.g. bilinear interpolation
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
    • G10L19/005—Correction of errors induced by the transmission channel, if related to the coding algorithm

Definitions

  • the present invention relates to a data processing method, a data processing apparatus, and a data processing program for analyzing and processing various types of data such as image data, and in particular, correcting missing portions of data by data interpolation processing and / or extrapolation processing.
  • the present invention relates to a data processing method, a data processing apparatus, and a data processing program.
  • an area with high brightness in the obtained image data causes so-called “overexposed” due to an excessive aperture of the imaging device, and vice versa.
  • so-called “blackout” or the like may occur, and information that is inherently lost may be lost.
  • a process of interpolating missing information based on adjacent macroblocks to generate a predicted image or performing various image corrections is performed during compression encoding. ing.
  • an interpolation technique for example, there is a technique to which an interpolation method is applied like the techniques described in Patent Documents 1 to 6 and the like.
  • Patent Document 1 discloses a technique for reducing image quality degradation due to judder at a low cost in an image interpolation device that converts a frame frequency using an interpolated image generated from an original image.
  • Patent Document 2 discloses a technique for appropriately generating pixels in any region of an image of an interpolation frame when performing frame interpolation on a video signal.
  • Patent Document 3 when interpolating an interpolated frame on an interpolated frame surface between temporally adjacent frames, there is no gap or overlap between images in the interpolated frame, and further block distortion occurs.
  • a technique for reducing the above is disclosed.
  • Patent Document 4 discloses a technique for expanding the resolution by interpolating between two points inside the frame subjected to frequency analysis.
  • Patent Document 5 discloses a technique for applying an interpolation method when creating a three-dimensional color information conversion table for color-converting RGB signal processing system image information to YMCK signal processing system image information. Yes.
  • Patent Document 6 discloses a technique for applying an interpolation method when smoothing the periphery of a texture image.
  • Patent Document 7 discloses a technique for linearly interpolating the total code amount calculated for each quantization step in order to realize image compression that can efficiently control the total code amount of the entire frame. .
  • Patent Document 8 discloses a technique for efficiently generating a prediction signal for an image having a complex pattern when generating an intra-screen prediction signal by applying an extrapolation method.
  • Patent Document 9 when an input image is divided into block units and a difference from a predicted image generated according to a prediction mode selected by intra prediction is encoded, pixel values are extrapolated.
  • a technique for generating a predicted image is disclosed.
  • Patent Document 10 discloses a technique for removing block artifacts from a compressed image by extrapolation.
  • Patent Document 11 discloses a technique for obtaining a texture map value using extrapolation instead of a mipmap that cannot be used when mapping a texture image.
  • Patent Document 12 discloses a technique for performing extrapolation processing when mapping a texture image and performing image correction processing.
  • Patent Document 5 described above also discloses a technique for applying an extrapolation method when creating a three-dimensional color information conversion table for color-converting RGB signal processing system image information into YMCK signal processing system image information.
  • Patent Document 6 described above also discloses a technique for applying an extrapolation method when smoothing the periphery of a texture image.
  • JP 2009-253626 A Japanese Patent Laid-Open No. 2009-200760 JP 2004-357215 A JP 2000-299862 A JP 2005-354219 A Japanese translation of PCT publication No. 2003-504697 JP 2008-42943 A JP 2007-300380 A JP 2006-295408 A Japanese National Patent Publication No. 11-504173 JP 2008-305408 A JP 2004-206672 A
  • Patent Document 6 Patent Document 11, and Patent Document 12 described above are techniques for extrapolating high frequency components in the frequency domain in order to obtain high resolution.
  • prediction is performed from adjacent macroblocks, and the error is encoded. This is not suitable for predicting a wide range, and there is a problem that block noise occurs.
  • the present invention has been made in view of such circumstances, and can analyze various data without generating block noise, and can perform interpolation and / or extrapolation over a wide range.
  • it is an object to provide a data processing method, a data processing device, and a data processing program that can make data after extrapolation significantly natural.
  • NHA Non-Harmonic Analysis
  • This NHA has a frequency f ′ at which the sum of squares of the difference between the signal to be analyzed and the sine wave model signal represented by the phase using the frequency f ′ and the initial phase ⁇ ′ and the amplitude A ′ becomes a minimum value.
  • Amplitude A ′, and initial phase ⁇ ′ are calculated as parameters of the Fourier transform equation of the aperiodic signal.
  • the present invention uses NHA having a high frequency resolution and a small influence on the analysis window length, thereby accurately extracting and interpolating and / or extrapolating the spectrum of the cut out data. The technology that essentially solves the problem is established.
  • the data processing method according to the present invention that achieves the above-described object is a data processing method that analyzes and processes various data, and inputs the original data to be processed to the data processing device and stores it in the memory.
  • a data input step and an arithmetic means of the data processing device reads the original data inputted in the data input step and stored in the memory, and an arbitrary dimension signal based on the original data, a frequency f ′ and an initial value
  • the frequency f ′, the amplitude A ′, and the initial phase ⁇ ′ at which the sum of squares of the difference between the phase using the phase ⁇ ′ and the sinusoidal model signal represented by the amplitude A ′ is minimized.
  • the spectrum is extracted as a Fourier transform parameter of the aperiodic signal, the data is interpolated and / or extrapolated based on the extracted spectrum, and the reconstructed data is extracted. Including an interpolation and / or extrapolation step for obtaining the data.
  • a data processing apparatus that achieves the above-described object is a data processing apparatus that analyzes and processes various data, and includes data input means for inputting original data to be processed, and the data input means.
  • the sum of squares of the difference between the arbitrary dimension signal based on the original data inputted via the sine wave model signal represented by the phase using the frequency f ′ and the initial phase ⁇ ′ and the amplitude A ′ is the minimum value
  • a spectrum is extracted by obtaining the frequency f ′, the amplitude A ′, and the initial phase ⁇ ′ as follows as parameters of a Fourier transform formula of an aperiodic signal, and data interpolation processing is performed based on the extracted spectrum And / or an extrapolation process to obtain reconstruction data.
  • a data processing program that achieves the above-described object is a computer-executable data processing program that analyzes and interpolates various data, and inputs the original data to be processed to the computer.
  • the frequency f ′, the amplitude A ′, and the initial phase ⁇ ′ that minimize the sum of squares of the difference from the signal are obtained as parameters of the Fourier transform equation of the aperiodic signal, and a spectrum is extracted and extracted. It is characterized by functioning as an arithmetic means for obtaining reconstructed data by performing data interpolation processing and / or extrapolation processing based on the spectrum obtained.
  • the data processing method, the data processing apparatus, and the apparatus in which the data processing program according to the present invention is mounted use a frequency analysis method that has a high frequency resolution and is less affected by the analysis window length.
  • interpolation and / or extrapolation based on the original data can be performed without impairing the characteristics of the original data and without causing an error between frames.
  • the present invention as described above, it is possible to interpolate the missing information with high accuracy without generating block noise regardless of the shape and size of the portion where the information is missing. Data can be significantly more natural. Similarly, in the present invention, it is possible to extrapolate unknown information with high accuracy without generating block noise regardless of any data and any portion of the data. In addition, when an image is configured with a small spectrum, high frequency components are insufficient and ringing occurs at the rising edge of the edge.In the present invention, the dynamic range at the time of data reconstruction is expanded to save the data to be stored. Since rounding can be performed according to the standard, occurrence of such ringing can be reduced.
  • FIG. 6 is a flowchart showing a series of processing when image data is reconstructed by interpolation and / or extrapolation based on original image data in the data processing apparatus shown as an embodiment of the present invention.
  • FIG. 6B is a diagram showing a portion corresponding to the missing portion shown in FIG. 6A in the reconstructed image data shown in FIG.
  • FIG. 10 is a diagram showing image data reconstructed by extrapolation processing by applying FFT based on the original image data shown in FIG. 9. It is a figure which shows the image data reconfigure
  • FIG. 9 It is a figure which shows the image data reconfigure
  • FIG. 16 is a diagram for explaining the effectiveness when this frequency analysis method is applied to the “exemplar based method” algorithm, and is a diagram showing image data obtained by deleting information over a wide area with respect to the image data shown in FIG. 15. It is a figure for demonstrating the effectiveness at the time of applying this frequency-analysis method to the "exemplar” based “method” algorithm, and is a figure which shows the image data interpolated by the "exemplar” based “method” algorithm. It is a figure for demonstrating the effectiveness at the time of applying this frequency-analysis method to an "exemplar-based-method” algorithm, and is a figure which shows the image data interpolated by applying this frequency-analysis method.
  • This embodiment is a data processing device that analyzes and / or extrapolates various data such as image data.
  • this data processing apparatus applies a new frequency analysis method in which the frequency resolution does not depend on the analysis window length by estimating a Fourier coefficient by solving a nonlinear equation, and performs data interpolation and / or extrapolation. Is.
  • a case will be mainly described in which reconstruction is performed by performing interpolation based on original image data by performing interpolation processing and / or extrapolation processing.
  • the data processing apparatus is composed of, for example, a computer or the like. As shown in FIG. 1, a CPU (Central Processing Unit) 11 that centrally controls each unit and a read-only ROM (Read Read ROM that stores various types of information including various programs). Only Memory 12 and RAM (Random) that functions as a work area (Access Memory) 13, a storage unit 14 that stores various information in a readable and / or writable manner, an input operation control unit 15 that performs processing and control of an input operation via a predetermined operation device (not shown) as a user interface, And a display unit 16 for displaying various information.
  • a CPU Central Processing Unit
  • ROM Read Read ROM
  • Only Memory 12 and RAM (Random) that functions as a work area (Access Memory) 13
  • a storage unit 14 that stores various information in a readable and / or writable manner
  • an input operation control unit 15 that performs processing and control of an input operation via a predetermined operation device (not shown) as a user interface
  • the CPU 11 executes various programs including various application programs stored in the storage unit 14 and the like, and comprehensively controls each unit.
  • the ROM 12 stores various information including various programs. Information stored in the ROM 12 is read under the control of the CPU 11.
  • the RAM 13 functions as a work area when the CPU 11 executes various programs. Under the control of the CPU 11, the RAM 13 temporarily stores various information and reads the stored various information.
  • the storage unit 14 stores various types of information including original image data and mask data to be processed in addition to application programs such as a data processing program according to the present invention.
  • a hard disk or a non-volatile memory can be used as the storage unit 14.
  • the storage unit 14 also includes a drive device that reads and / or writes various types of information on a storage medium such as a flexible disk or a memory card that can be attached to and detached from the main body.
  • Various types of information stored in the storage unit 14 are read out under the control of the CPU 11.
  • the input operation control unit 15 accepts an input operation via a predetermined operation device (not shown) as a user interface such as a keyboard, a mouse, a keypad, an infrared remote controller, a stick key, or a push button, for example, and indicates operation contents.
  • a predetermined operation device such as a keyboard, a mouse, a keypad, an infrared remote controller, a stick key, or a push button, for example, and indicates operation contents.
  • a control signal is supplied to the CPU 11.
  • the display unit 16 is, for example, a liquid crystal display (Liquid Crystal).
  • Various display devices such as a display (LCD), a plasma display panel (PDP), an organic electroluminescence (Organic ElectroLuminescent) display, or a CRT (Cathode Ray Tube). Display information.
  • the display unit 16 displays the screen, and the input original image data as the processing target and the image data after interpolation as the interpolation and / or extrapolation result Etc. are displayed.
  • the data processing apparatus including each unit executes a data processing program under the control of the CPU 11, the spectrum is extracted by performing frequency analysis of the input image data under the control of the CPU 11. Then, the image data is reconstructed based on the obtained spectrum.
  • a signal to be subjected to frequency analysis that is, image data to be processed is input to the CPU 11 via a data input unit (not shown).
  • the data processing apparatus interpolates image data captured by an imaging apparatus such as a digital camera by interpolation processing and / or extrapolation processing, a predetermined interface that connects the data processing apparatus and the imaging apparatus The image data as the signal to be processed is input by storing it in the storage unit 14.
  • the data processing apparatus may input arbitrary data created by the user by storing it in the storage unit 14 as a processing target signal. That is, the data input unit is a part having a function of causing the CPU 11 to input image data as a processing target. Needless to say, the data input unit also has a function of performing A / D conversion and converting it into a digital signal when an analog signal is input. At this time, the data input unit may be an A / D converter including an anti-aliasing filter as necessary. Under the control of the CPU 11, the data processing apparatus performs interpolation processing and / or extrapolation processing by performing frequency analysis of the image data as the processing target input in this manner, and reconstructed image data. And the like are stored in the storage unit 14 via an output unit (not shown) or output to other devices.
  • the problem of obtaining the frequency parameter of the Fourier transform equation of the non-periodic signal shown in the following equation (1) is obtained as the optimal solution of the nonlinear equation. Replaced with a problem.
  • the so-called steepest descent method is applied to the frequency parameters f ′ and ⁇ ′ constituting the phase of the sine wave model signal in the above equation (2), and the frequency parameters f m ′ and ⁇ m ′ are applied. Is obtained by the following equations (3) and (4).
  • the frequency parameter A ′ as a coefficient of the sine wave model signal in the above equation (2) can be uniquely obtained.
  • the frequency parameter A m ′ is converged by equation (6).
  • this frequency analysis method it is possible to converge the amplitude A ′, the frequency f ′, and the initial phase ⁇ ′ with high accuracy by repeatedly performing these series of calculations.
  • the calculation is performed by separately obtaining the frequency parameters f ′ and ⁇ ′ constituting the phase of the sine wave model signal in the above equation (2) and the frequency parameter A ′ as a coefficient. It can be performed simply.
  • this frequency analysis method it is desirable to converge the frequency parameters f m ′ and ⁇ m ′ to some extent by applying the steepest descent method, and then to converge with high accuracy by applying the so-called Newton method. .
  • the frequency parameters f m ′ and ⁇ m ′ are obtained by the recurrence formulas shown in the following equations (7) and (8) as the Newton method.
  • J is the following formula (9) and is abbreviated as the following formula (10).
  • [nu m is also a weighting coefficient based on the reduction method in the same manner as mu m, taking the value of timely 0-1.
  • the frequency parameters A ′, f ′, and ⁇ ′ are estimated at high speed and with high accuracy by using a hybrid method combining the steepest descent method and the Newton method. Can do.
  • the spectral parameter can be approximately derived by performing successive subtraction processing.
  • the analysis target signal x (n) is the sum of a plurality of sine waves and is expressed as the following equation (11).
  • This frequency analysis method can determine the frequency f ′, the amplitude A ′, and the initial phase ⁇ ′ of the sine wave model signal at high speed and with high accuracy by obtaining the optimal solution of the nonlinear equation.
  • the inventor of the present application verified accuracy by comparing DFT and GHA (Generalized Harmonic Analysis), which is said to have the highest analysis accuracy among the developed types of DFT. .
  • DFT and GHA apparently have a plurality of window lengths in one analysis window length, so the frequency resolution depends on the analysis window length, but the resolution frequency is finite, and the signal to be analyzed If the analysis signal is different from the frequency that can be analyzed accurately, the analysis signal cannot be analyzed when the frequency is other than the decomposition frequency. A frequency (sideband component) appears, and a plurality of frequencies appear.
  • the analysis window length is one second (1024 samples) and is very short.
  • a single sine wave was analyzed, one sine wave was extracted by each method, and the square error from the original signal was examined. The result is shown in FIG.
  • this frequency analysis method can perform analysis with surprisingly high accuracy even compared to GHA, which is said to have the highest analysis accuracy.
  • this frequency analysis method is applied and shown in FIG. A series of processes are performed.
  • the data processing apparatus converts the original image data I org (n 1 , n 2 ) into a two-dimensional signal via a data input unit (not shown) under the control of the CPU 11 in step S1.
  • image data I (n 1 , n 2 ) is input and stored in a memory such as the RAM 13, the minimum L required to reconstruct the image data under the control of the CPU 11 in step S 2. It is determined whether or not a book spectrum has been extracted.
  • step S3 When the data processing apparatus has not extracted all the L spectra, in step S3, the portion of the image data I (n 1 , n 2 ) that lacks information under the control of the CPU 11 To the image data I (n 1 , n 2 ), the mask data M (n 1 , n 2 ) corresponding to the missing information is applied to the image data I (n 1 , n 2 ) as shown in the following equation (15): New image data I ′ (n 1 , n 2 ) is generated.
  • n 1 , n 2 ) 1 and can be generated by using an arbitrary mask data generation algorithm, image processing software, or the like.
  • M ′ is the inverted data of the mask data M.
  • K is a predetermined constant, and takes the average value or median value of the image data I in the case of interpolation processing.
  • the constant K in the case of extrapolation processing is not practically used because the inverted data M ′ always takes a value of 0.
  • an image is used here.
  • An arbitrary value such as an average value or a median value of the data I is assumed. That is, the data processing apparatus substitutes the value K for an unknown area in the image data I (n 1 , n 2 ) and proceeds with the process. Specifically, for example, as shown in FIG. 4A, the data processing apparatus performs mask data M (n) when processing the image data I (n 1 , n 2 ) with some information missing. 1 , n 2 ), data is prepared that takes a value of 0 for an unknown region lacking information as shown in FIG. 4B and a value of 1 for a known region.
  • the data processing apparatus may subtract the constant C from the data I ⁇ M + K ⁇ M ′ under the control of the CPU 11.
  • the data processing apparatus is able to perform extrapolation over a wide area by extracting a spectrum of only the AC component of the texture from which information unnecessary for extrapolated information is removed while reducing the amount of information.
  • the low-frequency component with little deterioration is removed by quantization.
  • the data processing apparatus when extracting a plurality of spectra, it is necessary to set a new initial value for each spectrum extraction in order to perform the calculations of the above equations (2) to (10). . Therefore, after setting an initial value under the control of the CPU 11 in step S4, the data processing apparatus sets the image data I ′ (n 1 , n 1 , n 2 , obtained in step S3 consisting of a two-dimensional signal in step S5. The frequency analysis method described above is applied to n 2 ), and the above formulas (2) to (10) are calculated.
  • step S6 the data processing apparatus extracts the l-th spectrum as shown in the following equation (16) under the control of the CPU 11.
  • f xs and f ys are sampling frequencies [Hz] in the horizontal axis direction and the vertical axis direction of the image data, respectively
  • a l ′, f xl ′, f yl ′, ⁇ l ′ is the amplitude of the spectrum to be extracted, the frequency corresponding to each axis of the image data, and the initial phase, respectively.
  • the data processing apparatus performs DFT, Fast Fourier Transform (FFT), etc.
  • FFT Fast Fourier Transform
  • the data processing apparatus applies the frequency analysis method described above to a portion of the image data I ′ (n 1 , n 2 ) where the pixel value of the mask data M (n 1 , n 2 ) is not zero.
  • the data processing device expresses image data that is a two-dimensional signal using a sine wave model function, changes parameters so that the difference between the actual signal and the sine wave model signal is minimized, and obtains each frequency.
  • step S7 the data processing device converts the image data I ′′ (n 1 , n 2 ) represented by the extracted spectrum from the image data I (n 1 , n 2 ) under the control of the CPU 11.
  • the subtracted residual signal is substituted for the image data I (n 1 , n 2 ), and in step S8, the image data I ′ (n 1 , n 2 ) and the image data I ′′ (n 1 , n 2 ) Is added to the image data I ′ (n 1 , n 2 ), and in step S9, l is incremented, the constant K is updated, and the processing from step S2 is repeated.
  • the image data I (n 1 , n 2 ) is the original image data I org (n 1 , n 2 ) itself, but the second and subsequent ones are extracted.
  • the image data I (n 1 , n 2 ) is the image data I ′′ (n) composed of the original image data I (n 1 , n 2 ) and the spectrum extracted so far. 1 , n 2 ).
  • the interpolation and / or extrapolation processing of the image data is performed as shown in the following equation (17).
  • the reconstructed image data I ′ ′′ (n 1 , n 2 ) is output, and the series of processes is terminated.
  • the image data is obtained by reconstructing the constant C ′ based on the L spectra.
  • the data processing device removes the ringing of the texture image by extending the dynamic range when reconstructing the image data to enhance the image intensity and rounding it with a predetermined threshold. That is, the data processing apparatus, for example, in the case of image data quantized with 8 bits, an arbitrary constant k is applied to the image data constituted by the extracted spectrum so that 0 and 255 are set as threshold values. It is desirable to reconstruct the image data while expanding the dynamic range of the image data by multiplying.
  • the data processing apparatus reconstructs the image data I ′ ′′ (n 1 , n 2 ) based on the original image data I org (n 1 , n 2 ) by performing such a series of processes. Can do. Since the dynamic range of the reconstructed image data I ′ ′′ (n 1 , n 2 ) is rounded according to the standard of the image data, ringing can be reduced. In addition, the data processing apparatus can extrapolate information outside the measurement region by extending the possible range of (n 1 , n 2 ) to a length exceeding the measurement region.
  • image data in which information is lost due to a plurality of dots dispersed over the entire image area was prepared.
  • the peak SNR of the image data shown in FIG. 5B with respect to the image data shown in FIG. 5A is 21.2079 dB.
  • mask data having values corresponding to these dots was prepared.
  • reconstructed image data was obtained by interpolating the missing information based on information around the portion of the image data where the information is missing.
  • the inventor of the present application also examined the influence of the shape and size of a portion where information is missing when the frequency analysis method is applied. Specifically, for the same image data as the image data shown in FIG. 5 (a), as shown in FIG. 6 (a), image data provided with three missing portions having different shapes and sizes are prepared. did. Then, reconstructed image data was obtained by interpolating the missing information based on information around the portion of the image data where the information is missing.
  • the data processing apparatus to which this frequency analysis method is applied performs interpolation processing that eliminates the generation of block noise that could not be achieved by applying the conventional frequency analysis method, and significantly reconstructed image data is obtained. It is possible to obtain.
  • Image data having the same pattern as shown in FIG. 8 is prepared, and a portion surrounded by a rectangular frame (FIG. 9) is cut out as an analysis object, that is, original image data. Based on this original image data Then, the reconstructed image data was obtained by extrapolating the surrounding unknown information. Note that the size of the original image data shown in FIG. 9 is 140 pixels ⁇ 140 pixels, and based on this, reconstructed image data having a size of 420 pixels ⁇ 420 pixels was obtained.
  • the original image data is not lost without losing the characteristics of the original image data and without generating an error between frames.
  • Extrapolation based on even when a portion surrounded by a rectangular frame line in FIG. 12 is cut out as other original image data and surrounding unknown information is extrapolated by a data processing apparatus to which the present frequency analysis method is applied, the characteristics of the original image data The extrapolation based on the original image data can be performed without impairing the image quality and without causing an error between frames.
  • the data processing apparatus can perform accurate analysis without damaging the characteristics of the original image data, regardless of what texture image or any portion of the texture image is cut out.
  • the data processing apparatus can extrapolate information outside the measurement region by extending the range that (n 1 , n 2 ) can take beyond the measurement region.
  • the image data shown in FIG. 9 has a value in the range of (n 1 , n 2 ) from 0 to 1, whereas the image data shown in FIG. 11 has a value of (n 1 , n 2 ) from ⁇ 1 to 2. It is the result of restructuring by range.
  • the inventor of the present application also examined the influence of the removal of the low frequency component in step S3 in FIG. 3 and the expansion of the dynamic range in step S4.
  • FIG. 13A shows the same original image data consisting of 140 pixels ⁇ 140 pixels as shown in FIG.
  • step S3 in FIG. 3 after subtracting the constant C of the number of quantization bits / 2 from the original image data, reconstruction is performed based on 50 spectra extracted by analysis by this frequency analysis method. Then, by adding the subtracted constant C, image data in which ringing occurred at the edge portion of the image was obtained as shown in FIG.
  • the data processing apparatus to which this frequency analysis method is applied performs extrapolation processing that eliminates the generation of block noise that could not be achieved by applying the conventional frequency analysis method, and significantly reconstructed image data is obtained. It is possible to obtain.
  • the “exemplar based method” algorithm is as shown in FIG. That is, when the “exemplar based method” algorithm extracts the initial contour ⁇ 0 of the region ⁇ selected as the interpolation target for the original data, the contour ⁇ t is identified and prioritized in the t-th iteration in step S21.
  • the degree P (p) is calculated.
  • the contour [Delta] [omega t is 0 and ends the series of processes.
  • step S23 a sample ⁇ q ′ ⁇ that minimizes the distance d ( ⁇ p ′ , ⁇ q ′ ) between the two patches ⁇ p ′ and ⁇ q ′ is searched, and in step S24, from the patch ⁇ q ′. Copy the image data to the patch ⁇ p ′ . Note that the relationship of ⁇ p ⁇ p ′ ⁇ is satisfied.
  • exemplar based method algorithm, in step S25, the updated reliability term C a (p) such that ⁇ p ⁇ p ' ⁇ .
  • the inventor of the present application tried to interpolate over a wide area by applying this frequency analysis method by replacing the processing of step S22 to step S24 in the “exemplar based method” algorithm with this frequency analysis method.
  • the image data shown in FIG. 15 was processed to create image data in which information was lost over a wide area, as indicated by A to I in FIG. 16, and this was input and processed as original image data.
  • the data processing apparatus to which the present frequency analysis method is applied is extremely effective when performing interpolation over a wide area.
  • the data processing apparatus to which this frequency analysis method is applied can interpolate the missing information over a wide range with high accuracy without generating block noise regardless of the shape and size of the missing portion. And can make the data after interpolation much more natural.
  • the data processing apparatus can extrapolate unknown information with high accuracy without generating block noise regardless of what texture image or any portion of the texture image is cut out.
  • this data processing device can reduce ringing because it can be rounded in accordance with the standard of data to be stored by extending the dynamic range at the time of data reconstruction. As a result, in this data processing apparatus, only a small number of spectra need be retained in order to reconstruct a texture image from which ringing has been removed. By extrapolating surrounding unknown information, a wide range of information can be obtained with less information.
  • a texture image can be expressed.
  • Such a data processing apparatus is applied to a purpose of highly accurately interpolating unknown information lost due to “whiteout” or “blackout” of captured image data based on surrounding information or remaining information. It is also possible to convert an image represented by 8 bits per color into an arbitrary dynamic range. Therefore, this data processing apparatus is extremely useful in view of the recent expansion trend of the digital camera market.
  • a data processing apparatus to which the frequency analysis method is applied is a photograph taken by a user with a digital camera or the like when generating a texture used in 3D modeling software or image processing software. It is possible to freely cut out an arbitrary part included in the image and paste it as a texture on the surface of a three-dimensional object or the like, and to bring a more realistic texture expression. Further, this data processing apparatus can be expanded to an arbitrary size by performing extrapolation processing even when the cut out texture is small, and can be mapped to an arbitrary surface. Thus, this data processing and insertion device is extremely useful in light of the demands of the computer graphics market.
  • the present invention is not limited to the embodiment described above.
  • the interpolation processing and / or extrapolation processing of image data has been described.
  • the present invention can be applied to audio data such as a purpose of repairing audio data in which a burst error has occurred due to noise mixing.
  • the present invention can be applied to interpolation processing and / or extrapolation processing of arbitrary data such as moving image data that is three-dimensional data as well as one-dimensional data such as data.
  • the present invention can perform interpolation by interpolation regardless of the shape and size of a portion where information is lost. For example, this can be performed by converting an analog signal into a digital signal. In this case, even when the sampling interval of the A / D converter becomes unequal due to the influence of the jitter of the sampling clock, it is shown that the processing can be performed with high accuracy. In other words, the present invention can interpolate the missing information with high accuracy over a wide range even when using an inexpensive A / D converter with low accuracy, and contributes to reduction of system cost. be able to.
  • the data processing apparatus has been described as performing frequency analysis by software.
  • the present invention implements an algorithm for data interpolation processing and / or extrapolation processing including this frequency analysis method. If a product-sum operation can be performed, such as a DSP (Digital Signal Processor), it can also be realized by hardware.
  • DSP Digital Signal Processor

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Health & Medical Sciences (AREA)
  • Theoretical Computer Science (AREA)
  • Computational Linguistics (AREA)
  • Signal Processing (AREA)
  • General Physics & Mathematics (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Human Computer Interaction (AREA)
  • Acoustics & Sound (AREA)
  • Multimedia (AREA)
  • Image Processing (AREA)
  • Image Analysis (AREA)

Abstract

L'invention porte sur un dispositif de traitement de données dans lequel, sans la survenue d'un bruit de bloc, chaque type de données peut être analysé et interpolé et/ou extrapolé, et des données post-interpolation et/ou post-extrapolation peuvent être considérablement naturelles. Lorsque les données d'image originales, qui deviennent la cible de traitement, sont appliquées en entrée, le dispositif de traitement de données calcule une fréquence f', une amplitude A' et une phase initiale φ à titre de paramètres de transformation de Fourier acyclique de telle manière que la somme des carrés de la différence entre un signal de dimension arbitraire basé sur les données originales et un signal de modèle sinusoïdal, représenté par l'amplitude A' et une phase utilisant la fréquence f' et la phase initiale φ, prenne la plus petite valeur, et extrait un spectre. Le dispositif de traitement de données interpole et/ou extrapole les données sur la base du spectre extrait et calcule les données de reconstruction.
PCT/JP2011/050534 2010-01-14 2011-01-14 Procédé de traitement de données, dispositif de traitement de données et programme de traitement de données WO2011087083A1 (fr)

Applications Claiming Priority (6)

Application Number Priority Date Filing Date Title
JP2010-006234 2010-01-14
JP2010006234A JP2013084019A (ja) 2010-01-14 2010-01-14 データ外挿方法、データ外挿装置、及びデータ外挿プログラム
JP2010-007679 2010-01-18
JP2010007679 2010-01-18
JP2010214591 2010-09-25
JP2010-214591 2010-09-25

Publications (1)

Publication Number Publication Date
WO2011087083A1 true WO2011087083A1 (fr) 2011-07-21

Family

ID=44304352

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2011/050534 WO2011087083A1 (fr) 2010-01-14 2011-01-14 Procédé de traitement de données, dispositif de traitement de données et programme de traitement de données

Country Status (1)

Country Link
WO (1) WO2011087083A1 (fr)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104504660A (zh) * 2014-12-23 2015-04-08 北京数码大方科技股份有限公司 用于网格模型孔洞解除的数据处理方法及装置
CN106530223A (zh) * 2016-11-28 2017-03-22 清华大学 基于频域调制的快速Fourier鬼成像方法及系统
JPWO2021039180A1 (fr) * 2019-08-28 2021-03-04

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH1153534A (ja) * 1997-07-31 1999-02-26 Sony Corp 画像の修復方法及び装置
WO2009038056A1 (fr) * 2007-09-20 2009-03-26 National University Corporation University Of Toyama Procédé d'analyse de signal, dispositif d'analyse de signal et programme d'analyse de signal

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH1153534A (ja) * 1997-07-31 1999-02-26 Sony Corp 画像の修復方法及び装置
WO2009038056A1 (fr) * 2007-09-20 2009-03-26 National University Corporation University Of Toyama Procédé d'analyse de signal, dispositif d'analyse de signal et programme d'analyse de signal

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
CRIMINISI, A. ET AL.: "Region filling and object removal by exemplar-based image inpainting", IEEE TRANSACTIONS ON IMAGE PROCESSING, vol. 13, no. 9, September 2004 (2004-09-01), pages 1200 - 1208 *

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104504660A (zh) * 2014-12-23 2015-04-08 北京数码大方科技股份有限公司 用于网格模型孔洞解除的数据处理方法及装置
CN106530223A (zh) * 2016-11-28 2017-03-22 清华大学 基于频域调制的快速Fourier鬼成像方法及系统
CN106530223B (zh) * 2016-11-28 2020-01-10 清华大学 基于频域调制的快速Fourier鬼成像方法及系统
JPWO2021039180A1 (fr) * 2019-08-28 2021-03-04
WO2021039180A1 (fr) * 2019-08-28 2021-03-04 日本電気株式会社 Dispositif de traitement d'image, procédé de traitement d'image et support d'enregistrement lisible par ordinateur
JP7218812B2 (ja) 2019-08-28 2023-02-07 日本電気株式会社 画像処理装置、画像処理方法、及びプログラム

Similar Documents

Publication Publication Date Title
US8639056B2 (en) Contrast enhancement
JP6432214B2 (ja) 画像処理装置、画像処理方法、記憶媒体及びプログラム
JPH11505350A (ja) ピラミッド画像表示におけるウィーナ変形フィルタを用いる画像ノイズリダクションシステム
JP4776705B2 (ja) 画像処理装置および方法
JP2012060517A (ja) 画像処理装置、画像処理方法
Vishnukumar et al. Single image super-resolution based on compressive sensing and improved TV minimization sparse recovery
JPH08294001A (ja) 画像処理方法および画像処理装置
Agrawal et al. Image resolution enhancement using lifting wavelet and stationary wavelet transform
WO2011087083A1 (fr) Procédé de traitement de données, dispositif de traitement de données et programme de traitement de données
JP6276596B2 (ja) 空間超解像・階調補間装置及びプログラム
JP2019087778A (ja) 画像復元フィルタ及び学習装置
JP6174903B2 (ja) 空間・階調超解像装置及びプログラム
JP6202725B2 (ja) 画像差分量算出装置、画像符号化装置及びプログラム
KR101341617B1 (ko) 단일 영상의 오차모델을 기반으로 한 고해상도 영상 복원장치 및 방법
Ono et al. Optimized JPEG image decompression with super-resolution interpolation using multi-order total variation
JP6532151B2 (ja) 超解像装置およびプログラム
JP2002183727A (ja) 画像処理装置
JP2002074356A (ja) 画像処理方法および装置並びに記録媒体
JP2013084019A (ja) データ外挿方法、データ外挿装置、及びデータ外挿プログラム
Thai et al. Image tone mapping approach using essentially non-oscillatory bi-quadratic interpolations combined with a weighting coefficients strategy
JP2011049696A (ja) 画像処理装置及び画像処理方法
JP5837844B2 (ja) 擬似輪郭領域検出装置及びプログラム
KR101803065B1 (ko) 영상 처리 방법 및 그 장치
JP2000312294A (ja) 画像処理装置
JP2001056856A (ja) 画像処理方法および装置並びに記録媒体

Legal Events

Date Code Title Description
DPE2 Request for preliminary examination filed before expiration of 19th month from priority date (pct application filed from 20040101)
NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 11732953

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: JP