KR101906173B1 - Method and Device of Zoom Motion Estimation for Depth Video - Google Patents

Abstract

According to an embodiment of the present invention, a motion estimation method using expansion in depth images includes: a block division step of dividing a current depth screen to target blocks in a first preset size set in advance; selecting search blocks in a search area, which is in a second preset size, of a reference depth screen being the previous or next frame of the current depth screen; calculating an expansion ratio based on a first average value of depth values of pixels in the target blocks and a second average value of depth values of the pixels in the search blocks; expanding the search blocks in accordance with the expansion ratio; calculating motion estimation difference signals between the target blocks and the expanded search blocks; and determining the motion estimation difference signals between the individual ones of the expanded search blocks in the search area and target blocks.

Description

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a method and an apparatus for motion estimation using stretching in a depth image,

The present invention relates to a motion estimation method and apparatus using stretching in a depth image. More particularly, the present invention relates to a motion estimation method and apparatus that improves motion estimation performance by eliminating the influence of distances between an object and a camera.

The motion estimation means to find the closest block in the screen adjacent to the block in the current screen with respect to the color signal or brightness signal. Usually, the smaller the block size, the larger the accuracy of estimation, but the complexity required for estimation is increased.

In MPEG-1 and 2, blocks are composed of 16 pixels in the horizontal direction and 16 pixels in the vertical direction with respect to the lightness signal. In H. 264, block sizes of 16x16, 8x8, 4x4, etc. can be variably selected.

MSE (Mean Square Error) and MAE (Mean Absolute Error) are used as evaluation scales for motion estimation. The accuracy of MSE is excellent, but MAE is widely used because of its high complexity.

Various studies have been made to reduce the complexity in stretching motion estimation for a color signal or brightness signal. A method of estimating stretching motion by generating reference frames of various stretching ratios, various methods of improving repetitive least-squares estimation considering calculation efficiency and accuracy, and the like, by using block-based fast scaling . A method of estimating stretching motion by generating reference frames of various stretching ratios among stretching motion estimation methods in existing color images generates a stretching reference frame with various stretching ratios for each reference frame in time. After that, the most similar block among the reference screens is selected for the block of the current image, and the new and expanded blocks are estimated. In this method, a reference screen for a plurality of expansion / contraction ratios must be newly generated for each reference screen in order to estimate the expansion / contraction motion. Therefore, in order to increase accuracy in the expansion motion estimation, a reference screen is generated for many expansion / contraction ratios There is a problem that the computational complexity becomes large because a process of referring to it must be added.

 The method of estimating the expansion / contraction ratio only through the color information of the camera has a great difficulty in estimating the accurate expansion ratio and has a limitation in improving the calculation complexity. To overcome this difficulty, a new motion estimation apparatus and an estimation method thereof in Korean Patent Registration No. 10-1371826 propose motion estimation and motion compensation by estimating the expansion / contraction ratio of an object using a depth camera, While improving the accuracy of motion estimation. However, although the prior art reflects the influence of the change in the size of the object in the depth image as the object moves in the direction of the camera in calculating the motion estimation difference signal, it does not remove the influence of the change in the value of the pixel having the depth value , There is a limitation that it is limited to the encoding of a color image.

KR 10-1371826 B1

An object of the present invention is to provide a motion estimation method using a stretching method in a depth image capable of estimating motion by removing the influence of a change in size of an object in a depth image and a change in a pixel value as an object moves in a camera direction or in a camera direction And an apparatus.

It is another object of the present invention to provide a motion estimation method and apparatus using stretching in a depth image that can reduce a computation amount and a memory storage amount by applying a stretching operation only to a search block in a reference picture.

The embodiment includes a block dividing step of dividing a current depth screen into target blocks of a first size set in advance; Selecting a search block within a search area of a predetermined second size of a reference depth screen that is a previous frame or a next frame of the current depth screen; Calculating an expansion ratio based on a first average value of depth values of pixels in the object block and a second average value of depth values of pixels in the search block; Expanding and contracting the search block according to the expansion ratio; Calculating a motion estimation difference signal between the target block and the retracted search block; And a step of determining a motion estimation difference signal between the target block and each of the plurality of retracted search blocks in the search area.

In another aspect, an embodiment includes calculating an average absolute error or a mean square error for a determined plurality of motion estimation difference signals; And determining the relative coordinate information of the target block and the search block corresponding to the motion estimation difference signal having the minimum absolute error or mean square error as a motion vector and an expansion ratio of the search block as a motion vector, It is possible to provide a motion estimation method using stretching.

In yet another aspect, an embodiment includes calculating a motion estimation difference signal between the target block and a retracted search block, interpolating the retracted search block to determine a size of the retracted search block to a first size ; And calculating a motion estimation difference signal between the interpolated search block and the target block.

을 충족하고, X는 기 설정된 상수인 깊이 영상에서 신축을 이용한 움직임 추정 방법을 제공할 수도 있다.In another aspect, the stretch ratio is

, And X may provide a motion estimation method using stretching in a depth image which is a predetermined constant.

According to another aspect of the present invention, the step of calculating a motion estimation difference signal between the target block and the retracted search block may include: calculating a difference between the depth value of the pixels in the target block and the first average value, Calculating a second difference value by subtracting the second average value from the depth value of the pixels in the new and expanded search block to generate a second difference and expanded search block, A motion estimation method using stretching can be provided in a depth image for calculating a difference signal.

According to another aspect, the step of expanding and contracting the search block according to the expansion / contraction ratio provides a motion estimation method using expansion and contraction in a depth image for enlarging or reducing the size of each of the width and height of the search block at the expansion / contraction ratio It is possible.

In yet another aspect, an embodiment includes at least one memory; And at least one processor, wherein the at least one processor divides the current depth screen into target blocks of a first size of a predetermined size, 2 size search area and calculates an expansion ratio based on a first average value of depth values of pixels in the object block and a second average value of depth values of pixels in the search block, The motion estimation difference signal between the target block and the retracted search block is calculated and the motion estimation difference signal between each of the plurality of retracted search blocks in the target block and the search area is calculated It is possible to provide a motion estimation apparatus using stretching in a depth image configured to be determined.

In another aspect, the processor calculates an average absolute error or a mean square error for the determined plurality of motion estimation difference signals, and calculates a mean absolute error or a mean square error for the target block And a depth estimation unit configured to determine the relative coordinate information of the search block and the expansion ratio of the search block as a motion vector.

According to another aspect of the present invention, the processor may further comprise: a difference value calculation unit configured to subtract the first average value from the depth value of the pixels in the target block to generate a difference- And calculating a motion estimation difference signal between the difference-computed target block and the difference-computed and expanded search block, the motion estimation unit . ≪ / RTI >

In another aspect, there is provided a non-transitory computer readable medium having stored thereon instructions executable by the processor to cause at least one processor to perform operations, the operations comprising: A second block having a first size and a second block having a first depth and a second depth corresponding to a depth of the current depth screen, Calculating an expansion ratio based on a first average value and a second average value of depth values of pixels in the search block, expanding and contracting the search block according to the expansion ratio, Calculating a signal, and determining a plurality of new navigation blocks in the target block and the search area, And determining a motion estimation difference signal between each of the plurality of motion vectors.

The embodiment of the present invention can improve the motion estimation by removing the influence of the size and the variation of the depth value due to the movement of the object.

In addition, the embodiment of the present invention can reduce the computation amount and the memory capacity by applying the expansion and contraction operation only to the search block.

Also, since the motion estimation difference signal between the search block and the target block in the search area is calculated, the complexity of motion estimation can be reduced and the amount of calculation can be reduced.

1 is a block diagram illustrating a motion estimation apparatus for estimating motion using stretching in a depth image according to an embodiment of the present invention.
2 is an exemplary diagram of a motion estimation method in accordance with an embodiment of the present invention.
FIG. 3 is an exemplary view illustrating a current depth screen in which a target block is displayed, a reference depth screen in which a search block and a new and expanded search block are displayed.
4 is an exemplary view of the current and reference depth screens.
FIG. 5 illustrates a motion vector obtained according to a conventional motion estimation method and a motion vector obtained according to an embodiment of the present invention.

BRIEF DESCRIPTION OF THE DRAWINGS The present invention is capable of various modifications and various embodiments, and specific embodiments are illustrated in the drawings and described in detail in the detailed description. The effects and features of the present invention and methods of achieving them will be apparent with reference to the embodiments described in detail below with reference to the drawings. However, the present invention is not limited to the embodiments described below, but may be implemented in various forms. In the following embodiments, the terms first, second, and the like are used for the purpose of distinguishing one element from another element, not the limitative meaning. Also, the singular expressions include plural expressions unless the context clearly dictates otherwise. Also, the terms include, including, etc. mean that there is a feature, or element, recited in the specification and does not preclude the possibility that one or more other features or components may be added. Also, in the drawings, for convenience of explanation, the components may be exaggerated or reduced in size. For example, the size and thickness of each component shown in the drawings are arbitrarily shown for convenience of explanation, and thus the present invention is not necessarily limited to those shown in the drawings.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings, wherein like reference numerals refer to like or corresponding components throughout the drawings, and a duplicate description thereof will be omitted .

1 is a block diagram illustrating a motion estimation apparatus for estimating motion using stretching in a depth image according to an embodiment of the present invention.

The motion estimation apparatus 100 may provide the video data to the data reception apparatus 200. [

The motion estimation apparatus 100 and the data receiving apparatus 200 may be used in various applications such as desktop computers, laptop computers (e.g., laptop computers), tablet computers, set top boxes, telephone handsets such as so- Pads, televisions, cameras, display devices, digital media players, video gaming consoles, video streaming devices, and the like.

In some implementations, the motion estimation apparatus 100 and the data receiving apparatus 200 may be provided with a configuration 10 for wireless communication.

Also, the data receiving apparatus 200 may receive the image data processed through the computer readable medium.

The computer-readable medium may include any type of media or device capable of moving video data imaged from the motion estimation apparatus 100 to the data receiving apparatus 200. In one example, the computer-readable medium may include a communication medium, such as a transmission channel, that enables the motion estimation apparatus 100 to transmit video data directly to the data receiving apparatus 200 in real time.

The processed image data may be modulated according to a communication standard, such as a wireless communication protocol, and transmitted to the data receiving apparatus 200. The communication medium may comprise any wireless or wired communication medium, such as a radio frequency spectrum or one or more physical transmission lines. The communication medium may form part of a packet based network, such as a global network such as a local area network, a wide area network, or the Internet. The communication medium may include routers, switches, base stations, or any other equipment that may be useful in facilitating communication from the motion estimation apparatus 100 to the data receiving apparatus 200. In some instances, image processed image data may be output from the output interface 130 to a computer readable storage medium, such as a non-transitory computer readable storage medium, i.e., a data storage device. Similarly, the image data may be accessed from the storage device by the input interface 230 of the data receiving apparatus 200. [ The storage device may be a variety of distributed or removable media such as hard drives, Blu-ray discs, DVDs, CD-ROMs, flash memory, volatile or nonvolatile memory, or any other suitable digital storage media for storing image data. And non-volatile data storage media that are locally accessed. In a further example, the storage device may correspond to a file server or other intermediate storage device that may store the image data generated by the motion estimation apparatus 100.

The data receiving apparatus 200 may access the stored image data from the storage device through streaming or downloading.

In the example of FIG. 1, the motion estimation apparatus 100 may include an image source 110 and an image processing unit 120. In addition, the motion estimation apparatus 100 may further include an output interface 130.

The data receiving apparatus 200 may include an input interface 230 and a data processing unit 220. In addition, the data receiving apparatus 200 may further include a display device 210.

In another example, the motion estimation apparatus 100 and the data processing unit 220 may include other components.

For example, the motion estimation apparatus 100 may receive an external video source, such as an image from an external camera, and the external camera may be a depth image capturing device that generates a depth image. Likewise, the data receiving device 200 may interface with the external display device 210 rather than with the integrated display device 210.

The image source 110 of the motion estimation apparatus 100 may include a depth image capture device, such as a camera, an archive containing previously captured depth images, and / or an interface for receiving depth images from a depth image content provider . ≪ / RTI >

In some embodiments, the depth imaging device may provide a depth image that represents scene depth information in 256-level 8-bit images or the like. The number of bits for representing one pixel of the depth image can be changed instead of 8 bits. The depth image capturing device can measure the distance from the depth image capturing device to the object and the background by using infrared rays or the like to provide a depth image having a value proportional or inversely proportional to the distance.

The pixel value of the depth image may be, for example, depth information in the form of integers in mm, but not limited to, RGB color information, for example.

Each of motion estimation apparatus 100 and data receiving apparatus 200 may include one or more memories and one or more microprocessors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays FPGAs), discrete logic circuits, software, hardware, firmware, or any combination thereof.

The memory includes instructions (e.g., executable instructions) such as computer readable instructions or processor readable instructions. The instructions may include one or more instructions executable by the computer, such as by each of the one or more processors.

For example, the one or more instructions may be executable by one or more processors to perform operations including determining a motion vector of the depth image and processing the depth image to encode the depth image have.

In particular, the image processing unit 120 may include one or more memories 121 for storing instructions and at least one processor 122 for executing the instructions.

The processor 122 of the image processing unit 120 may be configured to apply the techniques for determining the motion vector of the depth image.

In some implementations, the processor 122 of the image processing unit 120 may be configured to apply techniques for encoding depth images.

In another embodiment, the processor 122 of the image processing unit 120 may be configured to apply a technique of encoding and decoding a depth image.

The data processing unit 220 may be configured to transmit image data from the image processing unit 120 to an external device, perform display, analysis, and the like.

In some implementations, the data processing unit 220 may be configured to decode the encoded image data from the image processing unit 120.

Although not shown in FIG. 1, in some embodiments, the motion estimation apparatus 100 and the data processing apparatus may be an integrated apparatus. For example, the motion estimation apparatus 100 may be configured to encode the depth image and to decode the encoded image data.

2 is an exemplary diagram of a motion estimation method in accordance with an embodiment of the present invention. 3 is an exemplary view of a current depth screen in which a target block is displayed, a reference depth screen in which a search block and a search block to which a new axis is applied are displayed, FIG. 4 is an exemplary view of a current depth and a reference depth screen, The motion vector obtained according to the estimation method and the motion vector obtained according to the embodiment of the present invention.

Referring to FIGS. 2 to 4, a motion estimation method according to an embodiment of the present invention includes a block division step (S100) of a current depth screen for dividing a current depth screen into a preset size, A step S300 of calculating an average value of depth values of pixels in a block for calculating average values of depth values of a current block and a current block of a current depth screen, (S500) for expanding and contracting the size of the search block using the calculated expansion ratio, a depth of the target block (S500) for expanding and contracting the size of the search block using the calculated expansion ratio, A step S6 (step S6) of removing the influence of a change in the depth value due to the movement of the object by using the depth value and the average value of the search block, 00), calculating a difference signal between the target block and the search block to determine a motion estimation difference signal (S700), and calculating the coordinates of the search block and the expansion ratio based on the minimum value of the motion estimation difference signal for the target block, And a target block motion vector determination step (S800) for determining the target block motion vector as a motion vector.

Each step of the motion estimation method according to the embodiment of the present invention will be described in more detail.

A block division step (S100) of a current depth screen for dividing a current depth screen into preset sizes.

The processor 122 of the motion estimation apparatus 100 may divide the current depth image into blocks of a predetermined first size.

For example, the depth image can be divided into blocks of m * n. Each of the plurality of blocks can be defined as an area made up of m * n (m, n is a natural number) pixels.

For example, a block of m * n can be defined as an area consisting of 8 * 8 pixels or 16 * 16 pixels. When the resolution is increased, the basic unit can be defined as an area consisting of 32 * 32 or 64 * 64 pixels. However, the present invention is not limited thereto, and the processor 122 may calculate the motion estimation difference signal of the depth image without dividing the depth image.

In addition, the predetermined first size may be predetermined in consideration of the accuracy and complexity inversely proportional to each other.

For example, as the size of a block decreases, the accuracy of motion estimation increases. However, since the number of blocks increases, the complexity of an operation required for estimation increases and information of a motion vector increases. Therefore, the first size of the block can be set in consideration of the accuracy and the complexity.

In addition, the processor 122 may divide the current depth screen into a plurality of blocks, and store information on the positions of the divided blocks and depth values in the pixels in the memory 121. [

A search block selection step (S200) of selecting a search block having a preset size in a search area of a reference depth screen .

The processor 122 of the motion estimation apparatus 100 determines whether or not the depth of the previous frame of the current depth screen or the depth of the next frame of the current depth screen, A search block of a first size can be selected. Here, the search area of the second size may be a size of K * L (K> M, L> N, K and L are natural numbers).

In addition, the reference point of the search area may be located at the same spatial position with respect to the position of the target block as a motion estimation target among a plurality of blocks of the current depth screen. That is, it can be the same position as the position of the target block in the reference depth screen.

In addition, the processor 122 may select a first size search block in a search area of a second size about a reference point.

In addition, the following operations may be performed for each search block, with each of all the blocks in the search area being a search block.

In addition, the processor 122 may store in the memory 121 information of the position of the search block and the depth value in the pixel.

A step (S300) of calculating an average value of depth values of pixels in a block for calculating an average value of depth values of a current block and a search block of a current depth screen.

The processor 122 may calculate a first average value Rc of depth values of pixels in the object block and a second average value Rr of depth values of pixels in the search block.

For example, when an object approaches the depth imaging device, the size of the object in the reference depth screen is larger than the size of the object in the current depth screen, and the depth value of the pixels corresponding to the object has a smaller value. Therefore, the first average value Rc may be larger than the second average value Rr.

On the other hand, when the object moves away from the depth imaging device, the size of the object in the reference depth screen is smaller than the size of the object in the current depth screen, and the depth value of the pixels corresponding to the object has a larger value. Therefore, the first average value Rc may be smaller than the second average value Rr.

In addition, the processor 122 may store the calculated first and second average values Rc and Rr in the memory 121.

A stretching ratio calculation step (S400) of calculating an expansion ratio using an average value of the target block and an average value of the search block.

The processor 122 may calculate a stretch ratio S that satisfies Equation 1 using the first and second average values Rc and Rr.

[Equation 1]

In Equation (1), x is a predetermined constant value according to the focus of the depth imaging device.

In addition, the processor 122 may store the calculated expansion ratio S in the memory 121. [

And expanding and contracting the size of the search block using the calculated expansion / contraction ratio (S500) .

The processor 122 can expand and contract the size of the search block using the calculated expansion ratio S. That is, the processor 122 can relatively reduce the amount of computation and the memory capacity by expanding and contracting the search block instead of expanding and contracting the size of the reference depth screen by using the expansion / contraction ratio (S).

The processor 122 may expand the search block to a size of (S * m) * (S * n) by stretching the stretch ratio S to both the width and the length of the search block first size.

For example, if the object in the reference depth screen is closer to the depth imaging device than the object in the current depth screen, the average of the depth values of the pixels in the search block is smaller than the average depth value in the target block. The expansion / contraction ratio S becomes larger than 1. Therefore, the size of the search block is enlarged.

In addition, the processor 122 may store the position information and the depth value information of the pixels belonging to the enlarged search block in the memory 121. [

(S600) for removing the influence of the depth value change due to the movement of the object using the depth value and the average value of the object block and the depth value and the average value of the search block.

For example, in a depth image, as an object moves toward a depth image pickup device, not only the size of the object changes in the depth image but also the value of the pixel made of the depth value changes. When the same object is stretched, the relative depth value distribution is similar for the pixels constituting the same object even if the depth value changes according to the expansion and contraction of the object. Therefore, the correct difference signal can be obtained by removing the influence of the distance between the object and the depth imaging device in the stretching motion estimation.

For this, the processor 122 performs a difference operation by subtracting the first average value Rc from the depth value of all the pixels of the current block of the current depth screen before calculating the motion estimation difference signal to generate a difference-processed target block, And subtracting the second average value (Rr) from the depth value of the pixels of the first search block to generate a second difference search block. Therefore, the effect of the change of the depth value of the moving object toward (or away from) the depth image capturing device by matching the average of the pixel values of the expanded and retrieved search block to the object block to which the difference operation is applied, .

The target block and the search block Car signal  Calculate motion estimation Car signal  (S700).

The processor 122 uses the interpolation method such as linear interpolation or cubic interpolation for the expanded search block so that the size of the expanded search block is made to coincide with the size of the target block.

The interpolation method can be applied to any interpolation method that can adjust the size of the block, and is not limited to the specific interpolation method.

In addition, the processor 122 determines a motion estimation difference signal by calculating a difference signal between a search block and a target block whose sizes coincide with each other, and calculates a MAE (Mean Absolute Error) or an MSE A mean square error) is calculated.

In addition, the processor 122 calculates the difference signal between the search block and the target block according to the above-described procedure for each block in the search area, and calculates either MAE or MSE for the calculated difference signal.

Motion estimation for target block Of the difference signal The minimum value  A target block motion vector determination step (S800) for determining the coordinates of the search block and the expansion ratio as a motion vector of the target block.

The processor 122 may select the search block having the calculated MAE (or MSE) minimum value and determine the relative coordinate information of the target block and the search block and the stretch ratio S of the search block as a motion vector.

Referring to FIG. 5, it can be seen that the motion vector according to the motion estimation method according to the embodiment of the present invention appears more accurately compared with the motion vector according to the conventional motion estimation method for the depth image of FIG. 4 (comparative example).

In addition, the motion estimation performance (Comparative Example 1) in which the expansion and contraction is not applied (Equation 2) and the motion estimation performance in the state in which the influence of the change in the depth value of the object is not removed (Comparative Example 2) The MES in a state in which the expansion or contraction is not applied or the influence of the change of the depth value of the object is not removed is smaller than the MSE of the motion estimation according to the embodiment of the present invention It can be seen that the motion estimation according to the embodiment of the present invention is more accurate.

&Quot; (2) "

Equation (2) represents the sum of the squares of the difference between the target block I and the block K in which the intra-picture motion estimation is predicted.

MSE Comparative Example 1 136.91 Comparative Example 2 98.17 Example 77.16

The embodiments of the present invention described above can be implemented in the form of program instructions that can be executed through various computer components and recorded in a computer-readable recording medium. The computer-readable recording medium may include program commands, data files, data structures, and the like, alone or in combination. The program instructions recorded on the computer-readable recording medium may be those specifically designed and configured for the present invention or may be those known and used by those skilled in the computer software arts. Examples of computer-readable media include magnetic media such as hard disks, floppy disks and magnetic tape, optical recording media such as CD-ROM and DVD, magneto-optical media such as floptical disks, medium, and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. Examples of program instructions include machine language code, such as those generated by a compiler, as well as high-level language code that can be executed by a computer using an interpreter or the like. The hardware device may be modified into one or more software modules for performing the processing according to the present invention, and vice versa.

The specific acts described in the present invention are, by way of example, not intended to limit the scope of the invention in any way. For brevity of description, descriptions of conventional electronic configurations, control systems, software, and other functional aspects of such systems may be omitted. Also, the connections or connecting members of the lines between the components shown in the figures are illustrative of functional connections and / or physical or circuit connections, which may be replaced or additionally provided by a variety of functional connections, physical Connection, or circuit connections. Also, unless explicitly mentioned, such as " essential ", " importantly ", etc., it may not be a necessary component for application of the present invention.

While the present invention has been described in connection with what is presently considered to be practical exemplary embodiments, it is to be understood that the invention is not limited to the disclosed embodiments, but, on the contrary, It will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention as defined by the appended claims. Therefore, the technical scope of the present invention should not be limited to the contents described in the detailed description of the specification, but should be defined by the claims.

100: motion estimation device
110: image source
120:
121: Memory
122: Processor
130: Output interface
200: Data receiving device
210: Display device
220:
230: Input Interface

Claims (10)

A block dividing step of dividing a current depth screen into target blocks of a first size set in advance;
Selecting a search block within a search area of a predetermined second size of a reference depth screen that is a previous frame or a next frame of the current depth screen;
Calculating an expansion ratio based on a first average value of depth values of pixels in the object block and a second average value of depth values of pixels in the search block;
Expanding and contracting the search block according to the expansion ratio;
Calculating a motion estimation difference signal between the target block and the retracted search block; And
Determining a motion estimation difference signal between the target block and each of the plurality of stretched search blocks in the search area,
Wherein the step of calculating a motion estimation difference signal between the target block and the retracted search block includes a step of removing a depth value variation amount,
In the removal of the depth value variation amount,
Calculating a difference value between the depth values of the pixels in the target block and the first average value to generate a difference-
A second average value is calculated by subtracting the second average value from the depth value of the pixels in the new and expanded search block,
Calculating a motion estimation difference signal between the difference-computed target block and the difference computed and stretched search block
Motion Estimation Method Using Extension in Depth Image. The method according to claim 1,
Calculating an average absolute error or a mean square error for the determined plurality of motion estimation difference signals; And
And determining the relative coordinate information of the target block and the search block corresponding to the motion estimation difference signal having the minimum absolute error or mean square error as a motion vector and the expansion ratio of the search block as a motion vector
Motion Estimation Method Using Extension in Depth Image. delete The method according to claim 1,
The stretch ratio

, And X is a predetermined constant
Motion Estimation Method Using Extension in Depth Image. delete The method according to claim 1,
Wherein the step of expanding and contracting the search block according to the expansion /
And enlarging or reducing the size of each of the width and height of the search block by the expansion ratio
Motion Estimation Method Using Extension in Depth Image. At least one memory; And
At least one processor,
The at least one processor
The current depth screen is divided into target blocks of a first size set in advance,
Selecting a search block in a search area of a second predetermined size of a reference depth screen, which is a previous frame or a next frame of the current depth screen,
Calculating an expansion ratio based on a first average value of depth values of pixels in the object block and a second average value of depth values of pixels in the search block,
Expanding and contracting the search block according to the expansion ratio,
Calculating a motion estimation difference signal between the target block and the retracted search block,
And to determine a motion estimation difference signal between the target block and each of the plurality of stretched search blocks in the search area,
The processor comprising:
Calculates an average absolute error or a mean square error for the determined plurality of motion estimation difference signals,
And determine the relative coordinate information of the target block and the search block corresponding to the motion estimation difference signal having the minimum absolute error or mean square error as a motion vector and an expansion ratio of the search block,
Wherein the processor is configured to remove a depth value variation amount,
The depth value variation amount removal may include:
Calculating a difference between the depth value of all pixels in the target block and the first average value to generate a difference-
A second average value is calculated by subtracting the depth value of the pixels within the expanded and expanded search block from the second average value,
And calculating a motion estimation difference signal between the difference-computed target block and the difference computed and expanded search block
A motion estimation apparatus using stretching in a depth image. delete delete 17. A non-transitory computer readable medium storing instructions,
Wherein the instructions are executable by the processor to cause at least one processor to perform operations,
The operations include,
The current depth screen is divided into target blocks of a first size set in advance,
Selecting a search block in a search area of a second predetermined size of a reference depth screen, which is a previous frame or a next frame of the current depth screen,
Calculating an expansion ratio based on a first average value of depth values of pixels in the object block and a second average value of depth values of pixels in the search block,
Expanding and contracting the search block according to the expansion ratio,
Calculating a motion estimation difference signal between the target block and the retracted search block,
And to determine a motion estimation difference signal between the target block and each of the plurality of stretched search blocks in the search area,
Calculating a motion estimation difference signal between the target block and the retracted search block includes removing a depth value variation amount,
The depth value variation amount removal may include:
Calculating a difference value between the depth values of the pixels in the target block and the first average value to generate a difference-
A second average value is calculated by subtracting the depth value of pixels within the expanded and expanded search block from the second average value to generate a difference-
And calculating a motion estimation difference signal between the difference-computed target block and the difference computed and expanded search block
Non-transitory computer readable medium.

Images