JP4630174B2 - Motion vector detection device - Google Patents

Description

  The present invention relates to a motion vector detection device that detects a motion vector in a moving image composed of, for example, Bayer array color pixels.

  Conventionally, as a color imaging device for obtaining a color moving image, a spectral prism that separates incident light into three primary colors of red (R), green (G), and blue (B), and a subsequent stage of the spectral prism are arranged. A square lattice type three-plate type color image pickup element constituted by three photoelectric conversion elements is known.

  A square lattice type three-plate color pixel element is driven by, for example, a sequential scanning method, and R, G, and B colors of a captured color moving image are sampled at the same spatial position to generate a frame signal as shown in FIG. Output. Hereinafter, the pixel array shown in FIG. 7 is referred to as a square lattice type color pixel array.

  As a method for detecting a motion vector in a moving image with a square lattice color pixel array, there are generally the following methods. The first method is a method of detecting a motion vector using the color signals R, G, and B, and the second method is to perform a linear conversion on the color signals R, G, and B to obtain a luminance signal Y and a color signal Pb, In this method, Pr is generated to detect a motion vector. Since the luminance signal Y and the color signals Pb and Pr used in the second method are obtained by linearly converting the color signals R, G, and B used in the first method, the sampling structure of the second method is basic. This is the same as the first method. Therefore, motion vector detection by the first method will be described below.

  When detecting a motion vector with the color signals R, G, and B, the horizontal and vertical sampling frequencies and sampling positions of the color signals R, G, and B are the same. , B are the same for each color. In general, motion vectors are detected for R, G, and B color images by the block matching method in images between frame numbers n and n + 1. Specifically, a matching value M is calculated while shifting a block configured with a predetermined number of pixels and a predetermined number of lines in the x and y directions, and the values of x and y with the largest matching value M are obtained. A vector value in the x and y directions of the motion vector is obtained.

  As described above, a motion vector in a moving image in a square lattice type color pixel array can be detected. However, a conventional square lattice type three-plate color pixel element includes a spectral prism and three photoelectric conversion elements. Since it is configured, there is a problem that it is large and heavy, and in order to realize a small and lightweight color imaging device, a single plate type that does not require a spectroscopic prism and includes a single light receiving portion is desired. ing.

  As a technique for realizing a reduction in size and weight of a color image sensor, a color filter of R, G, and B colors as color pixels is arranged in a Bayer type on a light receiving surface of a photoelectric conversion element (hereinafter referred to as a “Bayer type color pixel array”). This color image sensor can be made small and light because a color image can be obtained with a single plate (see, for example, Non-Patent Document 1).

The Bayer type color pixel array is different from the square lattice type color pixel array described above, and as shown in FIG. 8, the color signal G has a checkered pattern sampling structure, and the color signals R and B have a square lattice type sampling structure. ing. Accordingly, if the maximum horizontal spatial frequency and the maximum vertical spatial frequency of the color signal G are μ _max and ν _max , respectively, the maximum horizontal spatial frequency and the maximum vertical spatial frequency of the color signals R and B are respectively μ _max · 1/2 And ν _max · 1/2. Note that the sampling structure of the color signal G is also called a five-eye sampling structure because it is a five-eye arrangement structure of dice.

Yuji Kiuchi “Basics and Applications of Image Sensors”, Nikkan Kogyo Shimbun, 1991, P145

  However, detection of motion vectors in a moving image with a Bayer color pixel array has the following problems.

  The first problem is that the motion vector detection accuracy may be lower than that of a moving image in a square lattice type color pixel array.

  In the moving image of the Bayer type color pixel array, since the color signals R and B have lower horizontal and vertical sampling frequencies than the color signal G as described above, the motion by the block matching method between the frame numbers n and n + 1. In the vector detection, the motion vector detection result based on the color signals R and B may have lower detection accuracy than the motion vector detection result based on the color signal G. Therefore, the detection accuracy of motion vectors in a moving image with a Bayer color pixel array may be lower than that with a moving image in a square lattice color pixel array.

  The second problem is that it may be affected by the aliasing distortion associated with the Bayer sampling structure. This problem will be described with reference to FIGS. 9 and 10 showing a three-dimensional frequency spectrum structure. In FIG. 9 and FIG. 10, μ, ν, and f indicate horizontal spatial frequency, vertical spatial frequency, and time frequency, respectively, and in order to simplify the explanation, the frequency spectrum of only the first quadrant is displayed and the negative frequency spectrum is displayed. Is omitted.

  As described above, in the moving image of the Bayer type color pixel array, the color signal G has both the highest horizontal spatial frequency and the highest vertical spatial frequency twice as compared with the color signals R and B. Therefore, as shown in FIG. The spectrum shape is caused by the pattern-like sampling structure. On the other hand, the color signals R and B have a spectral shape with a square lattice sampling structure as shown in FIG.

  Next, let us consider a case in which, in a moving image having a Bayer color pixel array, an image moves between frames with u = 1 [pixel] and v = 1 [line]. The surface where the frequency spectrum exists in this case is shown in FIG. Here, u and v represent the amount of motion in the x-axis direction (horizontal direction) and the y-axis direction (vertical direction), respectively, in the moving image of the Bayer color pixel array. In the following description, a motion between frames of p pixels in the x-axis direction and q lines in the y-axis direction is represented as (u = p, v = q). In addition, a moving image having a Bayer type color pixel array described below is composed of 7680 pixels in the x-axis direction and 4320 lines in the y-axis direction, and the time frequency f is 60 (Hz).

In order to obtain the motion vector values x and y, θx on the μ-f plane and θy on the ν-f plane may be obtained as shown in FIG. Since the moving image is composed of 7680 pixels in the x-axis direction and 4320 lines in the y-axis direction as described above, μ _max is 4320/2 (cph), ν _{max in FIG.} Is 7680/2 (cpw). F _max is 60/2 (Hz). Note that (cph) and (cpw) represent (cycle per height) and (cycle per width), respectively.

  By the way, in FIG. 10, for example, an image having a spatial frequency corresponding to a plane where only the spectrum of the color signal G exists, that is, (4320/2) / 2 (cph) or more in the x-axis direction or ( If an image having a spatial frequency of 7680/2) / 2 (cpw) or more is captured by a camera, if there is a minute movement such as (u = 1, v = 1) between frames, it can be understood from FIG. Thus, an accurate motion vector can be detected from the color signal G. On the other hand, in the case of the color signals R and B, for example, an image having a spatial frequency at the position indicated by the black point 30 is not in a spectral space that can be originally taken by the color signals R and B. Are not detected, and the motion vector detection accuracy is low.

  The present invention has been made to solve the conventional problems, and an object thereof is to provide a motion vector detection device capable of detecting a motion vector with higher accuracy than the conventional one.

The motion vector detection device of the present invention includes a first motion vector detection means for detecting a first motion vector by any one of a plurality of predetermined color signals, and a luminance signal for generating a luminance signal from the plurality of color signals. Generating means; second motion vector detecting means for detecting a second motion vector based on the luminance signal when a vector value of the first motion vector is greater than or equal to a predetermined first threshold; and the first motion vector Correlation value calculation means for calculating a correlation value between the vector value of the second motion vector and the vector value of the second motion vector, comparison means for comparing the correlation value with a predetermined second threshold value, and the first motion A first color signal that is output by applying the vector value of the first motion vector to the vector value of the motion vector of each of the plurality of color signals when the vector value of the vector is less than the first threshold value And second color signal applying means for applying and outputting the vector value of the second motion vector to the vector value of the motion vector of each of the plurality of color signals when the correlation value is greater than or equal to the second threshold value; A specific process of detecting a motion vector individually for each of the plurality of color signals when the correlation value is less than the second threshold, and outputting a signal obtained by weighting a vector value of each detected motion vector in advance as a motion vector And means .

With this configuration, the motion vector detection device of the present invention outputs a signal based on the first threshold value and the second threshold value, and therefore detects a motion vector for each of a plurality of color signals even when the motion of the moving image is minute. The motion vector can be detected with higher accuracy than the conventional one.

  In the motion vector detection device of the present invention, the color signal for detecting the first motion vector is a color signal sampled in a checkered pattern.

  With this configuration, in the motion vector detection device of the present invention, since the first motion vector detection means detects the first motion vector from the color signal sampled in a checkered pattern, in the moving image of the Bayer type color pixel array A motion vector can be detected.

  Furthermore, the motion vector detection device of the present invention has a configuration in which the plurality of color signals include color signals of three primary colors of light.

  With this configuration, in the motion vector detection device of the present invention, the motion vector determination means has the vector value of the first motion vector detected by one of the three primary color signals of light and the color signal of the three primary colors of light. Since the motion vector for each of the plurality of color signals is determined based on the correlation value with the vector value of the second motion vector detected by the luminance signal generated from the video signal, even when the motion of the moving image is very small It is possible to detect a motion vector with higher accuracy than the conventional one that detects a motion vector for each color signal.

  Furthermore, the motion vector detection apparatus of the present invention has a configuration in which the plurality of color signals include complementary color signals of the three primary colors of light.

  With this configuration, in the motion vector detection device of the present invention, the motion vector determination means has the vector value of the first motion vector detected by any one of the complementary colors of the three primary colors of light and the complementary color signal of the three primary colors of light. Since the motion vector for each of the plurality of color signals is determined based on the correlation value with the vector value of the second motion vector detected by the luminance signal generated from the video signal, even when the motion of the moving image is very small It is possible to detect a motion vector with higher accuracy than the conventional one that detects a motion vector for each color signal.

  Furthermore, the motion vector detection device of the present invention has a configuration in which the color signal for detecting the first motion vector is a green signal.

  With this configuration, in the motion vector detection device of the present invention, the first motion vector detection means detects the first motion vector from the green signal sampled in a checkered pattern, so that in the moving image of the Bayer type color pixel array A motion vector can be detected.

  The present invention can provide a motion vector detection device having an effect that a motion vector can be detected with higher accuracy than the conventional one.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

  First, the configuration of the motion vector detection device according to the present embodiment will be described. The motion vector detection apparatus according to the present embodiment will be described with reference to an example in which a motion vector is detected in a moving image of a Bayer color pixel array (see FIG. 8) in which color signals G are arrayed in a checkered pattern. .

  As shown in FIG. 1, the motion vector detection device 10 according to the present embodiment stores a first frame memory 11 that stores a color signal G, and a color signal G that is one frame before the first frame memory 11. A second frame memory 12, a first motion vector detector 13 for detecting a motion vector of the color signal G, a switch 14 for switching output based on the vector value of the motion vector of the color signal G, and a motion vector of the color signal G And a first RGB application unit 15 that applies the vector values of the color vectors to the vector values of the motion vectors of the color signals R, G, and B.

  In addition, the motion vector detection device 10 includes a luminance signal generation unit 16 that generates a luminance signal Y from the color signals of R, G, and B, a third frame memory 17 that stores the luminance signal Y, and a third frame memory 17. A fourth frame memory 18 that stores the luminance signal Y one frame before, a second motion vector detection unit 19 that detects a motion vector of the luminance signal Y, a vector value of the motion vector of the color signal G, and the luminance signal Y A correlation calculation unit 20 for calculating the correlation with the vector value of the motion vector, a switch 21 for switching the output based on the calculation result of the correlation calculation unit 20, and the vector value of the motion vector of the luminance signal Y as the color signals R and G The second RGB application unit 22 to be applied to the vector value of the B motion vector, the specific processing unit 23 for executing the specific processing specified in advance, and the signal output by the device are selected. A switch 24, and a clock signal output unit 25 for outputting a clock signal.

  The first frame memory 11 inputs and stores the color signal G for one frame among the color signals constituting the moving image of the Bayer type color pixel array. The second frame memory 12 stores the color signal G shifted by one frame by the clock signal output from the clock signal output unit 25. Therefore, the second frame memory 12 stores the color signal G one frame before the frame stored in the first frame memory 11.

  The first motion vector detection unit 13 compares the color signal G stored in the first frame memory 11 with the color signal G stored in the second frame memory 12, and performs the motion of the color signal G by, for example, a block matching method. A vector is detected. The motion vector of the color signal G has a vector value of a motion vector including the direction and magnitude of the motion vector (hereinafter simply referred to as “motion vector value”). The first motion vector detection unit 13 acquires a motion vector value of the color signal G by detecting a motion vector of the color signal G, and sends a signal indicating the motion vector value of the color signal G to the switch 14 and the first RGB application unit. 15 is output. The first motion vector detection unit 13 constitutes first motion vector detection means of the present invention.

  The switch 14 switches the output based on the motion vector value of the color signal G acquired by the first motion vector detection unit 13. Specifically, the switch 14 operates to output a signal from the first motion vector detection unit 13 to the correlation calculation unit 20 when | u | ≧ 2 or | v | ≧ 2. On the other hand, if | u | <2 and | v | <2, that is, if the motion of the moving image is very small, the switch 14 sends a signal from the first motion vector detection unit 13 to a switch switching signal (hereinafter referred to as “SW switching signal”). The operation is performed to output to the switch 24. The operation of the switch 14 is controlled by the first motion vector detection unit 13.

  The first RGB application unit 15 applies the motion vector value of the color signal G acquired by the first motion vector detection unit 13 to the motion vector values of the color signals R, G, and B.

  The luminance signal generation unit 16 generates a luminance signal Y from color signals of R, G, and B colors that constitute a moving image having a Bayer color pixel array. The luminance signal generation unit 16 constitutes a luminance signal generation unit of the present invention.

  The third frame memory 17 is configured to input and store the luminance signal Y for one frame. The fourth frame memory 18 stores the luminance signal Y shifted by one frame by the clock signal output from the clock signal output unit 25. Therefore, the fourth frame memory 18 stores the luminance signal Y one frame before the frame stored in the third frame memory 17.

  The second motion vector detection unit 19 compares the luminance signal Y stored in the third frame memory 17 with the luminance signal Y stored in the fourth frame memory 18, and, for example, the motion of the luminance signal Y by a block matching method. A vector is detected. The motion vector of the luminance signal Y has a motion vector value including the direction and magnitude of the motion vector. The second motion vector detection unit 19 obtains a motion vector value of the luminance signal Y by detecting a motion vector of the luminance signal Y, and outputs a signal indicating the motion vector value of the luminance signal Y to the correlation calculation unit 20 and the second RGB. The data is output to the application unit 22. The second motion vector detection unit 19 constitutes second motion vector detection means of the present invention.

  The correlation calculation unit 20 receives the motion vector values of the color signal G and the luminance signal Y from the first motion vector detection unit 13 and the second motion vector detection unit 19, respectively. The correlation with the Y motion vector value is calculated. The correlation calculation unit 20 constitutes a correlation value calculation unit of the present invention.

  The switch 21 switches the output based on the correlation value calculated by the correlation calculation unit 20. Specifically, the switch 21 operates to output a signal indicating the correlation value to the switch 24 as an SW switching signal when the correlation value calculated by the correlation calculation unit 20 is equal to or greater than a predetermined threshold. On the other hand, the switch 21 operates to output a signal indicating the correlation value to the specific processing unit 23 when the correlation value calculated by the correlation calculation unit 20 is less than a predetermined threshold value. The operation of the switch 21 is controlled by the correlation calculation unit 20.

  The second RGB application unit 22 applies the motion vector value of the luminance signal Y detected by the second motion vector detection unit 19 to the motion vector values of the color signals R, G, and B, so that the color signals R, G, and B The motion vector value is determined. Note that the second RGB application unit 22 constitutes a motion vector determination unit of the present invention.

  When the correlation value calculated by the correlation calculation unit 20 is less than the threshold, the specific processing unit 23 executes a specific process specified in advance. Here, the specific process is, for example, a process in which the motion vector value detected by the luminance signal Y is determined to be incorrect and the detected motion vector value by the luminance signal Y is not adopted, or for each of R, G, and B colors. This refers to processing that acquires a motion vector value, performs predetermined weighting on the motion vector value, and adopts the obtained motion vector.

  The switch 24 selects an output signal output from the apparatus based on the SW switching signal output from the switch 14, the switch 21, and the specific processing unit 23. For example, when the switch 14 outputs the SW switching signal, the switch 24 selects the output signal of the first RGB application unit 15. Further, when the switch 21 outputs the SW switching signal, the switch 24 selects the output signal of the second RGB application unit 22. When the specific processing unit 23 outputs the SW switching signal, the switch 24 selects the output signal of the specific processing unit 23.

  The clock signal output unit 25 generates clock signals at predetermined time intervals and outputs them to the first frame memory 11, the second frame memory 12, the third frame memory 17, and the fourth frame memory 18, respectively. .

  Next, the operation of the motion vector detection device 10 of the present embodiment will be described using FIG. 1 and FIG.

  First, the color signal R, G, B is input by the luminance signal generation unit 16, and the color signal G is input by the first frame memory 11 (step S11). The color signal G input to the first frame memory 11 is shifted to the second frame memory 12 for each frame by the clock signal output unit 25.

Next, the luminance signal Y is generated by the luminance signal generator 16 (step S12). Specifically, as illustrated in FIG. 3, the luminance signal generation unit 16 generates a luminance signal Y from the color signals R, G, and B of the moving image having the Bayer color pixel array based on the following equation.

  Here, the coefficients r, g, and b are coefficients based on color vision characteristics. For example, values of r = 0.212, g = 0.701, and b = 0.087 are used.

  The luminance signal Y generated by the luminance signal generation unit 16 is output to the third frame memory 17. The luminance signal Y input to the third frame memory 17 is shifted to the fourth frame memory 18 for each frame by the clock signal output unit 25.

  Further, the first motion vector detection unit 13 detects the motion vector value of the color signal G (step S13). Here, since the moving image to be processed in the present embodiment is a Bayer type color pixel array in which the color signals G are arranged in a checkered pattern, as shown in FIG. Thus, the color signal G is sampled, and the motion vector value of the color signal G is detected.

  Specifically, the first motion vector detection unit 13 compares the color signal G stored in the first frame memory 11 with the color signal G stored in the second frame memory 12, and uses, for example, a block matching method. A motion vector value of the color signal G is detected.

  Next, the first motion vector detection unit 13 determines whether the detected motion vector value of the color signal G is | u | <2 and | v | <2 (step S14).

  If it is determined in step S14 that the motion vector value of the color signal G is | u | <2 and | v | <2, that is, if the motion image is very small, the first motion vector detection unit 13 A signal indicating the motion vector value of the color signal G is output to the switch 14 and the first RGB application unit 15, and the first RGB application unit 15 applies the motion vector value of the color signal G to each color of R, G, and B (steps). S15). The obtained motion vector value signals for R, G, and B colors are output signals via the switch 24.

  Specifically, the first motion vector detection unit 13 controls the switch 14 so that the switch 14 outputs the SW switching signal to the switch 24, and the switch 24 selects the output signal of the first RGB application unit 15.

  On the other hand, in step S14, when the motion vector value of the color signal G is not determined as | u | <2 and | v | <2, that is, | u | ≧ 2 or | v | ≧ 2. In this case, the motion vector value of the luminance signal Y is detected by the second motion vector detection unit 19 (step S16).

  Specifically, the second motion vector detection unit 19 compares the luminance signal Y stored in the third frame memory 17 with the luminance signal Y stored in the fourth frame memory 18, and uses, for example, a block matching method. A motion vector value of the luminance signal Y is detected. A signal indicating the detected motion vector value of the luminance signal Y is output to the correlation calculation unit 20 and the second RGB application unit 22.

  Here, the difference between the motion vector of the luminance signal Y and the motion vector of the color signal G will be described. As a precondition, it is assumed that the luminance signal Y includes R, G, and B color components to the same extent. If the motion vector value of the color signal G is | u | <2 and | v | <2, the color signal G has a higher sampling frequency in the horizontal and vertical directions than the color signals R and B. The motion vector detection accuracy is higher than that of the color signals R and B. On the other hand, since the luminance signal Y is generated by linearly converting the color signals of R, G, and B colors, the motion vector detection accuracy of the luminance signal Y is | u | <2 and | v | <2. It becomes lower than the color signal G for such movement.

  Therefore, in this embodiment, if the motion vector value of the color signal G is | u | <2 and | v | <2 in step S14, the accuracy of the motion vector detection result by the luminance signal Y is low. Therefore, in step S15, the motion vector value of the color signal G is adopted as the motion vector value of the color signals R, G, and B. On the other hand, if the motion vector value of the color signal G is | u | ≧ 2 or | v | ≧ 2 in step S14, processing is performed based on the motion vector value based on the luminance signal Y.

  Subsequently, the correlation calculation unit 20 calculates a correlation value between the motion vector value of the luminance signal Y from the second motion vector detection unit 19 and the motion vector value of the color signal G from the first motion vector detection unit 13. (Step S17).

  Specifically, the correlation calculation unit 20 calculates a correlation value between the motion vector value of the luminance signal Y and the motion vector value of the color signal G, for example. Note that the processing in step S17 is provided because it is presumed that the motion vector value detected by the luminance signal Y and the motion vector value detected by the color signal G do not always match.

  As shown in FIG. 5, the calculation of the correlation value by the correlation calculation unit 20 takes into account that the sampling position of the luminance signal Y is shifted from the color signal G by half pixel (half phase) in both the horizontal and vertical directions And executed. That is, for example, when a motion vector value is detected using the block matching method, the block position and the search range position are shifted by half a pixel in the luminance signal Y and the color signal G, and an area detected by the luminance signal Y; A difference occurs in the area detected by the color signal G.

  Therefore, in the process of step S17, when the correlation between the motion vector value of the luminance signal Y and the motion vector value of the color signal G is calculated, the sampling position of the luminance signal Y and the sampling position of the color signal G are half. In consideration of the pixel shift, for example, the correction is performed by correcting the sampling position.

  Subsequently, the correlation calculation unit 20 detects the result by comparing with the predetermined threshold Th using the correlation value of the motion vector values of the luminance signal Y and the color signal G calculated in step S17 (step S18). The likelihood of the motion vector values of the luminance signal Y and the color signal G is determined. Here, the threshold Th is set in order to determine the correlation between the direction and the magnitude of the motion vector.

  If the correlation value between the motion vector values of the luminance signal Y and the color signal G is greater than or equal to the threshold Th in step S18, the motion vector of the luminance signal Y and the motion vector of the color signal G are shown in FIG. Indicates a match. In this case, since the luminance signal Y is generated using the color signals of the R, G, and B colors, the motion vectors of the color signals R and B are likely to have the same motion vector values as the color signal G. The second RGB application unit 22 applies the motion vector value detected from the luminance signal Y to the motion vector values of the color signals R, G, and B (step S19). The obtained motion vector value signals for R, G, and B colors are output signals via the switch 24.

  On the other hand, when the correlation value between the motion vector values of the luminance signal Y and the color signal G is less than the threshold value Th in step S18, the motion vector of the luminance signal Y and the color signal G are converted as shown in FIG. It shows that it does not match the motion vector. In this case, since the luminance signal Y is generated using color signals of R, G, and B colors, the following two causes can be assumed.

  As a first cause, there is a case where the motion vector detection results of the luminance signal Y and the color signal G are erroneous due to some possibility. For example, there is a motion vector detection error in block matching processing in the block matching method.

  As a second cause, there are cases where the ratios of the R, G, and B color components of the target moving image are different from each other. For example, this is a case where a single blue image moves.

Therefore, if the correlation value of the motion vector value of the luminance signal Y and the color signal G is less than the threshold value Th, perform specific processing shown in below (step S20). A signal obtained by the specific processing becomes an output signal via the switch 24.

As specific processing, a motion vector is individually detected for each of R, G, and B colors, weighting as shown in the following equation is performed, and processing that is adopted as the motion vector Mv is performed to output a signal indicating the motion vector Mv.

  In Expression (2), Rf [] represents a reference frame, Cr [] represents a base frame, and R, G, and B in [] represent red, green, and blue component images in the frame, respectively. M and n are variables indicating the block size (m pixels in the x direction × n pixels in the y direction) in the block matching method, and i and j are variables indicating the block movement amounts (the x direction i and the y direction j). Note that the coefficients r, g, and b are coefficients based on the color vision characteristics shown in Expression (1), and for example, values of r = 0.212, g = 0.701, and b = 0.087 are used.

  As described above, according to the motion vector detection device 10 of the present embodiment, the correlation calculation unit 20 calculates the correlation between the motion vector value of the color signal G and the motion vector value of the luminance signal Y, and applies the second RGB. Since the unit 22 is configured to apply the motion vector for each of the plurality of color signals based on the correlation value between the vector value of the color signal G and the vector value of the luminance signal Y, even when the motion of the moving image is minute, It is possible to detect a motion vector with higher accuracy than the conventional one that detects a motion vector for each of a plurality of color signals.

  In the above-described embodiment, the motion vector detection device 10 has been described with reference to an example in which a motion vector is detected in a moving image of a Bayer-type color pixel array composed of three primary colors of light. However, the present invention is not limited to the same, and the same effect can be obtained by a configuration including complementary color signals of the three primary colors of light, for example, a configuration for detecting a motion vector in a moving image of a Bayer color pixel array using a green signal, a yellow signal, and a cyan signal Is obtained.

  In the above-described embodiment, the motion vector detection device 10 has been described with reference to an example in which the motion vector is detected based on the color signal G and the luminance signal Y of the Bayer color pixel array. For example, when the color signal R is arranged in a checkered pattern and the color signals G and B are arranged in a square lattice pattern, the motion vector is calculated based on the color signal R and the luminance signal Y. The same effect can be obtained as a detection configuration.

  Further, in the above-described embodiment, the moving image is described as an example in which the moving image is configured by the Bayer type color pixel array, but the present invention is not limited to this, and the color pixel array other than the Bayer type is described. It may be a moving image composed of

  As described above, the motion vector detection device according to the present invention has an effect of being able to detect a motion vector with higher accuracy than the conventional one, and in a moving image composed of color pixels of a Bayer array. It is useful as a motion vector detection device for detecting a motion vector.

Block diagram of motion vector detection apparatus according to the present embodiment Flowchart of each step of motion vector detection device according to the present embodiment The figure which shows the luminance signal Y which the luminance signal generation part of the motion vector detection apparatus which concerns on this Embodiment produces | generates from the color signals R, G, and B of the moving image of a Bayer type color pixel arrangement | sequence. The figure which shows the position which the 1st motion vector detection part 13 of the motion vector detection apparatus based on this Embodiment samples the color signal G of the moving image of a Bayer type color pixel arrangement | sequence. In the motion vector detection device according to the present embodiment, the luminance signal Y is a diagram showing that the sampling position of the luminance signal Y is shifted by half a pixel in both the horizontal and vertical directions with respect to the color signal G. (A) In the motion vector detection device according to the first embodiment of the present invention, a diagram showing that the detected motion signal values of the luminance signal Y and the color signal G match (b) The figure which shows that the motion vector value of the detected luminance signal Y and the color signal G does not correspond in the motion vector detection apparatus which concerns on 1 embodiment. The figure which shows that each R, G, B color of the imaged RGB color moving image is sampled in the same spatial position in the conventional square lattice type 3 plate type color pixel element In the conventional Bayer color pixel arrangement, the color signal G has a checkered sampling structure, and the color signals R and B have a square lattice sampling structure. The figure which shows the three-dimensional frequency spectrum structure in the conventional Bayer type color pixel arrangement | sequence. The figure which shows the surface where a frequency spectrum exists in the case of the motion of u = 1 [pixel] and v = 1 [line] between the frames in the conventional Bayer type color pixel arrangement.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Motion vector detection apparatus 11 1st frame memory 12 2nd frame memory 13 1st motion vector detection part (1st motion vector detection means)
14, 21, 24 Switch 15 First RGB application unit 16 Luminance signal generation unit (luminance signal generation means)
17 3rd frame memory 18 4th frame memory 19 2nd motion vector detection part (2nd motion vector detection means)
20 Correlation calculation unit (correlation value calculation means)
22 2nd RGB application part (motion vector determination means)
23 Specific processing section 25 Clock signal output section

Claims (5)

A first motion vector detecting means for detecting a first motion vector based on any of a plurality of predetermined color signals; a brightness signal generating means for generating a brightness signal from the plurality of color signals; and the first motion. Second motion vector detection means for detecting a second motion vector from the luminance signal when the vector value of the vector is equal to or greater than a predetermined first threshold, and the vector value of the first motion vector and the second motion Correlation value calculating means for calculating a correlation value with a vector value of the vector, comparison means for comparing the correlation value with a predetermined second threshold value, and a vector value of the first motion vector as the first threshold value. A first color signal applying means for applying the vector value of the first motion vector to the vector value of the motion vector of each of the plurality of color signals and outputting the vector value when the correlation value is less than the second value; Second color signal applying means for applying and outputting the vector value of the second motion vector to the vector values of the motion vectors of the plurality of color signals when the value is equal to or greater than the value, and when the correlation value is less than the second threshold value Specific processing means for individually detecting a motion vector from each of the plurality of color signals and outputting a signal obtained by weighting a vector value of each detected motion vector in advance as a motion vector. Motion vector detection device.   The motion vector detection apparatus according to claim 1, wherein the color signal for detecting the first motion vector is a color signal sampled in a checkered pattern.   The motion vector detection device according to claim 1, wherein the plurality of color signals include color signals of three primary colors of light.   4. The motion vector detection device according to claim 1, wherein the plurality of color signals include complementary color signals of the three primary colors of light.   5. The motion vector detection device according to claim 1, wherein the color signal for detecting the first motion vector is a green signal. 6.

Images