JP2008102659A - Motion detection method and motion detector - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a motion detection method and a motion detector, capable of processing quickly detection of a moving speed component in an image, with reduced data volume, using a relatively simple processing procedure. <P>SOLUTION: The motion detection method/motion detector detects the last feature area that is a feature area of the last input field image signal and a current feature area that is a feature area of current field image signal, prepares extension signals A, B with the extended last feature area and current feature area extended along the first direction, prepares extension signals C, D with the extended last feature area and current feature area extended along the second direction, and detects a moving direction and a moving amount of the current feature area, using the extension signal A, the extension signal B, the extension signal C and the extension signal D. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a motion detection method and a motion detection apparatus that detect an object included in continuous images and detect a moving speed, a moving direction, a position, and the like of the object.

  In recent years, various types of moving image processing have been performed on flat panel displays whose image quality has been remarkably advanced. When performing moving image processing, it is necessary to first detect the movement of an object included in the video. As a method of motion detection, optical flow (on the image generated by the relative movement between the camera observer and the moving object) A method using the velocity field of each point) is known (see Patent Document 1).

  As a conventional optical flow detection method for detecting a moving speed component from an image, a correlation method for determining a motion vector by determining a corresponding point between frames, a spatial and temporal gradient of brightness of each point, or the like. A method such as a gradient method using a relationship between the two is generally used.

First, the correlation method is a method using block-to-block matching that divides an image into small regions and finds a region similar to this from successive images. In the correlation method, in a region I (t + 1, x + u, y + v) corresponding to a region continuous with the region of interest I (x, y, t) at time t,
ΣΣ {I (x, y, t) −I (x + u, y + v, t + 1)} 2
ΣΣ | I (x, y, t) -I (x + u, y + v, t + 1) |
(U, v) that minimizes. This (u, v) is the amount of movement of the object between consecutive frames.

The gradient method then uses the relationship between the spatial and temporal gradients of brightness at each pixel. Assuming that the brightness at the coordinates (x, y) at a certain time t is f (x, y, t), the point (x, y) on the pattern after the time δt moved by δx, δy in the x and y directions. Assume that the brightness does not change. At this time, the following equation holds.
f (x, y, t) = f (x + δx, y + δy, t + δt)
Taylor expansion of the above equation and δt → 0,
δf / δx · dx / dt + δf / δy · dy / dt + δf / δt = 0
Get. Since the motion vector (dx / dt, dy / dt) cannot be determined only by the gradient condition of the motion vector, the motion vector is calculated by further adding a constraint condition to the gradient condition. As a constraint condition to be added, a condition in which a motion vector changes smoothly in space is generally used.
JP 2005-209155 A

  However, in the above-described correlation method, the processing procedure is relatively simple. However, since a region for calculating a correlation value is provided around the region of interest, and a correlation value of in-region luminance is calculated between frames, a template is used. There is a problem that block matching processing such as matching is required, the calculation amount is enormous, and the processing time becomes long. That is, in the technique using block matching, in order to calculate detailed position information by using a pixel group of sub-pixels or less, pixel group data that is a registered template is centered on the target position and its surroundings. Since a plurality of correlation values calculated while being shifted by a predetermined amount are interpolated with a high-order curve and the motion is estimated by calculating the position where the extreme value is given, the calculation amount becomes enormous.

  In addition, the gradient method has a problem that it is weak against changes in luminance of the image and the processing procedure is complicated.

  The present invention has been made to solve such a conventional problem. The object of the present invention is to move from within an image by using a relatively simple processing procedure with a smaller amount of data. It is an object of the present invention to provide a motion detection method and a motion detection apparatus that can process detection of velocity components at a higher speed.

  In order to solve this problem, the present invention detects a previous feature region that is a feature region of an input previous field video signal and a current feature region that is a feature region of a current field video signal, and the previous feature region and the current feature. The extension signals A and B are generated by extending the region in the first direction, and the extension signals C and D are generated by extending the previous feature region and the current feature region in the second direction. The motion direction and the motion amount of the current feature area are detected using the signal B, the decompressed signal C, and the decompressed signal D.

  Further, the present invention detects horizontal edge areas from the input previous field video signal and current field video signal, creates expanded signals A and B by extending the horizontal edge areas in the left direction of the screen, and The extension signals C and D are generated by extending the respective horizontal edge areas in the right direction of the screen, and the movement of the horizontal edge area is performed using the extension signal A, the extension signal B, the extension signal C, and the extension signal D. It is characterized by detecting the direction and the amount of motion.

  Further, the present invention creates a first binary image signal by binarizing the input previous field video signal and current field video signal with m different threshold values (m is an integer of 2 or more), respectively. The second binary image is obtained by changing the value of a pixel included in a region in which pixels having a value of 0 in each of the first binary image signals continue to n pixels or less (n is a natural number) in the horizontal direction to 1. A signal is created, a horizontal edge region is detected from each of the second binary image signals, stretched signals A and B are created by stretching the respective horizontal edge regions in the left direction of the screen, and the respective horizontal The extension signals C and D are generated by extending the edge region in the right direction of the screen, and the motion direction in the current field video signal is generated using the extension signal A, the extension signal B, the extension signal C, and the extension signal D. And wherein the detecting the movement speed.

  According to the present invention, the moving direction and speed of the feature area included in the video signal can be easily obtained without using block matching such as template matching. In addition, since it is sufficient to process a 1-bit signal indicating whether or not it is a feature region, the amount of calculation can be reduced.

  Hereinafter, a motion detection method and a motion detection device according to an embodiment of the present invention will be described with reference to the drawings.

  FIG. 1 is a diagram for explaining the concept of a motion detection method according to the present invention.

  In normal television broadcasting, it is generally known that the movement in the scanning line direction, that is, the horizontal direction is overwhelmingly larger than the movement in the vertical direction. Therefore, in this embodiment, in order to simplify the circuit, attention is paid only to the movement in the horizontal direction.

  As shown in FIG. 1A, for example, a case where a white window pattern is moving rightward will be described as an example.

  FIGS. 1B and 1C show the luminance cross section of the previous field video signal and the luminance cross section of the current field video signal, respectively. The video signal is not limited to the RGB primary color video signal, but may be a luminance signal or a color difference signal.

  First, in the detection method of the present invention, as shown in FIGS. 1 (d) and 1 (e), the previous feature area of the previous field video signal and the current field video signal and the horizontal edge area which is the current feature area are detected. To do.

  Then, based on the horizontal edge detected from the previous field video signal and the current field video signal, a predetermined number of pixels in the left direction (first direction) as shown in FIGS. 1 (f) and 1 (g). A signal that has been stretched by a predetermined length is created (decompressed signal A and expanded signal B). The number of pixels to be expanded is determined based on the amount of motion to be detected. For example, if the purpose is to detect a motion of 15 pixels or less in one field, it may be expanded by 15 pixels. That is, the number of pixels to be expanded may be set to the maximum number of pixels in which the horizontal edge that is the previous feature area and the current feature area moves between one field.

  Similarly, as shown in FIG. 1 (h) and FIG. 1 (i), a signal obtained by extending a predetermined number of pixels in the right direction (second direction) opposite to the left direction based on the horizontal edge. Create (referred to as expansion signal C and expansion signal A). Also in this case, the number of pixels to be expanded is determined based on the amount of motion to be detected. For example, if the purpose is to detect a motion of 15 pixels or less in one field, it may be expanded by 15 pixels. That is, the number of pixels to be expanded may be set to the maximum number of pixels in which the horizontal edge that is the previous feature area and the current feature area moves between one field.

  The extension signal A, the extension signal B, the extension signal C, and the extension signal D are used to detect the direction and amount of movement of the current feature region.

  This detection method will be described. First, comparing the expanded signal A and the expanded signal B, the signal increases from 0 to 1 in the shaded portion of FIG. 1 (g) from the previous field to the current field. Further, when the expanded signal C and the expanded signal D are compared, the signal decreases from 1 to 0 in the shaded portion of FIG. 1 (i) from the previous field to the current field.

  In this way, the signal obtained by extending the horizontal edge portion to the left increases from 0 to 1 from the previous field to the current field, and the signal obtained by extending the horizontal edge portion to the right decreases from 1 to 0 from the previous field to the current field. Is the portion where the horizontal edge is moving to the right (FIG. 1 (j)). Conversely, the signal with the horizontal edge stretched to the left decreases from the previous field to the current field, and the signal with the horizontal edge stretched to the right increases from the previous field to the current field, the horizontal edge moves to the left. It can be seen that it is a moving part. Further, by obtaining the number of pixels that is the horizontal width of the detected moving part, the number of pixels in which the horizontal edge part has moved between one field can be obtained.

  As described above, according to the detection method of the present invention, it is possible to detect the moving direction and the moving amount of the horizontal edge portion only by simple processing for the horizontal edge portion detected from the video signal.

  Note that, unlike a window pattern, a general natural image includes a signal having a high spatial frequency, and thus further ingenuity is required to apply the above method.

  That is, in the previous feature region of the previous field video signal and the current field video signal or the horizontal edge portion of the current feature region, the interval between the adjacent previous feature region or the horizontal edge portion of the current feature region is the extension signal A, B, It is necessary to adjust the previous field video signal and the current field video signal in advance so that they are larger than the number of pixels to be expanded when creating C and D and the expanded signals of the adjacent horizontal edge portions do not overlap. It is.

  FIG. 2 shows a case where motion detection is performed using the above-mentioned concept when a video signal having a high spatial frequency is moving horizontally. 2A shows the previous field video signal, FIG. 2B shows the current field video signal, and the thick vertical lines indicate the horizontal edge portions. 2C shows a signal obtained by extending the horizontal edge portion of the previous field to the left (corresponding to the extension signal A), and FIG. 2D shows a signal obtained by extending the horizontal edge portion of the current field to the left (to the extension signal B). 2 (e) is a signal obtained by extending the horizontal edge portion of the previous field to the right (corresponding to the expansion signal C), and FIG. 2 (f) is a signal obtained by extending the horizontal edge portion of the current field to the right ( Corresponding to the expansion signal D). It can be seen that the stretched signals overlap each other because the edge portions are close to each other. A portion indicated by shading in FIG. 2 (d) is a portion where the signal obtained by extending the horizontal edge portion to the left from the previous field to the current field is increased from 0 to 1, and in FIG. 2 (f). The shaded portion in FIG. 6 is a portion where the signal obtained by extending the horizontal edge portion to the right from the previous field to the current field decreases from 1 to 0. Since there is no portion where these two shaded areas overlap, as a result, no motion area is detected as shown in FIG. This problem is caused by the high spatial frequency of the video signal. However, if a low-pass filter is applied to a natural image in order to avoid this, the edge portion cannot be detected.

  Therefore, in the present invention, as shown in FIG. 3, the current field video signal and the previous field video signal are binarized in advance with a predetermined threshold, and based on the signal, a horizontal edge region is extracted, An extension signal obtained by extending the horizontal edge area is generated to detect the movement direction and the movement amount of the horizontal edge area.

  That is, when processing a video signal including a component having a high spatial frequency as shown in FIG. 3A, first, binarization is performed using a predetermined threshold (FIG. 3B). Next, in the binarized video signal, hole filling is performed by replacing the value of the region having a narrow horizontal width among the regions having a value of 0 sandwiched between regions having a value of 1 by 1 ( FIG. 3 (c)).

  Here, the region with a narrow horizontal width means the number of pixels that extend the edge region in the horizontal direction as shown in FIG. 1F, that is, the horizontal edge that is the previous feature region and the current feature region moves between one field. If the maximum number of pixels to be performed is n (n is a natural number), it indicates an area having a horizontal width of n pixels or less, and the current field video signal and the previous field video signal are binarized with a predetermined threshold value in advance. The value of a pixel included in a region in which pixels having a value of 0 continue to n pixels or less in the horizontal direction is changed to 1.

  If an edge region is detected for the video signal processed in this way, the interval between adjacent edge regions is widened, so that the stretched portions do not overlap each other when creating the stretched signal. Further, since the video signal is 1 bit, the amount of calculation data can be reduced, so that the movement direction and the movement amount of the horizontal edge portion can be easily obtained.

  In the present invention, when performing binarization processing as shown in FIG. 3, for example, as shown in FIG. 4A, a video signal is binarized by three different thresholds (threshold A> threshold B> threshold C). In this case, different binary video signals can be obtained as shown in FIGS. 4B, 4C, and 4D. The above motion detection may be performed independently using these binarized video signals, and the result may be comprehensively determined to determine the motion direction and the motion amount. Here, comprehensive judgment may be, for example, that the motion direction is majority, or that the motion determination is impossible if all of them do not match, and that the motion amount uses an average value. Furthermore, although it has been described here that there are three different threshold values, the present invention is not limited to this, and it is also possible to carry out using m (m is an integer of 2 or more) threshold values.

  That is, in the present invention, the input previous field video signal and current field video signal are binarized with different m threshold values (m is an integer of 2 or more) to create a first binary image signal, In each of the first binary image signals, the second binary value is obtained by changing the value of a pixel included in a region in which pixels having a value of 0 continue in the horizontal direction to n pixels or less (n is a natural number) to 1 An image signal is generated, a horizontal edge region is detected from each of the second binary image signals, and expanded signals A and B are generated by extending the respective horizontal edge regions in the left direction of the screen. Expanded signals C and D are generated by expanding the horizontal edge region in the right direction of the screen, and the current field video signal is generated using the expanded signal A, the expanded signal B, the expanded signal C, and the expanded signal D. And it detects the movement direction and movement speed of the.

  FIG. 5 is a functional block diagram of the motion detection device according to the embodiment of the present invention. The motion detection device 1 includes a plurality of motion detection units 2 and a motion calculation unit 3 that calculates a motion direction and a motion amount from signals detected by the plurality of motion detection units. The motion detection unit 2 binarizes the video signal of the current field and the previous field with a predetermined threshold value to create a binary image signal, and each of the created binary image signals has a value. A hole filling unit 5 for creating a binary image in which the value of a pixel included in a region in which pixels of 0 are continuous in the horizontal direction by n pixels or less (n is a natural number) is changed to 1, and a binary value of the current field after the hole filling A current field horizontal edge detector 6 for detecting a horizontal edge of the digitized video signal, a previous field horizontal edge detector 7 for detecting a horizontal edge of the binarized video signal of the previous field after hole filling, and the detected current field A decompression signal B creating unit 8 that stretches the horizontal edge to the left by a predetermined number of pixels, and a decompression signal D creation unit 9 that stretches the detected horizontal edge of the current field to the right by a predetermined number of pixels. A decompressed signal A creating unit 10 that stretches the detected horizontal edge of the previous field to the left by a predetermined number of pixels, and a decompressed signal C creating unit that stretches the detected horizontal edge of the previous field to the right by a predetermined number of pixels. 11.

  Since the binarization unit 4 and the hole filling unit 5 have a function of adjusting in advance so that the interval between the edge regions detected by the horizontal edge detection unit does not become too narrow, they may be integrated as the input video signal adjustment unit 12.

  With the above configuration, the motion detection method described above can be realized. In this embodiment, the motion detection is performed by paying attention to the edge portion of the video signal. However, the present invention is not limited thereto, and the motion detection is performed by detecting a characteristic portion of the video signal. May be performed. The characteristic part may be, for example, a luminance peak part of a video signal or a specific gradation part. In the case of an image after binarization and hole filling, the center of the region may be used.

  In this embodiment, the edge portion of the video signal is expanded to the left and right. However, the present invention is not limited to this, and two directions obtained by extending the characteristic portion of the video signal in the opposite directions are used to determine the direction of the video signal. It is possible to determine the amount of movement.

  As described above, the present invention can detect the motion amount and the motion direction of a moving image portion with a simple circuit configuration in an image processing apparatus that detects image motion and performs image processing. The display quality can be improved at a low cost.

The figure for demonstrating the concept of the motion detection method in embodiment of this invention Diagram for explaining problems that occur with images with high spatial frequency The figure for demonstrating the method for applying this invention with respect to the image with a high spatial frequency. The figure for demonstrating binarization with a some threshold value Functional block diagram of a motion detection apparatus according to an embodiment of the present invention

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Motion detection apparatus 2 Motion detection part 3 Motion calculation part 4 Binarization part 5 Hole filling part 6 Current field horizontal edge detection part 7 Previous field horizontal edge detection part 8 Decompression signal B production part 9 Decompression signal D production part 10 Decompression signal A Creation unit 11 Expanded signal C creation unit 12 Input video signal adjustment unit

Claims (15)

An expanded signal obtained by detecting a previous feature area that is a feature area of the input previous field video signal and a current feature area that is a feature area of the current field video signal, and extending the previous feature area and the current feature area in a first direction. A and B are generated, and extension signals C and D are generated by extending the previous feature area and the current feature area in the second direction, and the extension signal A, the extension signal B, the extension signal C, and the extension signal are generated. A motion detection method for detecting a motion direction and a motion amount of the current feature region using D. The motion detection method according to claim 1, wherein the first direction and the second direction are opposite to each other. A horizontal edge area is detected from the input previous field video signal and current field video signal, and the extension signals A and B are generated by extending the horizontal edge areas in the left direction of the screen. The expansion signals C and D expanded in the right direction of the screen are generated, and the movement direction and the movement amount of the horizontal edge region are detected using the expansion signal A, the expansion signal B, the expansion signal C, and the expansion signal D. A motion detection method characterized by: The motion detection method according to claim 1 or 3, wherein the current field video signal and the previous field video signal are binarized in advance with a predetermined threshold value. The current field video signal and the previous field video signal are binarized with a predetermined threshold value in advance, and then the values of pixels included in a region in which pixels having a value of 0 continue to be n pixels or less (n is a natural number) in the horizontal direction. The motion detection method according to claim 1, wherein a value obtained by changing to 1 is used. The input previous field video signal and current field video signal are binarized with different m threshold values (m is an integer of 2 or more) to create a first binary image signal, and the first binary image A pixel value of 0 in each of the signals is generated by changing the value of a pixel included in a region where n pixels or less (n is a natural number) in the horizontal direction to 1 to create a second binary image signal; A horizontal edge area is detected from each of the second binary image signals, and expanded signals A and B are generated by extending the horizontal edge areas in the left direction of the screen, and the horizontal edge areas are set in the right direction of the screen. The decompressed signals C and D are generated by using the respective decompressed signal A, the decompressed signal B, the decompressed signal C, and the decompressed signal D, and the motion direction and motion speed in the current field video signal are determined. Motion detecting method which is characterized in that output. The number of pixels to be expanded when generating the expansion signal A, the expansion signal B, the expansion signal C, or the expansion signal D is the maximum number of pixels that the horizontal edge region of the current field video signal moves between one field. The motion detection method according to claim 3 or 6, wherein the motion detection method is characterized in that: The motion detection according to claim 5 or 6, wherein the natural number n is the number of pixels to be expanded when the expansion signal A, the expansion signal B, the expansion signal C, or the expansion signal D is created. Method. The motion detection method according to claim 5 or 6, wherein the natural number n is a maximum number of pixels in which a horizontal edge region of the current field video signal moves between one field. A detection unit that detects a previous feature area that is a feature area of the input previous field video signal and a current feature area that is a feature area of the current field video signal, and extends the previous feature area and the current feature area in a first direction. The decompressed signal A and B, and the decompressed signal creating unit for creating the decompressed signals C and D obtained by stretching the previous feature area and the current feature area in the second direction; A motion detection apparatus comprising: a motion calculation unit that detects a motion direction and a motion amount of the current feature region using the expansion signal C and the expansion signal D. Detected by the previous field horizontal edge detection unit and current field horizontal edge detection unit for detecting a horizontal edge region from the input previous field video signal and current field video signal, and by the previous field horizontal edge detection unit and current field horizontal edge detection unit A decompressed signal A creating unit and a decompressed signal B creating unit for extending the horizontal edge region in the left direction of the screen, and a horizontal edge region detected by the previous field horizontal edge detecting unit and the current field horizontal edge detecting unit in the right direction of the screen The extension direction C and the extension signal D are generated by using the extension signal C generator and the extension signal D generator, the extension signal A, the extension signal B, the extension signal C, and the extension signal D. A motion detection apparatus comprising: a motion calculation unit for calculating. A binarization unit that binarizes the input video signals of the current field and the previous field to create a binary image signal of the current field and a binary image signal of the previous field, and each of the created binary image signals Created from the current field video signal and a hole filling unit that creates a binary image in which the value of a pixel included in a region where pixels having a value of 0 in the horizontal direction are continuous with n pixels or less (n is a natural number) in the horizontal direction is 1 A previous field video signal detecting a horizontal edge region from the binary image, a previous field horizontal edge detecting unit detecting a horizontal edge region from the current field video signal, a current field horizontal edge detecting unit, and the previous field horizontal edge detecting And a horizontal edge area detected by the current field horizontal edge detection unit A decompression signal B creation unit; a decompression signal C creation unit and a decompression signal D creation unit that stretches the horizontal edge areas detected by the previous field horizontal edge detection unit and the current field horizontal edge detection unit in the right direction of the screen; and A motion detection apparatus comprising: a motion calculation unit that calculates a motion direction and a motion amount of the horizontal edge region using the signal A, the decompression signal B, the decompression signal C, and the decompression signal D. The number of pixels to be expanded when generating the expansion signal A, the expansion signal B, the expansion signal C, or the expansion signal D is the maximum number of pixels that the horizontal edge region of the current field video signal moves between one field. The motion detection device according to claim 11 or 12, characterized in that: The motion detection according to claim 11 or 12, wherein the natural number n is the number of pixels to be expanded when the expansion signal A, the expansion signal B, the expansion signal C, or the expansion signal D is created. apparatus. 13. The motion detection apparatus according to claim 11, wherein the natural number n is a maximum number of pixels in which a horizontal edge region of the current field video signal moves between one field.

Images