TWI423170B - A method for tracing motion of object in multi-frame - Google Patents

Description

Tracking method for object movement in multi-picture

The present invention relates to a method for processing multi-picture images, and more particularly to a method for tracking multi-picture object movement.

As technology continues to advance, digital products are becoming more common in people's lives. In order to make it easier to record life, mobile phones, digital cameras, digital cameras and so on often have the ability to take still photos or dynamic movies. For example, these electronic products are internally provided with a charge-coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) photosensitive element.

In a digital camera or a digital camera based on a CMOS sensor, the download time of the image is relatively short, and the frame rate is also relatively high.

In traditional times, the mobile computing method based on block matching or the computing algorithm without considering the previous picture motion detection result is relatively large and time consuming, in high resolution and high picture demand. The next application will not be enough.

In the case of a full search type of moving comparison, in order to calculate the moving relationship between objects between two pictures, it is necessary to find all the positions of the two pictures one by one, and then obtain the moving relationship of the objects according to the most matching result. When you need to know the movement relationship of the object in the next picture, you need to find it again according to this logic. Although the full search algorithm does not require additional judgment, when the image is large and the processing time is limited, it is too time-consuming and inefficient to complete the analysis of the moving relationship under multi-screen, thereby reducing the implementation. possibility.

The present invention provides a tracking method for object movement in a multi-picture, which can shorten the time of a motion decision.

The invention provides a tracking method for object movement in a multi-picture, comprising the following steps. First, a first position of an object in a first picture and a second position of an object in a second picture are obtained to calculate a motion vector according to the first position and the second position. Next, it is determined whether the motion vector is greater than a threshold parameter. When the motion vector is greater than the threshold parameter, a main direction is selected from a plurality of predetermined directions according to the motion vector. Then, from the second position of a third picture, the area near the main direction is searched to find a third position of the object in the third picture. When the motion vector is smaller than the threshold parameter, the vicinity of the second position of the third picture is searched along these predetermined directions to find the third position of the object in the third picture.

In one embodiment of the invention, the number of these predetermined directions is eight and is equally divided by 360 degrees.

In an embodiment of the invention, the step of searching for a region near the main direction to find the third position of the object in the third picture comprises the following steps. First, from these predetermined directions, the other two main directions adjacent to the main direction are found. Next, in the three main directions, the objects in the third picture are searched.

In an embodiment of the invention, the step of searching for a region near the main direction to find the third position of the object in the third picture further includes the following steps. First, a scan direction is selected from the three main directions based on the search results in the three main directions. Next, the vicinity of the scanning direction is searched to find the third position of the object on the third screen.

In an embodiment of the present invention, the tracking method of the object movement in the multi-screen further includes discarding the third picture when the object is not found in the third picture.

In an embodiment of the present invention, the tracking method of object movement in a multi-screen further includes the following steps. First, a moving tendency of the object is analyzed according to the position of the object in the first picture to the N-1th picture. Then, according to the movement trend, the object is judged to be at an Nth position in an Nth picture.

In one embodiment of the invention, the step of analyzing the trend of movement comprises analyzing the trend of movement using a curve fitting or a linear extrapolation.

Based on the above, the tracking method of the object movement in the multi-screen of the present invention can first perform coarse scanning on a plurality of predetermined directions to shorten the time of the movement decision and the amount of data calculation, and can meet the requirement of the high picture update rate. In addition, through the comparison of threshold values and other mechanisms, the search results still have a certain degree of accuracy.

The above described features and advantages of the present invention will be more apparent from the following description.

1 is a schematic flow chart of a method for tracking an object movement in a multi-screen according to an embodiment of the present invention, and FIG. 2 is a schematic diagram of a plurality of consecutive pictures applied to FIG. In FIG. 2, for example, a continuous first picture F1, a second picture F2, a third picture F3, and a fourth picture F4 are illustrated, and an object O can be moved along the track T.

Referring to FIG. 1 and FIG. 2, step S110 is first performed to obtain a first position (x, y) of the object O in the first picture F1 and a second position (x+U1, y of the object O in the second picture F2). +V1) to calculate a motion vector (U1, V1) from the first position (x, y) and the second position (x + U1, y + V1). Next, in step S120, it is determined whether the motion vector (U1, V1) is greater than a threshold parameter, and the situation in which the amount of movement of the object O is large and the small situation are distinguished. In this embodiment, the threshold parameter may be a single value TH (not shown in the figure), but is not limited thereto. That is, the sizes of U1 and TH and V1 and TH are compared, respectively.

3A is a schematic view showing the predetermined direction and the main direction of FIG. 1 in the screen. Referring to FIG. 2 and FIG. 3A, when the motion vector (U1, V1) is larger than the threshold parameter, the amount of movement of the representative object O is larger. It is not easy to reverse the direction in a short time. At this time, step S130 is performed to select a main direction M1 from a plurality of predetermined directions P based on the motion vector (U1, V1). Then, proceeding to step S140, searching for the vicinity of the main direction M1 from the second position (x+U1, y+V1) of the third screen F3 to find a third position of the object O on the third screen F3 (x+ U2, y+V2). In this embodiment, the number of these predetermined directions may be eight and is equally divided by 360 degrees.

That is to say, when looking for the relative movement relationship between the third picture F3 and the first picture F1, the search is started directly from the second position (x+U1, y+V1) instead of the first position (x, y). Find it. Similarly, the motion vector (U2, V2) of the third picture F3 and the first picture F1 is recorded, and when the motion relationship between the object O in the fourth picture F4 and the object O in the first picture F1 is found, it is also directly based on The third position (x+U2, y+V2) recorded in the third screen F3 is made to start the search, and can be roughly swept according to the most likely main direction M1, and then swept.

FIG. 4 is a schematic diagram of the search range of the set moment type in FIG. 3A. Please refer to FIG. 2 and FIG. 4. In contrast, when the motion vector (U1, V1) is smaller than the threshold parameter, it means that the moving amount of the object O is close to the origin, and the next motion will make the direction reverse. At this time, step S150 is performed to search for the vicinity S4 of the second position (x+U1, y+V1) of the third screen F3 along the predetermined directions P to find the third position of the object on the third screen F3 ( x+U2, y+V2). For example, the nearby area S4 can be defined by a rectangular area, but is not limited thereto. That is to say, a rectangular search range is set centered on the current second position (x+U1, y+V1), and analyzed in eight directions to save time and complexity.

FIG. 5 is a schematic flowchart of a method for tracking an object movement in a multi-screen according to another embodiment of the present invention, and FIG. 3B to FIG. 3D are schematic diagrams of three adjacent main directions. The flow of FIG. 5 will be described below with reference to FIG. 2, FIG. 3B to FIG. 3D and FIG. 4, but is not limited thereto. Referring to FIG. 2 first, step S210 is first performed to obtain a first position (x, y) of the object O in the first picture F1 and a second position (x+U1, y+V1 of the object O in the second picture F2). And calculating a motion vector (U1, V1) according to the first position (x, y) and the second position (x + U1, y + V1). Next, proceeding to step S220, it is determined whether the motion vector (U1, V1) is greater than a threshold parameter.

When the motion vector (U1, V1) is greater than the threshold parameter, step S230 is performed to select a main direction M1 from the plurality of predetermined directions P according to the motion vector (U1, V1). Then, in step S240, from the second position (x+U1, y+V1) of the third screen F3, the region S1 near the main direction M1 is searched to find a third position of the object O on the third screen F3 (x) +U2, y+V2). Step S240 may include a plurality of sub-steps such as steps S242-248. Referring to FIG. 3B, step S242 is first performed. From these predetermined directions P, the other two main directions M2 and M3 adjacent to the main direction M1 are found. Next, in step S244, the object O in the third screen F3 is searched for in the three main directions M1, M2, and M3. That is to say, in order to further save computing resources, only the closest main direction is taken for searching. Assuming that the system efficiency is high and the computing resources are low, only the main direction within the closest 90 degrees is searched. In this embodiment, three adjacent main directions M1, M2, and M3 are searched.

Then, in step S246, a scan direction M1 is selected from the three main directions M1, M2, M3 according to the search results in the three main directions M1, M2, M3. Next, in step S248, the vicinity S1 of the scanning direction M1 is searched to find the third position (x+U2, y+V2) of the object O in the third picture F3 (as shown in FIG. 2). That is, the search results and errors of the three main directions M1, M2, and M3 are compared, and the most accurate and matching main direction M1 is taken out before the fine sweeping operation is performed to reduce the amount of data calculation. Although the present embodiment is illustrated by taking FIG. 3B as an example, depending on the motion vector, the vicinity S2 corresponding to the three main directions M1', M2', M3' in FIG. 3C or the vicinity S2 in FIG. 3D may be used. The search is performed in the vicinity S3 corresponding to the three main directions M1", M2", M3", and so on.

Referring to FIG. 4, when the motion vector (U1, V1) is smaller than the threshold parameter, step S250 is performed to search for the second position (x+U1, y+V1) of the third picture F3 along the predetermined directions P. The vicinity area S4 is to find the third position (x+U2, y+V2) of the object on the third screen F3.

In this embodiment, after step S240 or step S250 is performed, step S260 may also be performed. In step S260, when the object O is not found in the third screen F3, the third screen F3 may be discarded. That is to say, if there are some pictures between consecutive pictures and the result of the condition is not found by the aforementioned search method, the picture information is discarded.

It is worth mentioning that the tracking method of object movement in the multi-picture of the embodiment can be applied to image processing of other multi-pictures. Taking the application of 3D noise reduction as an example, the picture abandoned in step S260 is not subjected to a temporal filtering process. In addition, due to the high picture update rate, the time difference of each picture is low. Therefore, if an object moves quickly outside the main direction search range in a short time, it may be caused by the very fast movement of the object. In such a situation, the object is prone to motion blur, and the scene in which the motion blur has occurred is also discarded by the step S260 during the blending process.

Further, step S270 may be further performed to analyze a moving tendency of the object O according to the position of the object O in the first picture F1 to the N-1th picture (not shown). Then, step S280 is performed to determine an Nth position of the object in an Nth picture according to the movement trend. In this embodiment, a curve matching method or a linear extrapolation method may be used to analyze the movement trend, but not limited thereto. That is to say, this embodiment is not limited to searching only with the result of the previous stage, and the result of the N-1 level can also be used to derive the position of the object of the Nth picture.

In summary, the tracking method of object movement in the multi-picture of the present invention can first perform coarse scanning on a plurality of predetermined directions to shorten the time of the movement decision, and can meet the requirement of high picture update rate of the CMOS photosensitive element. In addition, in the process of rapid mobile analysis, the trend can be judged by different mechanisms, so that the results still have certain accuracy.

Although the present invention has been disclosed in the above embodiments, it is not intended to limit the invention, and any one of ordinary skill in the art can make some modifications and refinements without departing from the spirit and scope of the invention. The scope of the invention is defined by the scope of the appended claims.

F1. . . First picture

F2. . . Second screen

F3. . . Third screen

F4. . . Fourth screen

M1~M3, M1'~M3', M1"~M3". . . Main direction

O. . . object

S1~S4. . . Nearby area

S110~S150, S210~S280. . . step

P. . . Scheduled direction

T. . . Trajectory

(x,y). . . First position

(x+U1, y+V1). . . Second position

(x+U2, y+V2). . . Third position

FIG. 1 is a schematic flow chart of a method for tracking an object movement in a multi-screen according to an embodiment of the present invention.

2 is a schematic diagram of a plurality of consecutive pictures applied to FIG. 1.

3A is a schematic view showing a predetermined direction and a main direction of FIG. 1 in a screen.

3B to 3D are schematic views of three adjacent main directions.

FIG. 4 is a schematic diagram of the search range of the set moment type in FIG. 3A.

FIG. 5 is a schematic flow chart of a method for tracking object movement in a multi-screen according to another embodiment of the present invention.

S110~S150. . . step

Claims (7)

A tracking method for moving an object in a multi-picture, comprising: obtaining a first position of an object in a first picture and a second position of the object in a second picture, according to the first position and the second position Calculating a motion vector; determining whether the motion vector is greater than a threshold parameter; when the motion vector is greater than the threshold parameter, selecting a primary direction from the plurality of predetermined directions according to the motion vector; and selecting the second direction from a third screen Positioning, searching for an area near the main direction to find a third position of the object in the third picture; and when the moving vector is smaller than the threshold parameter, searching for the third picture along the predetermined directions A nearby area of the second location to find the object at the third location of the third frame. The tracking method for moving objects in a multi-screen as described in claim 1, wherein the number of the predetermined directions is eight and is equally divided by 360 degrees. The method for tracking the movement of an object in a multi-picture as described in claim 1, wherein the step of searching for the vicinity of the main direction to find the object in the third position of the third picture comprises: In the predetermined direction, the other two main directions adjacent to the main direction are found; and in the three main directions, the object in the third picture is searched. The method for tracking the movement of an object in a multi-screen as described in claim 3, wherein the step of searching for the vicinity of the main direction to find the object in the third position of the third screen further includes: The search results in the three main directions select a scanning direction from the three main directions; and search for a nearby area of the scanning direction to find the third position of the object in the third picture. The method for tracking the movement of an object in the multi-picture as described in claim 1 further includes: discarding the third picture when the object is not found in the third picture. The method for tracking the movement of an object in the multi-picture as described in claim 1 further includes: analyzing a moving tendency of the object according to the position of the object in the first picture to the N-1th picture; The movement trend, determining an Nth position of the object in an Nth picture, wherein N 3. The method for tracking the movement of an object in a multi-picture as described in claim 6, wherein the step of analyzing the movement trend comprises: analyzing the movement trend by using a curve matching method or a linear extrapolation method.