KR20130057585A - Apparatus and method for detecting scene change of stereo-scopic image - Google Patents

Abstract

PURPOSE: A stereoscopic image scene change detecting apparatus and a method thereof are provided to determine whether an image is separated by an independent scene change or is distinguished by a continual scene change using an image feature at a usual scene change point, thereby determining whether a scene change point continues or not in a stereoscopic view. CONSTITUTION: A scene change point detecting module(210) detects a scene change point from an input image. A stereoscopic scene continuity point calculating module(220) extracts feature points from a final frame of a detected scene change point and calculates a scene continuity point using a feature point which is tracked in a first frame of a scene changed image. A scene continuity determining module(230) determines whether a stereoscopic scene continues or not based on a result of comparing a calculated scene continuity point and a predetermined critical value. [Reference numerals] (210) Scene change point detecting module; (220) Stereoscopic scene continuity point calculating module; (222) Feature point extracting unit; (224) Feature point matching unit; (226) Stereoscopic scene continuity point calculating unit; (230) Scene continuity determining module

Description

Apparatus and Method for detecting scene change of stereo-scopic image}

The present invention relates to an apparatus and method for detecting a scene change of a stereoscopic image, and more particularly, to extract feature points from a last frame of a scene change point detected from an input image, and to extract the feature points from the first frame of the scene changed image. The present invention relates to an apparatus and method for detecting a scene of a stereoscopic image, wherein the stereoscopic scene continuity score is calculated using the tracked feature points, and the stereoscopic scene continuity score is compared with a preset threshold to determine the continuity of the stereoscopic scene.

In general, a scene change refers to a phenomenon in which a video is restarted in another scene after one scene is finished in a sequence of video scenes. A transition usually involves a fade-out and fade-in where any scene gradually fades away and then another scene, slowly overlapping over time in two different directions. overlap), simple scene transitions, etc.

Conventional scene change point detection basically separates the scene change point based on a numerical value such as brightness or histogram of an image. For most of these criteria, it is reasonable to select a numerically discontinuous part as a transition point as shown in the Histogram result of FIG. 1, but in the case of stereoscopic transformation, if there is visual continuity of the object, it should be separated into one continuous scene. There are many cases.

That is, unlike a general scene change point, when a specific object continues to move in a continuous scene, viewers must maintain stereoscopic characteristics of the object so that stereoscopic discomfort is small.

If this is divided into different scene change points and separated into different scenes, different stereoscopic transformations may be applied. Thus, despite the continuous scenes, the stereoscopic characteristics may be changed to increase visual discomfort.

Of course, if the three-dimensional transformation work by hand, there is no problem because the person visually considers the characteristics of the three-dimensional object without distinguishing the scene transition point, but when the three-dimensional transformation should be performed automatically through image processing, the three-dimensional transformation by the unit of the transition point This transition point is very important because it is efficient to use.

Korean Unexamined Patent Publication No. 2009-0037288 (April 15, 2009), Name of invention: Real-time scene change detection method for controlling video encoding data rate, video call quality improvement method and video call system using same

SUMMARY OF THE INVENTION The present invention has been made to solve the above-described problem, and an object of the present invention is to provide an apparatus and method for detecting a scene of a stereoscopic image capable of detecting an optimal scene transition point in view of stereoscopic transformation.

Another object of the present invention is to determine whether to separate the scenes into independent scene transitions or distinguish them as one continuous scene transition in a conventional scene change point using characteristics of the image, thereby determining the presence or absence of the continuity of the scene change point from a stereoscopic point of view. The present invention provides an apparatus and method for detecting a scene change of a stereoscopic image.

According to an aspect of the present invention, a scene change point detection module that detects a scene change point in an input image, extracts feature points from the last frame of the detected scene change point, and the extracted feature points are tracked in the first frame of the scene change image 3D scene continuity score calculation module for calculating a 3D scene continuity score using the extracted feature points, and a scene continuity for comparing the calculated 3D scene continuity score with a predetermined threshold value and determining the continuity of the 3D scene based on the comparison result. An apparatus for detecting a scene change of a stereoscopic image including a determination module is provided.

The scene change point detection module detects the scene change point using at least one method of correlation, statistical sequential analysis, and histogram.

The stereoscopic scene continuity score calculation module may include a feature point extracting unit extracting feature points from the last frame of the scene change point detected by the scene change point detection module, and matching the feature points extracted from the last frame with the first frame. A feature point matching unit for obtaining the number of feature points tracked in the first frame, a sum of absolute difference (SAD) in a predefined block based on the feature points of the last frame and the feature points tracked in the first frame, and the obtained SAD It includes a stereoscopic scene continuity score calculation unit for obtaining a stereoscopic scene continuity score.

The stereoscopic scene continuity score calculation module obtains the stereoscopic scene continuity score C (s) using the following equation.

[Mathematical Expression]

C (s) = (Tracked feature points / total feature points) * (Σ (1 / SAD (fn, bm))

Here, the total number of feature points is the number of feature points extracted from the last frame of the scene change point, the number of tracked feature points is the number of feature points tracked in the first frame of the scene changed image, fn is the number of the frame, bm is m of the frame Means the first block.

The SAD (fn, bm) is obtained by using the following equation.

[Mathematical Expression]

SAD (fn, bm) = Σ abs (Frame (fn, bm) pixel (i)-Frame (fn + 1, bm) pixel (i))

Here, fn is the number of the frame, bm is the m-th block of the frame, i is the order of each pixel of the block.

The scene continuity determining module determines that the three-dimensional scene continuity score is equal to or greater than the threshold, and determines that the scene is continuous.

According to another aspect of the present invention, a method for detecting a scene change of a stereoscopic image by the scene change detection apparatus, comprising: (a) detecting a scene change point in an input image, (b) a last frame of the detected scene change point Extracting feature points and calculating a stereoscopic scene continuity score using the feature points tracked in the first frame of the scene from which the extracted feature points are transitioned; (c) comparing the calculated stereoscopic scene continuity score with a preset threshold; There is provided a scene change detection method of a stereoscopic image comprising the step of determining the continuity of the stereoscopic scene based on the comparison result.

In the step (b), extracting feature points in the last frame of the detected scene change point, matching the feature points extracted in the last frame with the first frame, and extracting the number of feature points tracked in the first frame. Obtaining a sum of absolute difference (SAD) in a predefined block based on the feature points of the last frame and the feature points tracked in the first frame, and using the obtained SAD and the number of tracked feature points. Calculating a scene continuity score.

The stereoscopic scene continuity score C (s) is obtained using the following equation.

[Mathematical Expression]

C (s) = (Tracked feature points / total feature points) * (Σ (1 / SAD (fn, bm))

Here, the total number of feature points is the number of feature points extracted from the last frame of the scene change point, the number of tracked feature points is the number of feature points tracked in the first frame of the scene changed image, fn is the number of the frame, bm is m of the frame Means the first block.

In the step (c), if the stereoscopic scene continuity score is greater than or equal to the threshold, it is determined that the scene is continuous, and if it is not greater than or equal to the threshold, it is determined that a scene change is made.

According to another aspect of the present invention, (a) detecting a scene change point in an input image, (b) extracting feature points in the last frame of the detected scene change point, and the extracted feature points of the scene Calculating a stereoscopic scene continuity score using the feature points tracked in the first frame; and (c) comparing the calculated stereoscopic scene continuity score with a preset threshold and determining the continuity of the stereoscopic scene based on the comparison result. A scene change detection method of a stereoscopic image includes a program and a recording medium readable by an electronic device is provided.

According to the present invention, an optimum scene change point can be detected in terms of stereoscopic transformation.

In addition, it is possible to determine whether to separate the scene into independent scene transitions or separate them into one continuous scene transition at a normal scene change point by using the feature of the image, thereby determining whether or not the scene change point is continuous from a stereoscopic point of view.

1 is a diagram representing a numerical value of a conventional scene change point.
2 is a block diagram schematically showing the configuration of an apparatus for detecting a scene change of a stereoscopic image according to the present invention;
Figure 3 is an exemplary view for explaining a method for obtaining the inter-frame SAD according to the present invention.
4 is a flowchart illustrating a method for checking whether a scene is continuous by a scene change detection apparatus of a stereoscopic image according to the present invention;
5 is a flowchart illustrating a method for obtaining a stereoscopic scene continuity score by the scene change detection apparatus according to the present invention.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the following description with reference to the accompanying drawings, the same or corresponding components will be denoted by the same reference numerals, and redundant description thereof will be omitted.

FIG. 2 is a block diagram schematically illustrating a configuration of an apparatus for detecting a scene change of a stereoscopic image according to the present invention, and FIG. 3 is an exemplary diagram for describing a method for obtaining an inter-frame SAD according to the present invention.

Referring to FIG. 2, the scene change detection apparatus 200 for a stereoscopic image includes a scene change point detection module 210, a stereoscopic scene continuity score calculation module 220, and a scene continuity determination module 230.

The scene change point detection module 210 detects a scene change point from an input image. That is, the scene change point detection module 210 obtains a scene change point of an input image by using a method such as brightness, correlation, statistical sequential analysis, histogram, and the like.

Since the scene change point detection module 210 detects the scene change point according to the related art, a detailed description thereof will be omitted.

The stereoscopic scene continuity score calculation module 220 extracts feature points from the last frame of the scene change point detected by the scene change point detection module 210, and tracks the feature points tracked in the first frame of the image after the extracted feature points are scene changed. Calculate the stereoscopic scene continuity score using. Here, the last frame of the scene change point means the last frame before the scene change, the first frame of the scene change image means the first frame after the scene change, and before and after the scene change is divided by the scene change point.

The stereoscopic scene continuity score calculation module 220 performing the above role includes a feature point extractor 222, a feature point matcher 224, and a stereoscopic scene continuity score calculator 226.

The feature point extractor 222 extracts feature points using Harris's corner extraction method, SIFT (Scale Invariant Feature Transform) algorithm, etc. in the last frame of the scene change point detected by the scene change point detection module 210. Here, the feature point may be defined in advance as referring to an edge or a corner.

The feature point extractor 222 extracts a feature point from the last frame of the scene change point in order to find an association of objects between images before and after the scene change point.

The feature point matching unit 224 matches the feature points extracted in the last frame of the scene change point with the first frame of the scene changed image to obtain the number of feature points tracked in the first frame.

For example, the feature points extracted from the last frame before the scene change point are feature point 1, feature point 2, feature point 3, feature point 4 and feature point 5, and as a result of matching the extracted feature points to the first frame after the scene change point feature point 1, feature point When 2 and 4 are tracked, the feature point matching unit 224 obtains "3" as the number of tracked features.

The stereoscopic scene continuity score calculator 226 obtains a sum of absolute difference (SAD) in a predefined block based on the feature points of the last frame and the feature points tracked in the first frame, and uses the obtained SAD. Obtain a stereoscopic scene continuity score. Here, the predefined block refers to a block having a predetermined predetermined size including neighboring pixels based on the feature point. The stereoscopic scene continuity score refers to a score indicating how much the feature points are maintained in the next scene-switched image based on the feature points extracted in the last frame.

The three-dimensional scene continuity score calculator 226 determines the continuity of the scene (s) and the scene (s + 1) in consideration of the number of feature points that can be actually tracked and the reliability of the feature points obtained by tracking the number of feature points to be performed. Find the score for. Here, Scene (s) refers to a scene corresponding to the scene change point, and Scene (s + 1) refers to a scene after the scene change point.

Accordingly, the stereoscopic scene continuity score calculator 226 calculates the stereoscopic scene continuity score C (s) using Equation 1.

Here, the total number of feature points refers to the number of feature points extracted from the last frame of the scene change point, and the number of tracked feature points refers to the number of feature points tracked in the first frame after the scene change point obtained by the feature point matching unit.

F (n) denotes reliability of each feature point, n denotes an index of traceable feature points, and s denotes an index of each scene.

The reliability (F (n)) of each feature point is easy to track when tracking at the first frame of the scene (s + 1), which is then transitioned according to the conventional method at the last frame of the scene (s), for any feature point n. Means. In other words, the reliability of each feature point represents how similarly some regions of the image formed based on the feature points exist in and close to each other in the next image to be tracked.

If the transition is made to a completely different image regardless of the previous image, it will not be possible to find areas with similar characteristics, which are excluded from the trace because they are essentially impossible to trace. However, if a certain degree of similarity is found, the measure of similarity found is obtained using the sum of absolute difference (SAD).

Accordingly, the stereoscopic scene continuity score calculator 226 calculates the stereoscopic scene continuity score using the SAD.

In this case, the stereoscopic scene continuity score calculator 226 calculates SAD by using Equation 2.

Here, fn is the number of the frame, bm is the m-th block of the frame, i is the order of each pixel of the block, abs is the absolute value.

The sum of absolute difference (SAD) is a sum of absolute differences of all pixels in a block, and as a result, represents a difference between blocks at the same position between frames. Therefore, the large SAD value means that the change of the image is large.

Accordingly, the stereoscopic scene continuity score calculator 226 divides the frame into a predetermined number of blocks based on the feature points, and obtains SAD between the current frame and the previous frame for each block.

Referring to FIG. 3 for how the stereoscopic scene continuity score calculator 226 obtains SAD, the current frame is Frame (fn), the previous frame is Frame (fn-1), and the subsequent frame is Frame (fn + 1). ), The stereoscopic scene continuity score calculator 226 determines the SAD between blocks bm at the same position in each frame, that is, SAD (fn-1, bm) between the current frame block bm and the previous frame block bm, and the current frame. And then SAD (fn, bm) between frame blocks bm are obtained, respectively.

Therefore, when the current frame is Frame (fn) and the subsequent frame is Frame (fn + 1), the three-dimensional scene continuity score calculator 226 determines the SAD between the blocks bm at the same position in each frame, that is, the current frame. SAD (fn, bm) between block bm and subsequent frame block bm is obtained, respectively.

As described above, the stereoscopic scene continuity score calculator 226 calculates SAD for each block by a predetermined spatial size for the feature points selected at the scene change point. If the obtained SAD value is greater than or equal to the threshold value, this means that tracking is not practical in reality, and thus, except for the tracking number of feature points, tracking may be possible if the SAD value is not greater than or equal to the threshold value. Therefore, the smaller the actual SAD value, the greater the similarity and the greater the reliability. In other words, the smaller the SAD, the greater the reliability of the inverse scaffold relationship.

Accordingly, the stereoscopic scene continuity score calculator 226 may obtain the reliability F (n) of each feature point by using the SAD.

Therefore, the stereoscopic scene continuity score calculator 226 may express Equation 1 as Equation 3 below.

Here, the total number of feature points is the number of feature points extracted from the last frame of the scene change point, the number of tracked feature points is the number of feature points tracked in the first frame of the scene changed image, fn is the number of the frame, bm is m of the frame Means the second block.

The stereoscopic image continuity score calculation module 220 configured as described above obtains a sum of absolute difference (SAD) in a predefined block based on the feature points of the last frame and the feature points tracked in the first frame that has been transitioned. The stereoscopic scene continuity score is calculated using SAD.

In addition, the stereoscopic image continuity score calculation module 220 may correct the stereoscopic scene continuity score by performing feature point matching on at least one frame to the left, right (ie, before or after the scene change point) of the scene change point. .

The scene continuity determining module 230 compares the stereoscopic scene continuity score obtained by the stereoscopic scene continuity score calculation module 220 with a preset threshold value, and determines the continuity of the stereoscopic scene based on the comparison result.

That is, the scene continuity determining module 230 determines that the scene is a continuous scene when the three-dimensional scene continuity score is greater than or equal to a preset threshold value, and determines that a scene change is performed when the scene continuity score is not more than a threshold value.

In other words, the scene continuity determining module 230 determines that two scene (s) and a scene (s + 1) separated in the conventional manner by the scene change point detection module 210 when the three-dimensional scene continuity score is greater than or equal to a threshold. ) Are processed by combining them into a scene in the stereoscopic image.

The scene change detection apparatus 200 of the stereoscopic image configured as described above automatically considers the detection of the scene change point for stereoscopic conversion, and basically selects the existing scene change point first and performs the additional detection as described above. This is done by examining whether or not you should.

In addition, the apparatus 200 for detecting a scene change of the stereoscopic image defines a criterion for determining whether to separate the scene into independent scene changes or to distinguish one continuous scene change from the normal scene change point.

4 is a flowchart illustrating a method of checking whether a scene is continuous by a scene change detection apparatus of a stereoscopic image according to the present invention.

Referring to FIG. 4, a scene change detection apparatus of a stereoscopic image detects a scene change point with respect to an input image (S302). That is, the scene change detection apparatus detects a scene change point of an input image by using a method such as correlation, statistical sequential analysis, histogram, and the like.

After performing the step S302, the scene change detection apparatus extracts the feature points from the last frame of the detected scene change point (S304), and the stereoscopic scene using the feature points tracked in the first frame of the scene where the extracted feature points are scene changed. The continuity score is calculated (S306). A detailed description of a method for obtaining a stereoscopic scene continuity score by the scene change detection apparatus will be given with reference to FIG. 5.

After performing S306, the scene change detection apparatus determines whether the obtained stereoscopic scene continuity score is equal to or greater than a preset threshold (S308).

If the stereoscopic scene continuity score is greater than or equal to a threshold as a result of the determination of S308, the scene change detection device determines that the frame is a continuous scene (S310).

If the stereoscopic scene continuity score is not greater than or equal to the threshold as a result of the determination in S308, the scene change detection apparatus determines that the scene change is performed (S312).

5 is a flowchart illustrating a method for obtaining a stereoscopic scene continuity score by the scene change detection apparatus according to the present invention.

Referring to FIG. 5, the scene change detection apparatus obtains the number of feature points tracked in the first frame after the scene change point in the feature points of the last frame where the scene change point is detected (S402). That is, the scene change detection apparatus obtains the number of feature points tracked in the first frame by matching the feature points extracted in the last frame with the first frame.

After performing S402, the scene change detection apparatus obtains a sum of absolute difference (SAD) in a predefined block based on the feature points of the last frame and the feature points tracked in the first frame (S404).

That is, the scene change detection apparatus obtains SAD by using Equation 2.

After performing S404, the scene change detection apparatus calculates a stereoscopic scene continuity score using the obtained SAD and the number of tracked feature points (S406).

That is, the scene change detection apparatus obtains a stereoscopic scene continuity score by using Equation 3.

Meanwhile, the present invention can be implemented by a computer-readable code on a computer-readable recording medium. A computer-readable recording medium includes all kinds of recording apparatuses in which data that can be read by a computer system is stored.

Thus, those skilled in the art will appreciate that the present invention may be embodied in other specific forms without departing from the spirit or essential characteristics thereof. It is therefore to be understood that the embodiments described above are to be considered in all respects only as illustrative and not restrictive. The scope of the present invention is defined by the appended claims rather than the detailed description and all changes or modifications derived from the meaning and scope of the claims and their equivalents are to be construed as being included within the scope of the present invention do.

According to the present invention, it is possible to determine whether to separate the scene into independent scene transitions or separate them into one continuous scene transition using the features of the image. It can be applied to a scene change detection apparatus and method of an image.

200: scene change detection device 210: scene change point detection module
220: stereoscopic scene continuity score calculation module 222: feature point extraction unit
224: feature point matching unit 226: three-dimensional scene continuity score calculation unit
230: scene continuity determination module

Claims (11)

A scene change point detection module detecting a scene change point in an input image;
A stereoscopic scene continuity score calculation module for extracting feature points in the last frame of the detected scene change point and calculating stereoscopic scene continuity scores using feature points tracked in the first frame of the scene from which the extracted feature points are scene changed; And
A scene continuity determining module for comparing the calculated stereoscopic scene continuity score with a preset threshold value and determining whether the stereoscopic scene is continuity based on the comparison result;
Scene change detection device of a three-dimensional image comprising a. The method of claim 1,
The scene change point detection module detects a scene change point by using at least one of a correlation, statistical sequential analysis, and histogram. The method of claim 1,
The stereoscopic scene continuity score calculation module,
A feature point extraction unit for extracting feature points from the last frame of the scene change point detected by the scene change point detection module;
A feature point matching unit for matching the feature points extracted in the last frame with the first frame to obtain the number of feature points tracked in the first frame; And
Calculate a stereoscopic scene continuity score for obtaining a sum of absolute difference (SAD) in a predefined block based on the feature points of the last frame and the feature points tracked in the first frame, and obtaining a stereoscopic scene continuity score using the obtained SAD And a scene change detection device of a stereoscopic image. The method of claim 3,
And the stereoscopic scene continuity score calculator calculates a stereoscopic scene continuity score (C (s)) using the following equation.
[Mathematical Expression]
C (s) = (Tracked feature points / total feature points) * (Σ (1 / SAD (fn, bm))
Here, the total number of feature points is the number of feature points extracted from the last frame of the scene change point, the number of tracked feature points is the number of feature points tracked in the first frame of the scene changed image, fn is the number of the frame, bm is m of the frame Means the first block. 5. The method of claim 4,
And SAD (fn, bm) is obtained by using the following equation.
[Mathematical Expression]
SAD (fn, bm) = Σ abs (Frame (fn, bm) pixel (i)-Frame (fn + 1, bm) pixel (i))
Here, fn is the number of the frame, bm is the m-th block of the frame, i is the order of each pixel of the block, abs is the absolute value. The method of claim 1,
And the scene continuity determining module determines that the scene is a continuous scene when the three-dimensional scene continuity score is greater than or equal to the threshold value, and determines that a scene change is performed when the stereoscopic scene continuity score is not greater than or equal to the threshold value. In the scene change detection device detects a scene change of a stereoscopic image,
(a) detecting a scene change point from an input image;
(b) extracting feature points in the last frame of the detected scene change point, and calculating a stereoscopic scene continuity score using the feature points tracked in the first frame of the scene from which the extracted feature points are transitioned; And
(c) comparing the calculated stereoscopic scene continuity score with a preset threshold and determining whether the stereoscopic scene is continuous based on the comparison result;
Scene change detection method of a stereoscopic image comprising a. The method of claim 7, wherein
The step (b)
Extracting feature points in the last frame of the detected scene change point;
Matching the feature points extracted in the last frame with the first frame to obtain the number of feature points tracked in the first frame;
Obtaining a sum of absolute difference (SAD) in a predefined block based on the feature points of the last frame and the feature points tracked in the first frame; And
And calculating a stereoscopic scene continuity score by using the obtained SAD and the tracked number of feature points. 9. The method of claim 8,
And the stereoscopic scene continuity score (C (s)) is calculated using the following equation.
[Mathematical Expression]
C (s) = (Tracked feature points / total feature points) * (Σ (1 / SAD (fn, bm))
Here, the total number of feature points is the number of feature points extracted from the last frame of the scene change point, the number of tracked feature points is the number of feature points tracked in the first frame of the scene changed image, fn is the number of the frame, bm is m of the frame Means the first block. The method of claim 7, wherein
The step (c)
And if the stereoscopic scene continuity score is greater than or equal to the threshold, determine that the scene is a continuous scene, and if it is not greater than or equal to the threshold, determine that a scene change is made. (a) detecting a scene change point from an input image;
(b) extracting feature points in the last frame of the detected scene change point, and calculating a stereoscopic scene continuity score using the feature points tracked in the first frame of the scene from which the extracted feature points are transitioned; And
and (c) comparing the calculated stereoscopic scene continuity score with a preset threshold and determining whether the stereoscopic scene is continuous based on the comparison result. Readable record carrier.

Images