WO2013077546A1 - Apparatus and method for detecting a scene change in a stereoscopic video - Google Patents

Apparatus and method for detecting a scene change in a stereoscopic video Download PDF

Info

Publication number
WO2013077546A1
WO2013077546A1 PCT/KR2012/008237 KR2012008237W WO2013077546A1 WO 2013077546 A1 WO2013077546 A1 WO 2013077546A1 KR 2012008237 W KR2012008237 W KR 2012008237W WO 2013077546 A1 WO2013077546 A1 WO 2013077546A1
Authority
WO
WIPO (PCT)
Prior art keywords
scene
feature points
frame
stereoscopic
scene change
Prior art date
Application number
PCT/KR2012/008237
Other languages
French (fr)
Korean (ko)
Inventor
우대식
박재범
전병기
김종대
정원석
Original Assignee
에스케이플래닛 주식회사
시모스 미디어텍(주)
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority to KR10-2011-0123373 priority Critical
Priority to KR1020110123373A priority patent/KR101667011B1/en
Application filed by 에스케이플래닛 주식회사, 시모스 미디어텍(주) filed Critical 에스케이플래닛 주식회사
Publication of WO2013077546A1 publication Critical patent/WO2013077546A1/en

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06KRECOGNITION OF DATA; PRESENTATION OF DATA; RECORD CARRIERS; HANDLING RECORD CARRIERS
    • G06K9/00Methods or arrangements for reading or recognising printed or written characters or for recognising patterns, e.g. fingerprints
    • G06K9/00624Recognising scenes, i.e. recognition of a whole field of perception; recognising scene-specific objects
    • G06K9/00711Recognising video content, e.g. extracting audiovisual features from movies, extracting representative key-frames, discriminating news vs. sport content
    • G06K9/00765Segmenting video sequences, i.e. computational techniques such as parsing or cutting the sequence, low-level clustering or determining units such as shots and scenes
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/142Detection of scene cut or scene change
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/597Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding specially adapted for multi-view video sequence encoding
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N2013/0074Stereoscopic image analysis
    • H04N2013/0081Depth or disparity estimation from stereoscopic image signals

Abstract

The present invention relates to an apparatus and method for detecting a scene change in a stereoscopic video, wherein the apparatus includes: a module for detecting a scene change point in an input video; a module for calculating a stereoscopic scene continuity score which extracts feature points in the last frame of the detected scene change point and calculates a stereoscopic scene continuity score by using the extracted feature points tracked in the first frame of the scene change video; and a module for determining scene continuity which compares the calculated stereoscopic scene continuity score with a preset threshold, and determines whether the stereoscopic scene is continuous on the basis of the compared result. According to the present invention, whether a typical scene change point is to be divided into independent scene changes or distinguished as one continuous scene change and whether the scene change point is continuous in a stereoscopic view can be determined by using video features.

Description

Scene change detection apparatus and method for stereoscopic images

The present invention relates to an apparatus and method for detecting a scene change of a stereoscopic image, and more particularly, to extract feature points from a last frame of a scene change point detected from an input image, and to extract the feature points from the first frame of the scene changed image. The present invention relates to an apparatus and method for detecting a scene of a stereoscopic image, wherein the stereoscopic scene continuity score is calculated using the tracked feature points, and the stereoscopic scene continuity score is compared with a preset threshold to determine the continuity of the stereoscopic scene.

In general, a scene change refers to a phenomenon in which a video is restarted in another scene after one scene is finished in a sequence of video scenes. A transition usually involves a fade-out and fade-in where any scene gradually fades away and then another scene, slowly overlapping over time in two different directions. overlap), simple scene transitions, etc.

Conventional scene change point detection basically separates the scene change point based on a numerical value such as brightness or histogram of an image. For most of these criteria, it is reasonable to select a numerically discontinuous part as a transition point as shown in the Histogram result of FIG. 1, but in the case of stereoscopic transformation, if there is visual continuity of the object, it should be separated into one continuous scene. There are many cases.

That is, unlike a general scene change point, when a specific object continues to move in a continuous scene, viewers must maintain stereoscopic characteristics of the object so that stereoscopic discomfort is small.

If this is divided into different scene change points and separated into different scenes, different stereoscopic transformations may be applied, and thus, even though the scenes are continuous, the stereoscopic characteristics may be changed to increase visual discomfort.

Of course, if the three-dimensional transformation work by hand, there is no problem because the person visually considers the characteristics of the three-dimensional object without distinguishing the scene transition point, but when the three-dimensional transformation should be performed automatically through image processing, the three-dimensional transformation by the unit of the transition point This transition point is very important because it is efficient to use.

SUMMARY OF THE INVENTION The present invention has been made to solve the above-described problem, and an object of the present invention is to provide an apparatus and method for detecting a scene of a stereoscopic image capable of detecting an optimal scene transition point in view of stereoscopic transformation.

Another object of the present invention is to determine whether to separate the scenes into independent scene transitions or distinguish them as one continuous scene transition in a conventional scene change point using characteristics of the image, thereby determining the presence or absence of the continuity of the scene change point from a stereoscopic point of view. The present invention provides an apparatus and method for detecting a scene change of a stereoscopic image.

According to an aspect of the present invention, a scene change point detection module that detects a scene change point in an input image, extracts feature points from the last frame of the detected scene change point, and the extracted feature points are tracked in the first frame of the scene change image 3D scene continuity score calculation module for calculating a 3D scene continuity score using the extracted feature points, and a scene continuity for comparing the calculated 3D scene continuity score with a predetermined threshold value and determining the continuity of the 3D scene based on the comparison result. An apparatus for detecting a scene change of a stereoscopic image including a determination module is provided.

The scene change point detection module detects the scene change point using at least one method of correlation, statistical sequential analysis, and histogram.

The stereoscopic scene continuity score calculation module may include a feature point extracting unit extracting feature points from the last frame of the scene change point detected by the scene change point detection module, and matching the feature points extracted from the last frame with the first frame. A feature point matching unit for obtaining the number of feature points tracked in the first frame, a sum of absolute difference (SAD) in a predefined block based on the feature points of the last frame and the feature points tracked in the first frame, and the obtained SAD It includes a stereoscopic scene continuity score calculation unit for obtaining a stereoscopic scene continuity score.

The stereoscopic scene continuity score calculation module obtains the stereoscopic scene continuity score C (s) using the following equation.

[Equation]

C (s) = (Tracked feature points / total feature points) * (Σ (1 / SAD (fn, bm))

Here, the total number of feature points is the number of feature points extracted from the last frame of the scene change point, the number of tracked feature points is the number of feature points tracked in the first frame of the scene changed image, fn is the number of the frame, bm is m of the frame Means the first block.

The SAD (fn, bm) is obtained by using the following equation.

[Equation]

SAD (fn, bm) = Σ abs (Frame (fn, bm) pixel (i)-Frame (fn + 1, bm) pixel (i))

Here, fn is the number of the frame, bm is the m-th block of the frame, i is the order of each pixel of the block.

The scene continuity determining module determines that the three-dimensional scene continuity score is equal to or greater than the threshold, and determines that the scene is continuous.

According to another aspect of the present invention, a method for detecting a scene change of a stereoscopic image by the scene change detection apparatus, comprising: (a) detecting a scene change point in an input image, (b) a last frame of the detected scene change point Extracting feature points and calculating a stereoscopic scene continuity score using the feature points tracked in the first frame of the scene from which the extracted feature points are transitioned; (c) comparing the calculated stereoscopic scene continuity score with a preset threshold; There is provided a scene change detection method of a stereoscopic image comprising the step of determining the continuity of the stereoscopic scene based on the comparison result.

In the step (b), extracting feature points in the last frame of the detected scene change point, matching the feature points extracted in the last frame with the first frame, and extracting the number of feature points tracked in the first frame. Obtaining a sum of absolute difference (SAD) in a predefined block based on the feature points of the last frame and the feature points tracked in the first frame, and using the obtained SAD and the number of tracked feature points. Calculating a scene continuity score.

The stereoscopic scene continuity score C (s) is obtained using the following equation.

[Equation]

C (s) = (Tracked feature points / total feature points) * (Σ (1 / SAD (fn, bm))

Here, the total number of feature points is the number of feature points extracted from the last frame of the scene change point, the number of tracked feature points is the number of feature points tracked in the first frame of the scene changed image, fn is the number of the frame, bm is m of the frame Means the first block.

In the step (c), if the stereoscopic scene continuity score is greater than or equal to the threshold, it is determined that the scene is continuous, and if it is not greater than or equal to the threshold, it is determined that a scene change is made.

According to another aspect of the present invention, (a) detecting a scene change point in an input image, (b) extracting feature points in the last frame of the detected scene change point, and the extracted feature points of the scene Calculating a stereoscopic scene continuity score using the feature points tracked in the first frame; and (c) comparing the calculated stereoscopic scene continuity score with a preset threshold and determining the continuity of the stereoscopic scene based on the comparison result. A scene change detection method of a stereoscopic image includes a program and a recording medium readable by an electronic device is provided.

According to the present invention, an optimum scene change point can be detected in terms of stereoscopic transformation.

In addition, it is possible to determine whether to separate the scene into independent scene transitions or separate them into one continuous scene transition at a normal scene change point by using the feature of the image, thereby determining whether or not the scene change point is continuous from a stereoscopic point of view.

1 is a diagram representing a numerical value of a conventional scene change point.

2 is a block diagram schematically showing the configuration of an apparatus for detecting a scene change of a stereoscopic image according to the present invention;

Figure 3 is an exemplary view for explaining a method for obtaining the inter-frame SAD according to the present invention.

4 is a flowchart illustrating a method for checking whether a scene is continuous by a scene change detection apparatus of a stereoscopic image according to the present invention;

5 is a flowchart illustrating a method for obtaining a stereoscopic scene continuity score by the scene change detection apparatus according to the present invention.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the description with reference to the accompanying drawings, the same or corresponding components will be given the same reference numerals and redundant description thereof will be omitted.

FIG. 2 is a block diagram schematically illustrating a configuration of an apparatus for detecting a scene change of a stereoscopic image according to the present invention, and FIG. 3 is an exemplary diagram for describing a method for obtaining an inter-frame SAD according to the present invention.

Referring to FIG. 2, the scene change detection apparatus 200 for a stereoscopic image includes a scene change point detection module 210, a stereoscopic scene continuity score calculation module 220, and a scene continuity determination module 230.

The scene change point detection module 210 detects a scene change point from an input image. That is, the scene change point detection module 210 obtains a scene change point of an input image by using methods such as brightness, correlation, statistical sequential analysis, histogram, and the like.

Since the scene change point detection module 210 detects the scene change point according to the related art, a detailed description thereof will be omitted.

The stereoscopic scene continuity score calculation module 220 extracts feature points from the last frame of the scene change point detected by the scene change point detection module 210, and tracks the feature points tracked in the first frame of the image after the extracted feature points are scene changed. Calculate the stereoscopic scene continuity score using. Here, the last frame of the scene change point means the last frame before the scene change, the first frame of the scene change image means the first frame after the scene change, and before and after the scene change is divided by the scene change point.

The stereoscopic scene continuity score calculation module 220 performing the above role includes a feature point extractor 222, a feature point matcher 224, and a stereoscopic scene continuity score calculator 226.

The feature point extractor 222 extracts feature points using Harris's corner extraction method, SIFT (Scale Invariant Feature Transform) algorithm, etc. in the last frame of the scene change point detected by the scene change point detection module 210. Here, the feature point may be defined in advance as referring to an edge or a corner.

The feature point extractor 222 extracts a feature point from the last frame of the scene change point in order to find an association of objects between images before and after the scene change point.

The feature point matching unit 224 matches the feature points extracted in the last frame of the scene change point with the first frame of the scene changed image to obtain the number of feature points tracked in the first frame.

For example, the feature points extracted from the last frame before the scene change point are feature point 1, feature point 2, feature point 3, feature point 4 and feature point 5, and as a result of matching the extracted feature points to the first frame after the scene change point feature point 1, feature point When 2 and 4 are tracked, the feature point matching unit 224 obtains "3" as the number of tracked features.

The stereoscopic scene continuity score calculator 226 obtains a sum of absolute difference (SAD) in a predefined block based on the feature points of the last frame and the feature points tracked in the first frame, and uses the obtained SAD. Obtain a stereoscopic scene continuity score. Here, the predefined block refers to a block having a predetermined predetermined size including neighboring pixels based on the feature point. The stereoscopic scene continuity score refers to a score indicating how much the feature points are maintained in the next scene-switched image based on the feature points extracted in the last frame.

The three-dimensional scene continuity score calculator 226 determines the continuity of the scene (s) and the scene (s + 1) in consideration of the number of feature points that can be actually tracked and the reliability of the feature points obtained by tracking the number of feature points to be performed. Find the score for. Here, Scene (s) refers to a scene corresponding to the scene change point, and Scene (s + 1) refers to a scene after the scene change point.

Accordingly, the stereoscopic scene continuity score calculator 226 calculates the stereoscopic scene continuity score C (s) using Equation 1.

Equation 1

Figure PCTKR2012008237-appb-M000001

Here, the total number of feature points refers to the number of feature points extracted from the last frame of the scene change point, and the number of tracked feature points refers to the number of feature points tracked in the first frame after the scene change point obtained by the feature point matching unit.

F (n) denotes reliability of each feature point, n denotes an index of traceable feature points, and s denotes an index of each scene.

The reliability (F (n)) of each feature point is easy to track when tracking at the first frame of the scene (s + 1), which is then transitioned according to the conventional method at the last frame of the scene (s), for any feature point n. Means. In other words, the reliability of each feature point represents how similarly some regions of the image formed based on the feature points exist in and close to each other in the next image to be tracked.

If the transition is made to a completely different image regardless of the previous image, it will not be possible to find areas with similar characteristics, which are excluded from the trace because they are essentially impossible to trace. However, if a certain degree of similarity is found, the measure of similarity found is obtained using the sum of absolute difference (SAD).

Accordingly, the stereoscopic scene continuity score calculator 226 calculates the stereoscopic scene continuity score using the SAD.

In this case, the stereoscopic scene continuity score calculator 226 calculates SAD by using Equation 2.

Equation 2

Figure PCTKR2012008237-appb-M000002

Here, fn is the number of the frame, bm is the m-th block of the frame, i is the order of each pixel of the block, abs is the absolute value.

The sum of absolute difference (SAD) is a sum of absolute differences of all pixels in a block, and as a result, represents a difference between blocks at the same position between frames. Therefore, the large SAD value means that the change of the image is large.

Accordingly, the stereoscopic scene continuity score calculator 226 divides the frame into a predetermined number of blocks based on the feature points, and obtains SAD between the current frame and the previous frame for each block.

Referring to FIG. 3 for how the stereoscopic scene continuity score calculator 226 obtains SAD, the current frame is Frame (fn), the previous frame is Frame (fn-1), and the subsequent frame is Frame (fn + 1). ), The stereoscopic scene continuity score calculator 226 determines the SAD between blocks bm at the same position in each frame, that is, SAD (fn-1, bm) between the current frame block bm and the previous frame block bm, and the current frame. And then SAD (fn, bm) between frame blocks bm are obtained, respectively.

Therefore, when the current frame is Frame (fn) and the subsequent frame is Frame (fn + 1), the three-dimensional scene continuity score calculator 226 determines the SAD between the blocks bm at the same position in each frame, that is, the current frame. SAD (fn, bm) between block bm and subsequent frame block bm is obtained, respectively.

As described above, the stereoscopic scene continuity score calculator 226 calculates SAD for each block by a predetermined spatial size for the feature points selected at the scene change point. If the obtained SAD value is greater than or equal to the threshold value, this means that tracking is not practical in reality, and thus, except for the tracking number of feature points, tracking may be possible if the SAD value is not greater than or equal to the threshold value. Therefore, the smaller the actual SAD value, the greater the similarity and the greater the reliability. In other words, the smaller the SAD, the greater the reliability of the inverse scaffold relationship.

Accordingly, the stereoscopic scene continuity score calculator 226 may obtain the reliability F (n) of each feature point by using the SAD.

Therefore, the stereoscopic scene continuity score calculator 226 may express Equation 1 as Equation 3 below.

Equation 3

Figure PCTKR2012008237-appb-M000003

Here, the total number of feature points is the number of feature points extracted from the last frame of the scene change point, the number of tracked feature points is the number of feature points tracked in the first frame of the scene changed image, fn is the number of the frame, bm is m of the frame Means the second block.

The stereoscopic image continuity score calculation module 220 configured as described above obtains a sum of absolute difference (SAD) in a predefined block based on the feature points of the last frame and the feature points tracked in the first frame that has been transitioned. The stereoscopic scene continuity score is calculated using SAD.

In addition, the stereoscopic image continuity score calculation module 220 may correct the stereoscopic scene continuity score by performing feature point matching on at least one frame to the left, right (ie, before or after the scene change point) of the scene change point. .

The scene continuity determining module 230 compares the stereoscopic scene continuity score obtained by the stereoscopic scene continuity score calculation module 220 with a preset threshold value, and determines the continuity of the stereoscopic scene based on the comparison result.

That is, the scene continuity determining module 230 determines that the scene is a continuous scene when the three-dimensional scene continuity score is greater than or equal to a preset threshold value, and determines that a scene change is performed when the scene continuity score is not more than a threshold value.

In other words, the scene continuity determining module 230 determines that two scene (s) and a scene (s + 1) separated in the conventional manner by the scene change point detection module 210 when the three-dimensional scene continuity score is greater than or equal to a threshold. ) Are processed by combining them into a scene in the stereoscopic image.

The scene change detection apparatus 200 of the stereoscopic image configured as described above automatically considers the detection of the scene change point for stereoscopic conversion, and basically selects the existing scene change point first and performs the additional detection as described above. This is done by examining whether or not you should.

In addition, the apparatus 200 for detecting a scene change of the stereoscopic image defines a criterion for determining whether to separate the scene into independent scene changes or to distinguish one continuous scene change from the normal scene change point.

4 is a flowchart illustrating a method of checking whether a scene is continuous by a scene change detection apparatus of a stereoscopic image according to the present invention.

Referring to FIG. 4, a scene change detection apparatus of a stereoscopic image detects a scene change point with respect to an input image (S302). That is, the scene change detection apparatus detects a scene change point of an input image by using a method such as correlation, statistical sequential analysis, histogram, and the like.

After performing the step S302, the scene change detection apparatus extracts the feature points from the last frame of the detected scene change point (S304), and the stereoscopic scene using the feature points tracked in the first frame of the scene where the extracted feature points are scene changed. The continuity score is calculated (S306). A detailed description of a method for obtaining a stereoscopic scene continuity score by the scene change detection apparatus will be given with reference to FIG. 5.

After performing S306, the scene change detection apparatus determines whether the obtained stereoscopic scene continuity score is equal to or greater than a preset threshold (S308).

If the stereoscopic scene continuity score is greater than or equal to a threshold as a result of the determination of S308, the scene change detection device determines that the frame is a continuous scene (S310).

If the stereoscopic scene continuity score is not greater than or equal to the threshold as a result of the determination in S308, the scene change detection apparatus determines that the scene change is performed (S312).

5 is a flowchart illustrating a method for obtaining a stereoscopic scene continuity score by the scene change detection apparatus according to the present invention.

Referring to FIG. 5, the scene change detection apparatus obtains the number of feature points tracked in the first frame after the scene change point in the feature points of the last frame where the scene change point is detected (S402). That is, the scene change detection apparatus obtains the number of feature points tracked in the first frame by matching the feature points extracted in the last frame with the first frame.

After performing S402, the scene change detection apparatus obtains a sum of absolute difference (SAD) in a predefined block based on the feature points of the last frame and the feature points tracked in the first frame (S404).

That is, the scene change detection apparatus obtains SAD by using Equation 2.

After performing S404, the scene change detection apparatus calculates a stereoscopic scene continuity score using the obtained SAD and the number of tracked feature points (S406).

That is, the scene change detection apparatus obtains a stereoscopic scene continuity score by using Equation 3.

Meanwhile, the present invention can be embodied as computer readable codes on a computer readable recording medium. The computer-readable recording medium includes all kinds of recording devices in which data that can be read by a computer system is stored.

An apparatus for detecting a scene change of a stereoscopic image according to the present invention may include a processor, a memory, a storage device and an input / output device as components, which components may be interconnected using, for example, a system bus.

The processor may process instructions for execution within the device. In one implementation, the processor may be a single-threaded processor, and in other implementations, the processor may be a multi-threaded processor. The processor processes instructions stored on memory or storage devices. It is possible to do

On the other hand, the memory stores information in the apparatus. In one embodiment, the memory is a computer readable medium. In one implementation, the memory may be a volatile memory unit, and for other implementations, the memory may be a nonvolatile memory unit. The storage device described above can provide a mass storage for the device. In one embodiment, the storage device is a computer readable medium. In various different implementations, the storage device may include, for example, a hard disk device, an optical disk device, or some other mass storage device.

The above-described input / output device provides an input / output operation for the device according to the present invention. In one implementation, the input / output device may include one or more network interface devices such as, for example, an Ethernet card, such as a serial communication device such as an RS-232 port and / or a wireless interface device such as, for example, an 802.11 card. In other implementations, the input / output device can include driver devices, such as keyboards, printers, and display devices, configured to send output data to and receive input data from other input / output devices.

The apparatus according to the invention may be driven by instructions that cause one or more processors to perform the functions and processes described above. For example, such instructions may include instructions that are interpreted, for example, script instructions such as JavaScript or ECMAScript instructions, or executable code or other instructions stored on a computer readable medium. Furthermore, the device according to the present invention may be implemented in a distributed manner over a network, such as a server farm, or may be implemented in a single computer device.

Although the specification and drawings describe exemplary device configurations, the functional operations and subject matter implementations described herein may be embodied in other types of digital electronic circuitry, or modified from the structures and structural equivalents disclosed herein. It may be implemented in computer software, firmware or hardware, including, or a combination of one or more of them. Implementations of the subject matter described herein relate to one or more computer program products, ie computer program instructions encoded on a program storage medium of tangible type for controlling or by the operation of an apparatus according to the invention. It may be implemented as the above module. The computer readable medium may be a machine readable storage device, a machine readable storage substrate, a memory device, a composition of materials affecting a machine readable propagated signal, or a combination of one or more thereof.

The terms "processing system", "processing device" and "subsystem" encompass all the instruments, devices and machines for processing data, including, for example, programmable processors, computers or multiple processors or computers. The processing system may include, in addition to hardware, code that forms an execution environment for a computer program on demand, such as code constituting processor firmware, a protocol stack, a database management system, an operating system, or a combination of one or more thereof. .

A computer program (also known as a program, software, software application, script or code) mounted on an apparatus according to the invention and executing a method according to the invention is a programming comprising a compiled or interpreted language or a priori or procedural language. It can be written in any form of language, and can be deployed in any form, including stand-alone programs or modules, components, subroutines, or other units suitable for use in a computer environment. A computer program does not necessarily correspond to a file in a file system. A program may be in a single file provided to the requested program, in multiple interactive files (eg, a file that stores one or more modules, subprograms, or parts of code), or part of a file that holds other programs or data. (Eg, one or more scripts stored in a markup language document). The computer program may be deployed to run on a single computer or on multiple computers located at one site or distributed across multiple sites and interconnected by a communication network.

Computer-readable media suitable for storing computer program instructions and data include, for example, semiconductor memory devices such as EPROM, EEPROM, and flash memory devices, such as magnetic disks such as internal hard disks or external disks, magneto-optical disks, and CD-ROMs. And all forms of nonvolatile memory, media and memory devices, including DVD-ROM discs. The processor and memory can be supplemented by or integrated with special purpose logic circuitry.

Implementations of the subject matter described herein may include, for example, a backend component such as a data server, or include a middleware component such as, for example, an application server, or a web browser or graphical user, for example, where a user may interact with the implementation of the subject matter described herein. It can be implemented in a computing system that includes a front end component such as a client computer having an interface or any combination of one or more of such back end, middleware or front end components. The components of the system may be interconnected by any form or medium of digital data communication such as, for example, a communication network.

Although the specification includes numerous specific implementation details, these should not be construed as limiting to any invention or the scope of the claims, but rather as a description of features that may be specific to a particular embodiment of a particular invention. It must be understood. Certain features that are described in this specification in the context of separate embodiments may be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable subcombination. Furthermore, while the features may operate in a particular combination and may be initially depicted as so claimed, one or more features from the claimed combination may in some cases be excluded from the combination, the claimed combination being a subcombination Or a combination of subcombinations.

Likewise, although the operations are depicted in the drawings in a specific order, it should not be understood that such operations must be performed in the specific order or sequential order shown in order to obtain desirable results or that all illustrated operations must be performed. In certain cases, multitasking and parallel processing may be advantageous. Moreover, the separation of the various system components of the above-described embodiments should not be understood as requiring such separation in all embodiments, and the described program components and systems will generally be integrated together into a single software product or packaged into multiple software products. It should be understood that it can.

Specific embodiments of the subject matter described in this specification have been described. Other embodiments are within the scope of the following claims. For example, the actions recited in the claims can be performed in a different order but still achieve desirable results. As an example, the process depicted in the accompanying drawings does not necessarily require that particular illustrated or sequential order to obtain desirable results. In certain implementations, multitasking and parallel processing may be advantageous.

The foregoing description presents the best mode of the invention, and provides examples to illustrate the invention and to enable those skilled in the art to make and use the invention. The specification thus produced is not intended to limit the invention to the specific terms presented. Thus, while the present invention has been described in detail with reference to the examples described above, those skilled in the art can make modifications, changes and variations to the examples without departing from the scope of the invention.

As such, those skilled in the art will appreciate that the present invention can be implemented in other specific forms without changing the technical spirit or essential features thereof. Therefore, the above-described embodiments are to be understood as illustrative in all respects and not as restrictive. The scope of the present invention is shown by the following claims rather than the above description, and all changes or modifications derived from the meaning and scope of the claims and their equivalent concepts should be construed as being included in the scope of the present invention. do.

According to the present invention, it is possible to determine whether to separate the scene into independent scene transitions or separate them into one continuous scene transition using the features of the image. It can be applied to a scene change detection apparatus and method of an image.

Claims (11)

  1. A scene change point detection module detecting a scene change point in an input image;
    A stereoscopic scene continuity score calculation module for extracting feature points in the last frame of the detected scene change point and calculating stereoscopic scene continuity scores using feature points tracked in the first frame of the scene from which the extracted feature points are scene changed; And
    A scene continuity determining module for comparing the calculated stereoscopic scene continuity score with a preset threshold value and determining whether the stereoscopic scene is continuity based on the comparison result;
    Scene change detection device of a three-dimensional image comprising a.
  2. The method of claim 1,
    The scene change point detection module detects a scene change point by using at least one of a correlation, statistical sequential analysis, and histogram.
  3. The method of claim 1,
    The stereoscopic scene continuity score calculation module,
    A feature point extraction unit for extracting feature points from the last frame of the scene change point detected by the scene change point detection module;
    A feature point matching unit for matching the feature points extracted in the last frame with the first frame to obtain the number of feature points tracked in the first frame; And
    Calculate a stereoscopic scene continuity score for obtaining a sum of absolute difference (SAD) in a predefined block based on the feature points of the last frame and the feature points tracked in the first frame, and obtaining a stereoscopic scene continuity score using the obtained SAD And a scene change detection device of a stereoscopic image.
  4. The method of claim 3,
    And the stereoscopic scene continuity score calculator calculates a stereoscopic scene continuity score (C (s)) using the following equation.
    [Equation]
    C (s) = (Tracked feature points / total feature points) * (Σ (1 / SAD (fn, bm))
    Here, the total number of feature points is the number of feature points extracted from the last frame of the scene change point, the number of tracked feature points is the number of feature points tracked in the first frame of the scene changed image, fn is the number of the frame, bm is m of the frame Means the first block.
  5. The method of claim 4, wherein
    And SAD (fn, bm) is obtained by using the following equation.
    [Equation]
    SAD (fn, bm) = Σ abs (Frame (fn, bm) pixel (i)-Frame (fn + 1, bm) pixel (i))
    Here, fn is the number of the frame, bm is the m-th block of the frame, i is the order of each pixel of the block, abs is the absolute value.
  6. The method of claim 1,
    And the scene continuity determining module determines that the scene is a continuous scene when the three-dimensional scene continuity score is greater than or equal to the threshold value, and determines that a scene change is performed when the stereoscopic scene continuity score is not greater than or equal to the threshold value.
  7. In the scene change detection device detects a scene change of a stereoscopic image,
    (a) detecting a scene change point from an input image;
    (b) extracting feature points in the last frame of the detected scene change point, and calculating a stereoscopic scene continuity score using the feature points tracked in the first frame of the scene from which the extracted feature points are transitioned; And
    (c) comparing the calculated stereoscopic scene continuity score with a preset threshold and determining whether the stereoscopic scene is continuous based on the comparison result;
    Scene change detection method of the stereoscopic image comprising a.
  8. The method of claim 7, wherein
    In step (b),
    Extracting feature points in the last frame of the detected scene change point;
    Matching the feature points extracted in the last frame with the first frame to obtain the number of feature points tracked in the first frame;
    Obtaining a sum of absolute difference (SAD) in a predefined block based on the feature points of the last frame and the feature points tracked in the first frame; And
    And calculating a stereoscopic scene continuity score by using the obtained SAD and the tracked number of feature points.
  9. The method of claim 8,
    And the stereoscopic scene continuity score (C (s)) is calculated using the following equation.
    [Equation]
    C (s) = (Tracked feature points / total feature points) * (Σ (1 / SAD (fn, bm))
    Here, the total number of feature points is the number of feature points extracted from the last frame of the scene change point, the number of tracked feature points is the number of feature points tracked in the first frame of the scene changed image, fn is the number of the frame, bm is m of the frame Means the first block.
  10. The method of claim 7, wherein
    In step (c),
    And if the stereoscopic scene continuity score is greater than or equal to the threshold, determine that the scene is a continuous scene, and if it is not greater than or equal to the threshold, determine that a scene change is made.
  11. (a) detecting a scene change point from an input image;
    (b) extracting feature points in the last frame of the detected scene change point, and calculating a stereoscopic scene continuity score using the feature points tracked in the first frame of the scene from which the extracted feature points are transitioned; And
    and (c) comparing the calculated stereoscopic scene continuity score with a preset threshold and determining whether the stereoscopic scene is continuous based on the comparison result. Readable record carrier.
PCT/KR2012/008237 2011-11-24 2012-10-11 Apparatus and method for detecting a scene change in a stereoscopic video WO2013077546A1 (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
KR10-2011-0123373 2011-11-24
KR1020110123373A KR101667011B1 (en) 2011-11-24 2011-11-24 Apparatus and Method for detecting scene change of stereo-scopic image

Publications (1)

Publication Number Publication Date
WO2013077546A1 true WO2013077546A1 (en) 2013-05-30

Family

ID=48469955

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/KR2012/008237 WO2013077546A1 (en) 2011-11-24 2012-10-11 Apparatus and method for detecting a scene change in a stereoscopic video

Country Status (2)

Country Link
KR (1) KR101667011B1 (en)
WO (1) WO2013077546A1 (en)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103489191B (en) * 2013-09-24 2016-04-13 中国科学院自动化研究所 Significant target remote sensing image change detection method
KR20190005651A (en) 2017-07-07 2019-01-16 삼성에스디에스 주식회사 Apparatus and method for detecting scene change

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2003196662A (en) * 2001-12-27 2003-07-11 Ntt Data Corp Cut detection device and its program
JP2009540667A (en) * 2006-06-08 2009-11-19 トムソン ライセンシングThomson Licensing Method and apparatus for detecting a scene change
KR20100060330A (en) * 2008-11-27 2010-06-07 삼성전자주식회사 Apparatus and method for creating multi-view image for stereo image
KR20110050364A (en) * 2009-11-06 2011-05-13 삼성전자주식회사 Method and apparatus for parallax adjusting in stereoscopic video

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR101490521B1 (en) 2007-10-10 2015-02-06 삼성전자주식회사 Method for real-time scene-change detection for rate control of video encoder, method for enhancing qulity of video telecommunication using the same, and system for the video telecommunication
KR101050255B1 (en) * 2009-08-25 2011-07-19 주식회사 노매드커넥션 video scene segmentation system and method
KR101167645B1 (en) * 2010-07-27 2012-07-20 (주)자람테크놀로지 Method for detecting scene change and apparatus therof

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2003196662A (en) * 2001-12-27 2003-07-11 Ntt Data Corp Cut detection device and its program
JP2009540667A (en) * 2006-06-08 2009-11-19 トムソン ライセンシングThomson Licensing Method and apparatus for detecting a scene change
KR20100060330A (en) * 2008-11-27 2010-06-07 삼성전자주식회사 Apparatus and method for creating multi-view image for stereo image
KR20110050364A (en) * 2009-11-06 2011-05-13 삼성전자주식회사 Method and apparatus for parallax adjusting in stereoscopic video

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
CHUNG-LIN HUANG ET AL. IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY vol. 11, 31 December 2001, pages 1281 - 1288 *

Also Published As

Publication number Publication date
KR101667011B1 (en) 2016-10-18
KR20130057585A (en) 2013-06-03

Similar Documents

Publication Publication Date Title
US8204312B2 (en) Moving image editing apparatus
KR100645300B1 (en) Method and apparatus for summarizing and indexing the contents of an audio-visual presentation
Cernekova et al. Information theory-based shot cut/fade detection and video summarization
CN100593335C (en) Motion detection method and device
CN101271514B (en) Image detection method and device for fast object detection and objective output
CN102292726B (en) Video identifier extraction equipment
US20140169663A1 (en) System and Method for Video Detection and Tracking
TW200907824A (en) Image processing apparatus, imaging apparatus, image processing method, and computer program
BRPI0721462A2 (en) 2d image region classification system and method for 2d to 3d conversion
KR20060117890A (en) Method and apparatus for video copy detection
US20160092561A1 (en) Video analysis techniques for improved editing, navigation, and summarization
CN102981604B (en) The image processing apparatus and an image processing method
Zhang et al. Robust metric reconstruction from challenging video sequences
WO2013085148A1 (en) Apparatus and method for eliminating noise in stereo image
TWI551134B (en) And generating a video fingerprint method for performing recognition of at least one computing device and system
JP5166409B2 (en) Video processing method and video processing apparatus
Fan et al. Matching slides to presentation videos using sift and scene background matching
CN1489112A (en) Sports image detecting method
CN101359368B (en) Video image clustering method and system
US20110311100A1 (en) Method, Apparatus and Computer Program Product for Providing Object Tracking Using Template Switching and Feature Adaptation
JP4909840B2 (en) Video processing apparatus, program, and method
CN103593464A (en) Video fingerprint detecting and video sequence matching method and system based on visual features
US20130328773A1 (en) Camera-based information input method and terminal
JP2009048490A (en) Similar shot detection device, program and method
WO2009056038A1 (en) A method and device for describing and capturing video object

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 12850966

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase in:

Ref country code: DE

122 Ep: pct app. not ent. europ. phase

Ref document number: 12850966

Country of ref document: EP

Kind code of ref document: A1