KR100361939B1 - Recording medium and method for constructing and retrieving a data base of a mpeg video sequence by using a object - Google Patents

Abstract

The present invention effectively indexes object motions of various sizes and directions using object motion information, constructs a database for MPEG video sequences based on this indexing vector, and query indexing extracted from query video sequences or query motion trajectories. The present invention relates to a database construction and retrieval method of an MPEG video sequence using an object motion vector, which enables a high speed retrieval of MPEG video sequence scenes having similarity from a database of an MPEG video sequence using a vector. In addition, the object motion information can be extracted more quickly and efficiently by using the motion vector information of the MPEG video sequence, and the object motion vector histogram and the average value of the object motion are drawn to search the database. Because of extracting one indexing vector, unlike the foregoing patents, which require a large storage space and a large amount of decoding time, it is possible to effectively database an MPEG video sequence without wasting large storage space and unnecessary time. The object motion in the MPEG video sequence is retrieved by using the query object motion vector histogram detected from the query data and the average value of the query object motion in the MPEG video sequence stored on the storage medium. The search for information can be realized with high accuracy and speed.

Description

Database construction and retrieval method of MPE video sequence using object movement and its recording medium {RECORDING MEDIUM AND METHOD FOR CONSTRUCTING AND RETRIEVING A DATA BASE OF A MPEG VIDEO SEQUENCE BY USING A OBJECT}

The present invention relates to a technique of databaseting an MPEG video sequence and searching for a scene of a similar MPEG video sequence by using the same. More particularly, the present invention relates to an MPEG video sequence using motion vector (MV) information embedded in the MPEG video sequence. MPEG video sequence database construction and retrieval method using object motion suitable for constructing a database and searching for scenes of an MPEG video sequence similar to a query video sequence or a query motion trajectory using the database of the constructed MPEG video sequence. And its recording medium.

As is well known in the art, the transmission of discrete video signals can maintain better image quality than analog signals, when a video signal consisting of a series of image "frames" is represented in digital form, a very large amount of digital Video data is generated.

However, due to the limited frequency domain available in the transmission channel for transmitting the video signal and the storage space limitations in the storage medium for storing the video signal, the compression of the video signal, for example using well-known hybrid encoding techniques An example of a video sequence compression-coded using such a hybrid encoding technique is an MPEG video sequence. Such MPEG encoding and decoding algorithms are used for HDTV, VCD, video conferencing system, video telephony and the like. Herein, the present invention relates to a technique for constructing and retrieving a database of MPEG video sequences compression-coded through a hybrid encoding technique.

In other words, in the hybrid coding scheme, the spatial and temporal correlation of the video signal is used to compress and encode the digital video signal at a predetermined compression rate, that is, motion compensation DPCM (differential pulse code modulation), two-dimensional discrete cosine transform (DCT), and DCT coefficients. Compression coding is performed using quantization, VLC (variable coding), and the like.

Here, the motion compensation DPCM determines the movement of the object between the current frame and the previous frame, predicts the current frame according to the movement of the object, and generates a differential signal representing the difference between the current frame and the predicted value. In DPCM, the current frame is predicted from the previous frame according to the estimated motion of the object between the current frame and the previous frame. Here, the estimated motion may be represented by a two-dimensional motion vector (for example, a motion vector in macroblock units) representing a displacement between the previous frame and the current frame.

Accordingly, motion vector (MV) information detected in units of macroblocks is inserted into an MPEG video sequence that is compression-coded at a predetermined bit rate according to a hybrid coding scheme, and the MV information includes the original signal before encoding the encoded MPEG video sequence. It is used when restoring (decoding) the data.

On the other hand, various kinds of techniques are required to store (database) MPEG video sequences in a storage medium and to search for motion objects in the stored MPEG video sequences. It is used to retrieve the desired moving objects.

As described above, a conventional technique for searching for moving objects in an MPEG video image is proposed by Karmann et al., And registered with US Patent Office at 5,034,986 under the name of “Method for detecting and tracking moving objects in a digital image sequence having a stationary background”. There is a prior patent (hereinafter referred to as Patent 1).

This patent relates to a method for finding and tracking moving objects in an MPEG video image having a static background. First, in order to find a moving object in a static image, a binary object mask is formed by using difference information between two images. In this case, the appropriate threshold is applied. Then, the shape of segments estimated to be objects in the binary object mask sequence is obtained, and the position information on the obtained shape is recorded.

Next, in Patent Document 1, in order to obtain a motion vector of a detected object, a position at a next frame is found at a position where the gray scale values of the previous frame of a specific position of the mask sequences (that is, the position of the object) are minimized between two frames. The method is disclosed.

However, Patent Document 1, which detects the movement of an object in an MPEG video image in the manner described above, has a fundamental limitation that the MPEG video image is a decoded still image, and also has several MPEG video images for detecting a motion region. Since it has to go through the process of making a still image, there is a fundamental problem that it requires a lot of decoding time and a large storage space.

Furthermore, since the prior patent 1 uses binary mask sequences to detect the motion regions of several objects, it has no problem causing additional decoding time and waste of storage space.

On the other hand, another conventional technique for extracting moving objects from MPEG video images has been proposed by Jang Gyu-hwan, Jeong Hae-muk et al., On April 8, 1995, entitled "Moving Object Segmentation of Motion Images and Its Motion Estimation Method." There is a prior patent (hereinafter referred to as Prior Patent 2) filed with the Korean Patent Office as No. 8100.

This prior patent 2 relates to a technique for tracking the movement of an object by dividing a moving object included in the current frame and describing the movement of the moving object as a set motion parameter. Detects the motion region of the current frame through comparison between previous frames, detects the motion vector field by detecting the motion vector for each pixel of the motion region, and divides the motion vector field using a two-dimensional translational motion model. Technical means for dividing a segmentation result second by using a body motion model, and tertiary segmentation of a second segmentation result using a curved body motion model are disclosed.

That is, the prior patent 2 can extract the motion of an object more accurately than the conventional method by calculating a motion vector that can be represented by 12 components from a basic motion vector that can be represented by two components.

However, this prior patent 2 requires a storage space for storing each frame because the frames of the image must be compared in order to detect a motion vector that can be represented by the first two components, that is, the aforementioned patent 1 In addition to the problem of waste of storage space, as well as in the case that the image is input in real time unit has a problem that requires a lot of decoding time (that is, time to decode the coded video image) to make a frame.

In addition, the prior patent 2 has another problem in that a large number of errors occur in the object motion because the object motion is composed of a plurality of motions are represented as a single two component vector.

On the other hand, another conventional technique for extracting moving objects from MPEG video images is proposed by Nagaya et al., Which is a prior patent (hereinafter referred to as "patent 3") registered by the US Trademark Office as 5,721,692 under the name of "Moving Object detection apparatus". have.

In the prior patent 3, an input unit for an input image and an output unit for an output image are provided to detect an object movement. When a moving region is detected in the background, a movement speed and a direction are calculated as the moving object. That is, each frame of the input image is divided into windows of a specific size, and the window correlation between the previous frame and the current frame is calculated for each frame, and the frame having the window correlation less than or equal to no object movement with respect to the previous frame. Disclosed is a technical means for judging.

That is, the prior patent 3 discloses a method of extracting object motion by creating a window having a specific size and detecting a correlation between the next frame and windows to detect an object motion per frame when an input image is received. It has the advantage that the motion detection unit and the output unit can be extracted simply and efficiently.

However, the prior patent 3, as already mentioned in the foregoing patents 1 and 2, still has a problem of wasting storage space, and also has a problem of specifying a suitable window size according to the MPEG video sequence. That is, the prior patent 3 relates to an object motion extraction technique of a specific size based on a window rather than an object motion extraction technique of various sizes.

Accordingly, the present invention is to solve the above-mentioned problems of the prior art, by using the object motion histogram information to effectively index the object motion of various sizes and directions, and to use the indexing vector to create a database for the MPEG video sequence An object of the present invention is to provide a method for constructing a database of MPEG video sequences using constructable object motions.

Another object of the present invention is to rapidly search for MPEG video sequence scenes having similarity using an indexing vector extracted from a query video sequence or a query motion trajectory from a database of an MPEG video sequence constructed using an indexing vector based on object motion. The present invention provides a video retrieval method of an MPEG video sequence using object motion.

It is still another object of the present invention to construct an MPEG video sequence database using an indexing vector based on object motion, and to have an MPEG video having similarity from a database constructed using an indexing vector extracted from a query video sequence or a query motion trajectory. A program for retrieving a sequence scene is recorded, and a computer-readable recording medium is provided.

According to an aspect of the present invention, there is provided a method of constructing a searchable and readable database of object motion from which background motion is removed from MPEG video sequence data encoded at a predetermined compression rate. Extracting a motion vector present in the MPEG video sequence; A second step of detecting a clustered object motion using object motion vector information after removing a background motion vector from the extracted motion vector; A third step of detecting an object motion vector scene by using the detected cluster motion number; A fourth step of analyzing a content feature histogram of the detected object motion cluster and assigning an identifier for each object; A fifth step of generating an object motion histogram using the detected object motion scene and the assigned identifier, and generating an indexing vector for the detected object motion scene by indexing the generated object motion histogram; And a sixth process of storing the generated indexing vector along with corresponding input MPEG video sequence information in a storage medium.

According to another aspect of the present invention, there is provided a method for retrieving object motion information corresponding to query data from a storage medium in which MPEG video sequence data encoded at a predetermined compression rate is stored together with an object motion indexing vector for retrieval. The method may include: extracting a query motion vector from the input query data when the query data to be searched in the storage medium is input; A second process of removing a background motion vector from the extracted query motion vector and then detecting clustered query object motion; A third step of detecting a query object motion scene by using the detected number of clusters of the query object motion; A fourth step of analyzing a content feature histogram of the detected query object motion cluster and assigning an identifier to each query object; A fifth step of generating a query object motion histogram using the detected query object motion scene and the assigned identifier, and generating the generated query object motion histogram and the average value of the object motion as a query indexing vector; And searching for a similar object motion scene image by comparing and searching the generated query indexing vector and each indexing vector stored in the storage medium, and extracting information about the found similar object motion scene image from the storage medium and corresponding thereto. The present invention provides a method for searching a database of an MPEG video sequence using an object movement consisting of a sixth process of delivering the MPEG video sequence to a display side.

The present invention according to another aspect to achieve the above object, the object motion is removed from the MPEG video sequence data to build a database on a retrievable and readable storage medium, and the aim is to search in the constructed database Recording a program for retrieving similar object motion scenes corresponding to the query data; and extracting a motion vector existing in an input MPEG video sequence in a computer-readable recording medium, and extracting a background motion vector from the extracted motion vector. A first process of removing and then detecting a clustered object motion vector; A second step of detecting an object motion scene by using the detected number of clusters of object motions, analyzing a content feature histogram of the detected object motion clusters, and assigning an identifier for each object; A third process of generating an object motion histogram using the detected object motion scene and the assigned identifier, and generating an indexing vector for the detected object motion scene by indexing the generated object motion histogram and the average value of the object motion. ; A fourth step of storing the generated indexing vector in the storage medium together with information of a corresponding input MPEG video sequence; A fifth query for extracting a query motion vector from the input query data, removing a background motion vector from the extracted query motion vector, and detecting a clustered query object motion when the query data to be searched is input from the storage medium. process; A sixth step of detecting a query object motion scene by using the detected number of query object motion clusters, analyzing a content feature histogram of the detected query object motion clusters, and assigning an identifier for each query object; Generating a query indexing vector by generating a query object motion histogram using the detected query object motion scene and the assigned identifier, and indexing the generated query object motion histogram and an object motion average value; And searching for a similar object motion scene image by comparing and searching the generated query indexing vector and each indexing vector stored in the storage medium, and retrieving information corresponding to the retrieved similar object motion scene image from the storage medium. A computer readable recording medium having recorded thereon a program for executing an eighth process of delivering an MPEG sequence to a display side.

1 illustrates the construction of a database of MPEG video sequences using an indexing vector of object motions detected on the basis of object motions extracted from an MPEG video sequence according to the present invention. Block diagram of an MPEG video sequence processing system configured by functionally blocking each process of searching a query video sequence,

2 is an exemplary diagram illustrating an example of detecting and clustering an object motion in a macroblock unit according to the present invention;

3 is an exemplary diagram illustrating an example of labeling a detected object motion vector into 32 groups by dividing it into four sizes and eight directions;

4 is an exemplary diagram showing that the object motion histogram of a scene continuously changes with time or as the number of frames increases.

5 is a flowchart illustrating a process of constructing a database of MPEG video sequences using object motions according to a preferred embodiment of the present invention;

FIG. 6 is a flowchart illustrating a process of searching for an arbitrary query video sequence or an MPEG video sequence corresponding to a query motion trajectory in a database of an MPEG video sequence constructed using object motion according to the present invention. FIG.

<Description of the code | symbol about the principal part of drawing>

102: selection signal input block 104: motion vector (MV) extraction block

106: object motion detection block 108: object motion scene detection block

110: object movement identifier allocation block

112: object movement indexing block 114: storage medium

116: Object Movement Indexing Search Block

118: video sequence fetch block

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 illustrates a method of constructing a database of MPEG video sequences using an indexing vector of object motions detected based on a motion vector extracted from an MPEG video sequence, and targeting a database constructed using an indexing vector. A block diagram of an MPEG video sequence processing system is constructed by functionally blocking each process of searching a query video sequence.

Referring to FIG. 1, the MPEG video sequence processing system constructed by functional blocking includes a selection signal input block 102, a motion vector (MV) extraction block 104, an object motion vector detection block 106, and an object motion scene detection block ( 108, object motion identifier assignment block 110, object motion indexing block 112, storage medium 114, object motion indexing search block 116, and video sequence retrieval block 118.

First, the selection signal input block 102 includes input means such as a keyboard and a mouse, for example, assuming that the MPEG video sequence processing system is mounted on a personal computer. When a user manipulation signal is inputted using the control signal, a construction control signal corresponding to a corresponding selection signal (or a control signal), for example, information about an MPEG video sequence input from the outside is stored in the database 114. MV extraction block 104 and / or object movement over line L11 by generating a search control signal or the like for searching the database 114 for a query MPEG video sequence or a similar MPEG video sequence scene corresponding to the provided query motion trajectory. To the indexing block 112.

Next, the MPEG video sequence provided from the outside through line L12 for database construction of the MPEG video sequence is provided to the MV extraction block 104 and the database 114, respectively (i.e., database construction mode), and also A query video sequence or query motion trajectory for searching for similar MPEG video sequence scenes in the database 114 is provided to the MV extraction block 104 (i.e., database search mode) in accordance with user requirements. 104 extracts the motion vector inserted into the MPEG video sequence provided through the line L12, that is, the MV detected and inserted in units of macroblocks in the database building mode, and provides the data to the object MV detection block 106; Query video sequence or query input from outside in response to a search selection signal from line L11 in the base search mode The query derived from the MV jikim trajectory provides the object MV detection block 106. Here, the extracted motion vectors have two components, namely, magnitude and direction.

Meanwhile, the object MV detection block 106 detects and clusters an object motion vector from which the background motion vector information is removed using the motion vector MV or the query motion vector MV provided from the MV extraction block 104. Clustering. For example, an object motion vector is detected by comparing a motion vector of a corresponding position between two frames (two temporally adjacent frames), where the detected object motion vector is provided to the object motion scene detection block 108. .

FIG. 2 is a diagram illustrating an example of detecting and clustering an object motion in a macroblock unit according to the present invention, wherein mb denotes a macroblock, mv denotes a motion vector per macroblock, and A and B Denotes a clustered object motion vector.

Subsequently, the object motion scene detection block 108 detects the object motion scene Shot using the number of clustered object motions provided from the object MV detection block 106, and the detected object motion scene information is determined in the following stage. An object motion identifier assignment block 110 is provided. Here, a scene refers to a collection of scenes that are typically consistent in an MPEG video sequence, where a set of identical numbers of moving objects (ie, identical numbers of clustered objects) in an MPEG video sequence is defined as an object motion scene, It is called OMS (Object Motion in Shot). The detected scene is a basic unit of indexing and retrieval for storage.

Next, the object motion identifier assignment block 110 assigns an identifier per object based on the object motion scene information provided from the object motion scene detection block 108 and the frame information provided through the line L12, that is, the clustered motion object. The histogram of the content feature (i.e., color, texture, shape, etc. based on the pixel data) is analyzed and given an identifier for each object.

That is, the frames inside the OMS are clustered into object motion vectors, which can be represented by polar coordinates as shown in FIG. 3 and also labeled with regions of a similar group. In this case, regions of similar groups may be divided into m sizes and n directions according to video sequence characteristics, and each labeled group may represent an object motion type. FIG. 3 is an exemplary diagram illustrating a result of labeling a detected object motion vector into 32 groups by dividing the detected object motion vector into four sizes and eight directions.

Meanwhile, the object motion indexing block 112 analyzes the object motions labeled for each frame in the OMS, draws an object motion histogram, and then indexes the object motion histogram in a shot (OMHS) and the object motion average value. Here, the OMHS is a quantification of the object motion histogram itself, which can reasonably express object motion information having various sizes and directions, and also an indexing vector that can efficiently detect the trajectory of the object motion vector as the average value of the object motion. admit.

That is, FIG. 4 is an exemplary diagram illustrating that the object motion histogram of a scene continuously changes with time or as the number of frames increases.

4, reference numeral S31 denotes a label (object movement type) of the object motion group, S32 denotes the number of object motion vectors, S33 denotes a frame number or time, S34 denotes an object motion vector histogram, and S35 is the trajectory of the object motion vector, that is, the average value of the object motion is projected on the slopes S31 and S33, and represents the object motion vector trajectory indicating how the object motion changes with time.

In this case, the clustered object motion histogram is expressed by Equation 1 below.

OMHS = {H (ft, i); 0≤t≤M, i = 1,2, ----, N}

In Equation 1, f denotes a frame, t denotes a time interval or frame numbers of frames separated by a shot, and i denotes a group label of an object motion vector OM. In addition, H (f _t , i) means the number of object movements having a group label i in a specific scene (by an interval of t), and the set of values is referred to as OMHS described above.

Next, the average value of the object motion in the scene is expressed by Equation 2 below.

In Equation 2, f denotes a frame, and t denotes a time interval or frame numbers of frames separated by a shot. Therefore, the average value of the object motion in the scene is a value that lists the average value of the object motion for a specific scene and represents a motion trajectory for the scene.

Thus, the indexing values obtained from the object motion indexing block 112, ie OMHS and the average value of the object motion, are provided to the storage medium 114 via line L13 or to the object motion indexing search block 116 via line L14. . That is, the object motion indexing block 112 is a motion vector extracted from an MPEG video sequence input when a database construction selection signal is provided from the selection signal input block 102 via the line L11 (that is, in the database construction mode). The indexing vector (OMHS, the object motion average value) detected on the basis of the information is transferred to the database 114 through the line L13 so that it is stored together with the information of the corresponding MPEG video sequence, and the selection signal input block 102 via the line L11. When the database search selection signal is provided (i.e., in the database search mode), the object index is moved to the query indexing vector (query OMHS, mean value of query object motion) detected in the query video sequence or query motion trajectory. Forward to indexing search block 116.

Therefore, in the storage medium 114, when the current mode is the database building mode, the information of the MPEG video sequence provided through the line L11 and the associated indexing vector (OMHS, the object motion average value) provided through the line L13 are randomly selected. Will be stored in the storage space.

That is, according to the present invention, the storage medium 112 stores the indexing vectors corresponding to the information and the moving object, respectively, of the MPEG video sequence to be databased, where the indexing vector corresponding to the moving object is the query video sequence or the query motion. It is used as search index (or search key) information when trying to find a similar scene for the trajectory.

As described above, according to the present invention, the object motion information is extracted more quickly and efficiently using the motion information of the MPEG video sequence, and the indexing vector for searching in the database using the object motion histogram drawn through the object. Because of extracting, unlike the aforementioned patents requiring a large storage space and a large decoding time, it is possible to effectively database the MPEG video sequence without wasting large storage space and unnecessary time.

On the other hand, the object motion indexing search block 116 is provided when the query indexing vector (query OMHS, query object motion average value) of the query video sequence or the query motion trajectory is provided through the line L14, that is, the current mode is the database search mode. When the indexing vector (OMHS) of the MPEG video sequence stored in the storage medium 114 is searched and compared with the query indexing vector, the scene of the MPEG video sequence having similarity with the query video sequence or the query motion trajectory. Sequentially search for (Shot).

That is, a method of retrieving the object motion vector similarity S1 for the query data using OMHS is a histogram intersection method, which is expressed by Equation 3 below.

In Equation 3, n and m are one of OMSs extracted from a video sequence. Accordingly, H (f _n , i) are query object motion vector histograms of a specific scene (Shot), and H (f _m , i) are object motion vector histograms stored in units of scenes in the storage medium 114. Therefore, the larger the S1 value, the greater the similarity is detected (recognized) as the object motion scene.

In addition, a method of retrieving the object motion vector similarity (S2) and the object motion vector trajectory for the query data using the object motion average value is expressed by Equation 4 below.

In Equation 4, n and m are time intervals of the object motion trajectory or the average value of the object motion extracted from the video sequence. therefore, Is the query object motion trajectory or query object motion average value of a specific scene. Are object motion average values stored in the storage medium 114 in units of scenes. Therefore, the larger the S2 value, the greater the similarity is detected (recognized) as the object motion scene.

In this case, the present invention may be configured to search for an object motion similar to the query video sequence using either component of the OMHS and the object motion average value, or alternatively set to search for an object motion similar to the query video sequence using both components. It may be. The former has the advantage of obtaining similar object motion trajectories for faster search speeds or query object motion trajectories. The latter results in similar scene results for search or query scenes for more precise object motion than the former. It has the advantage to be obtained.

Next, the similarity S1 and / or S2 calculated through the process as described above is transferred to the video sequence retrieval block 118 together with the address information of the retrieved similar scene, where the object motion indexing is performed in the video sequence retrieval block 118. Similar object motion scene data corresponding to the query data is fetched from the storage medium 114 based on the similarity information and the address information provided from the search block 116 and transferred to the display side, not shown. At this time, assuming that a plurality of object motion scenes have been searched, the retrieved object motion scenes are retrieved from the highest similarity scenes to the lowest similarity scenes.

As a result, retrieved video sequence scenes similar to the query data input (or selected) input by the user will be sequentially displayed in the similarity order on the monitor not shown.

Therefore, according to the present invention, an object motion (or object motion) corresponding to a query video sequence or a query motion trajectory in an MPEG video sequence corresponding to the information of an MPEG video sequence stored on a storage medium using an object motion vector histogram is used. Scenes with high precision and speed.

Next, a process of constructing a database of MPEG video sequences using a content feature histogram based on object motion in the MPEG video sequence processing system having the above-described configuration will be described.

FIG. 5 is a flowchart illustrating a process of constructing a database of MPEG video sequences using object motion according to a preferred embodiment of the present invention.

Referring to FIG. 5, when MPEG video sequence data is input when the MPEG video sequence processing system is in a database building mode (step 502), the MV extraction block 104 of FIG. 1 uses a macroblock unit in the input MPEG video sequence. The motion vector MV is extracted (step 504).

In operation 506, the object motion vector from which the background motion vector information is removed is detected and clustered using the extracted motion vector MV. For example, an object motion vector is detected by comparing the motion vectors of corresponding positions between two frames (two temporally adjacent frames).

Next, in step 508, the object motion scene Shot is detected using the number of clustered object motions, which means a collection of scenes that are typically consistent in the MPEG video sequence. That is, a set of the same number of moving objects in the MPEG video sequence (that is, the same number of clustered objects) is defined as an object motion scene, and is called an OMS (Object Motion in Shot). Here, the information about the detected scene (Shot) is a basic unit of indexing and retrieval for storage.

Then, based on the detected object motion scene information and input frame information (one specific scene forming a scene), an identifier per detected object is assigned, that is, the content characteristics of the clustered motion object (i.e., color, texture, Shapes, etc.) are analyzed and given an identifier for each object. As an example, as shown in FIG. 3, the detected object motion vectors are divided into four sizes and eight directions and labeled into 32 groups.

Meanwhile, in step 510, the object motion histogram is analyzed by analyzing the object motions labeled for each frame in the OMS, and indexed by the OMHS and the average value of the object motion as expressed by Equations 1 and 2 described above. Here, OMHS is a digitization of the histogram itself of object motions of various sizes and directions, and the object motion average value is used as indexing vectors capable of efficiently detecting the trajectory of the object motion.

Accordingly, the indexing vectors of the moving object of the MPEG video sequence obtained through the above-described process are stored in a predetermined area of the storage medium 114 together with the MPEG video sequence data information corresponding thereto (step 512).

That is, in the MPEG video sequence database construction method according to the present invention, an indexing vector (OMHS and object motion mean value) obtained by analyzing each object motion histogram based on the object motion extracted from the input MPEG video sequence and indexing the analysis result Is stored in the storage medium as search key (or search index) information for searching for object movement in the MPEG video sequence.

Accordingly, the database construction method according to the present invention extracts object motion more quickly and efficiently by using motion information of an MPEG video sequence, and uses the object motion histogram drawn through this to search for a database. By extracting the indexing vector, unlike the foregoing patents, which require a large storage space and a large decoding time, the MPEG video sequence can be effectively databased without wasting large storage space and unnecessary time.

Next, in the MPEG video sequence processing system having the above-described configuration, a moving object is searched for in an MPEG video sequence corresponding to MPEG video sequence information stored in a storage medium using an object motion histogram based on object motions according to the present invention. Explain the process.

FIG. 6 is a flowchart illustrating a process of searching for an arbitrary query video sequence or an MPEG video sequence corresponding to a query motion trajectory in a database of an MPEG video sequence constructed using an object motion indexing vector according to the present invention. to be.

Referring to FIG. 6, as described above, the MPEG video sequence database is constructed according to the present invention, that is, the information about the MPEG video sequence images and the indexing vector for the object motion detected therein are stored. While performing the mode (step 602), the user selects a search mode, i.e. checks whether a query video sequence or query motion trajectory is input (step 604).

As a result of the check in step 604, if the query video sequence or the query motion trajectory is input, the MV extraction block 104 of FIG. 1 extracts a motion vector (MV) from the input query data (step 606). In operation 608, the query object motion vector from which the background motion vector information is removed is detected and clustered using the extracted query motion vector MV.

Next, in step 610, the query object motion scene is detected using the clustered query object motion count. Here, the detected query object motion scene is a basic unit for searching in the storage medium.

Meanwhile, in step 612, the query object motion histogram is analyzed by analyzing the query object movements labeled for each frame in the OMS, and indexed by the query OMHS and the query object motion average value as expressed by Equations 1 and 2 described above. .

Further, in step 614, the indexing vector (OMHS, object motion average value) of the MPEG video sequence stored in the storage medium 114 based on the query video sequence or the query indexing vector (query OMHS, query object motion average value) of the query motion trajectory. In order to search and compare, the scenes of the MPEG video sequence having similarity with the query video sequence or the query motion trajectory are sequentially searched.

Here, the search of the scene is performed by comparing the similarity through a comparison between the query indexing vector and the indexing vector, and the calculation of the similarity is performed by using Equations 3 and 4 described above. That is, the search for the object motion similarity with respect to the query data using OMHS is the same as Equation 3, and the search for the object motion similarity with respect to the query data using the average value of the object motion is performed as shown in Equation 4 above. same.

In this case, the present invention may be configured to search for an object motion similar to the query video sequence in the MPEG video sequence stored in the storage medium by using one component of the OMHS and the object motion average value. Alternatively, the query video sequence may be used by using both components. You can also set it to search for object movement similar to The former has the advantage of faster search speed and search results for similar object motion trajectories, while the latter has the advantage of more accurate object motion search and similar scene search results. Have

Next, in step 616, the object motion scenes of the retrieved object motion scenes are arranged in the order of high similarity, and the object motion scenes are sequentially retrieved (read) from the storage medium according to the arrangement order, and the object motion scene data thus retrieved is shown. It is delivered to the omitted display side. Therefore, on the monitor not shown, the object motion scenes searched in relation to the query video sequence or query motion trajectory selected by the user are sequentially displayed according to the similarity.

Accordingly, according to the present invention, an object motion histogram is used to precisely correct an object motion (or object motion scene) corresponding to a query video sequence or a query motion trajectory in an MPEG video sequence corresponding to MPEG video sequence information stored on a storage medium. And fast automatic search can be realized.

On the other hand, according to the database construction method of the present invention to build a database for MPEG video sequence using the object motion, and similar object motion in the database constructed by using the query object motion histogram and the average value of the object motion extracted from the query data A program or algorithm for searching for a scene can be recorded and used on a portable and portable recording medium (for example, CD), and the recording medium can be read using a device such as a computer.

As described above, according to the present invention, the object motion information is extracted more quickly and efficiently by using the motion information of the MPEG video sequence, and the object motion histogram and the average value of the object motion are drawn to search the database. Because of extracting the indexing vector for, it is possible to effectively database an MPEG video sequence without wasting large storage space and unnecessary time, unlike the foregoing patents, which require a large storage space and a large decoding time.

Further, using the query object motion histogram detected from the query data, searching for an object motion corresponding to the query data (query video sequence or query motion trajectory) in the MPEG video sequence corresponding to the information of the MPEG video sequence stored on the storage medium. Therefore, the search for the object motion information in the MPEG video sequence can be realized with high accuracy and speed.

Claims (15)

delete A method of constructing a searchable and readable database of object motion from which background motion is removed from MPEG video sequence data encoded at a predetermined compression rate, the method comprising: A first step of extracting a motion vector present in an MPEG video sequence input for storage; A second step of detecting a clustered object motion using object motion vector information after removing a background motion vector from the extracted motion vector; A third step of detecting an object motion vector scene by using the detected cluster motion number; A fourth step of analyzing a content feature histogram of the detected object motion cluster and assigning an identifier for each object; A fifth step of generating an object motion histogram using the detected object motion scene and the assigned identifier, and generating an indexing vector for the detected object motion scene by indexing the generated object motion histogram; And And a sixth step of storing the generated indexing vector along with corresponding input MPEG video sequence information in a storage medium. 3. The method of claim 2, wherein the content feature histogram is a color histogram of pixels corresponding to the object motion cluster. 3. The method of claim 2, wherein the content feature histogram is a texture histogram for pixels corresponding to the object motion cluster. 3. The method of claim 2, wherein the content feature histogram is a shape histogram of a pixel region corresponding to the object motion cluster. The method of claim 2, wherein the fifth process is: Grouping the detected object motions by region in a polar coordinate system; Generating the object motion histogram using the grouped object motion vectors; And Analyzing the generated object motion histogram to calculate an OMHS value numerically calculating the object motion histogram, and assigning the calculated OMHS and an average value of the object motion to the indexing vector. How to build a database of video sequences. The method of claim 6, wherein the OMHS is calculated by the following equation. OMHS = {H (f _t , i); 0≤t≤M, i = 1,2, ----, N} (In the above equation, f denotes a frame, t denotes a time interval or frame numbers of frames separated by a shot, i denotes a group label of an object motion vector OM, and H (f _t i) represents the number of object movements with group label i in a particular scene (by t interval) A method for retrieving object motion information corresponding to query data from a storage medium in which MPEG video sequence data encoded at a predetermined compression rate is stored together with an object motion indexing vector for retrieval. A first step of extracting a query motion vector from the input query data when the query data to be searched in the storage medium is input; A second process of removing a background motion vector from the extracted query motion vector and then detecting clustered query object motion; A third step of detecting a query object motion scene by using the detected number of clusters of the query object motion; A fourth step of analyzing a content feature histogram of the detected query object motion cluster and assigning an identifier to each query object; A fifth step of generating a query object motion histogram using the detected query object motion scene and the assigned identifier, and generating the generated query object motion histogram and the average value of the object motion as a query indexing vector; And A similar object motion scene image is searched through a comparison search between the generated query indexing vector and each indexing vector stored in the storage medium, and information about the retrieved similar object motion scene image is retrieved from the storage medium and corresponding MPEG is searched. A method for searching a database of an MPEG video sequence using an object movement consisting of a sixth step of delivering a video sequence to a display side. The method of claim 8, wherein the fifth process is: Grouping the detected query object motion by region in a polar coordinate system; Generating the query object motion histogram using the grouped query object motion; And Analyzing the generated query object motion histogram to calculate a query OMHS value digitized the query object motion histogram and assigning the calculated query OMHS value and the query object motion average value to the query indexing vector. A method for searching a database of MPEG video sequences using object movement. 10. The method of claim 9, wherein the query OMHS is calculated by the following equation. OMHS = {H (f _t , i); 0≤t≤M, i = 1,2, ----, N} (In the above equation, f denotes a frame, t denotes a time interval or frame numbers of frames separated by a shot, i denotes a group label of an object motion vector OM, and H (f _t i) represents the number of object movements with group label i in a particular scene (by t interval) The method of claim 8, wherein when the plurality of similar object motion scenes corresponding to the query data is searched, the data retrieval method sequentially retrieves the indexing vector among the plurality of similar object motion scenes searched in order of increasing similarity. The method of claim 1, further comprising the step of transmitting to the display side. 12. The method of claim 11, wherein the similarity is calculated by the following equation. Where n and m are one of the object motion scenes extracted from the video sequence, H (f _n , i) is a query object motion vector histogram of a specific scene, and H (f _m , i) is the storage medium. Object histogram vector stored in 12. The method of claim 11, wherein the similarity is calculated by the following equation. (In the above equation, n and m are time intervals of the object motion trajectory or the object motion average value extracted from the video sequence, Is the query object motion trajectory or query object motion average value of a specific scene. Is an average value of the object movement stored in the unit of scene in the storage medium) A program for constructing a database of object motions from which MPEG motion sequence data is removed from the MPEG video sequence data as a database on a retrievable and readable storage medium, and retrieving similar object motion scenes corresponding to the query data targeted for retrieval from the constructed database. A computer-readable recording medium for recording thereon, A first step of extracting a motion vector present in the input MPEG video sequence, removing a background motion vector from the extracted motion vector, and then detecting clustered object motion; A second step of detecting an object motion scene by using the detected number of clusters of object motions, analyzing a content feature histogram of the detected object motion clusters, and assigning an identifier for each object; A third process of generating an object motion histogram using the detected object motion scene and the assigned identifier, and generating an indexing vector for the detected object motion scene by indexing the generated object motion histogram and the average value of the object motion. ; A fourth step of storing the generated indexing vector in the storage medium together with information of a corresponding input MPEG video sequence; A fifth query for extracting a query motion vector from the input query data, removing a background motion vector from the extracted query motion vector, and detecting a clustered query object motion when the query data to be searched is input from the storage medium. process; A sixth step of detecting a query object motion scene by using the detected number of query object motion clusters, analyzing a content feature histogram of the detected query object motion clusters, and assigning an identifier for each query object; Generating a query indexing vector by generating a query object motion histogram using the detected query object motion scene and the assigned identifier, and indexing the generated query object motion histogram and an object motion average value; And The similar object motion scene image is searched through a comparison search between the generated query indexing vector and each indexing vector stored in the storage medium, and information corresponding to the retrieved similar object motion scene image is retrieved from the storage medium and corresponding thereto. The computer-readable recording medium having recorded thereon a program for executing an eighth process of delivering an MPEG sequence to a display side. 15. The method of claim 14, wherein the program recorded on the recording medium is arranged in order of similarity of the indexing vector among the plurality of similar object motion scenes searched when a plurality of similar object motion scenes corresponding to the query data is searched. And sequentially extracting the image to be transferred to the display side.