JP2008262560A - Video search method and video search device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a video information description method for allowing a video including an object whose background is moving to be searched for. <P>SOLUTION: A feature quantity 703 including information, such as a position, shape, and movement of an object and a feature quantity 704 including information such as a background movement, are described from an original video as notation data. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to a video information description method that focuses on an object in a video, and a video search method and video search device that use this to search for a specific object or a frame that includes the object.

  With the increase in broadcasting multi-channel accompanying the spread of digital satellite broadcasting and cable television, video information available to users is increasing. On the other hand, with the advancement of computer technology and the practical use of large-capacity recording media represented by DVD, it is becoming easy to store a large amount of video information as digital information and handle it on a computer.

  In order for a user to actually use video information, an effective video search technique is required to efficiently access a target video from such a large amount of video information. As such a video search technique, a method is considered in which some information is attached to an object in the video, and a video including an object satisfying the information required by the user is searched and used for viewing by the user. In order to attach information to an object in a video, a process for extracting the object from the video is required. However, it is not realistic to manually extract an object from the increasing video information.

  Regarding the automatic object detection technology, for example, literature (Yoneyama, Nakajima, Yanagihara, Kanno “Detection of Moving Objects from MPEG Video Streams”, Science Theory Vol. J81-D-II, No. 8, pp. 1776-1786 (1998-08) proposes a method for detecting an object from a still image of a background. However, this method assumes that the background is stationary, and it is difficult to detect an object when the background moves.

  That is, even if the shape of the object is given in advance, if the background movement is not known, when searching using the object movement, the search using the accurate movement is affected by the influence of camera work. Can not. For example, when shooting is performed so as to track an object moving to the left, the object is almost stationary in the screen, and the background moves relatively to the right. Therefore, it is impossible to search for a video including an object moving to the left in the screen.

  As described above, the conventional video search technique cannot detect an object whose background is moving, and cannot search a video including such an object.

  SUMMARY OF THE INVENTION An object of the present invention is to provide a video information description method for enabling a search for a video including an object whose background is moving.

  Another object of the present invention is to provide an object detection method capable of detecting an object whose background is moving.

  Another object of the present invention is to provide a video search method and a video search apparatus that enable various searches for videos containing objects whose background is moving.

  In order to solve the above-described problems, the video information description method according to the present invention has a basic feature of describing, as video information, a feature quantity related to a specific object in a video and a feature quantity related to a background in the video. .

  In addition, another video information description method according to the present invention describes a feature amount related to a specific object in a video and a feature amount related to a background in the video, and further describes a difference between both feature amounts as video information. Features. Further, the difference between the feature quantity related to a specific object in the video and the feature quantity related to the background in the video and the feature quantity related to the background may be described as video information.

  Here, it is desirable to describe at least information on the position, shape, and motion of the object as the feature amount relating to the specific object, and to describe at least information on the background motion as the feature amount relating to the background.

  In addition, according to the present invention, there is provided a recording medium that stores the feature quantity related to the object described in this way, the feature quantity related to the background, and the difference between the feature quantities together with the video data or separately from the video data. Is done.

  The object detection method according to the present invention includes a motion vector extraction step of extracting a motion vector of an input video, an estimation step of estimating a motion of a background of the video using the extracted motion vector, and the estimated background And a detection step of extracting a motion vector relating to a specific object in the video by removing the motion and detecting a region of the object.

  Here, in the estimation step for estimating the background motion, the background motion is approximated by a predetermined conversion model (for example, affine transformation or perspective transformation), and the conversion coefficient of the conversion model is estimated from the motion vector of the video. The background motion is estimated by The conversion coefficient of the conversion model is estimated by a robust estimation method, for example.

  In the estimation step of estimating the background motion, the motion vector in the video screen is divided into regions according to the similarity, the divided regions are clustered based on the similarity of the motion vectors, and the region of the largest cluster is determined. Processing for determining a background region may be included.

  Further, in the estimation step for estimating the background motion, the motion vector of a plurality of frames of the video is divided into regions according to the similarity in each frame, and these regions are associated between the frames, and the associated regions are the same. Processing may be included in which each region is clustered within each frame so as to belong to the cluster, and the largest cluster region is determined as the background region.

  According to the present invention, it is possible to perform various searches for videos including objects whose background is moving by describing the feature quantity related to the object in the video, the feature quantity related to the background, and the difference between the feature quantities of both. It is.

  First, a feature quantity related to a specific object in a video to be searched and a feature quantity related to a background in the video are described, and a feature quantity related to a background is subtracted from a feature quantity related to an object in the video to be searched. After that, by comparing with the feature amount related to the object input from the outside, the object in the image including the same object as the object input from the outside from the image to be searched or the same object as the object input from the outside At least one of the frames can be searched.

  Second, a feature amount related to a specific object in the video to be searched, a feature amount related to the background in the video, and a difference between the feature amounts are described, and this difference and a feature related to the object input from the outside are described. By comparing the amount, at least one of the frames in the video including the same object as the object input from the outside or the same object as the object input from the outside can be searched from the video to be searched. .

  Third, a feature amount related to a specific object in the video to be searched and a feature amount related to the background in the video are described, and a feature amount related to the background and a feature amount related to the background in the video input from the outside, By comparing these two, it is possible to search for a frame in which almost the same camera work is used as the camera work when the externally input video is obtained from the video to be searched.

 Fourth, a feature amount including at least motion information about a specific object in the video to be searched and a feature amount related to the background in the video are described, and the motion information of the object in a plurality of consecutive frames in the video And a frame including the same object as the object input from the outside or the same object as the object input from the outside by comparing a series of movement information of the object input from the outside At least one of them can be searched.

  According to the present invention, by describing the feature quantity related to the object and the feature quantity related to the background, it is possible to search based on the original motion of the object by removing the motion of the background.

  In addition, it automatically detects an object without relying on human resources for a large amount of stored video data, extracts its feature value, searches for an object that matches another feature value input from the outside, It is possible to easily perform a video search suitable for each user's purpose, such as searching for a frame that includes the same background movement as that of the camera movement input from the outside.

  Further, by describing the feature quantity detected in advance, it is not necessary to perform feature quantity extraction processing every time it is searched, and high-speed search is possible, and even when the user does not have an object detection function, Search is possible.

Embodiments of the present invention will be described below with reference to the drawings.
[First Embodiment]
In the present embodiment, three functions are roughly classified. First, in addition to the function of reproducing video data, a function of automatically detecting an object with motion in the video and notifying the user of its presence by overlaying and displaying figures such as ellipses and rectangles I will provide a.

  Second, it provides a function that separates feature quantities such as the position, size, and movement of the detected object and feature quantities related to the background and describes them as notation data in an external file or the like.

  Third, the feature quantity data relating to the detected object or the feature quantity notation data described in advance in an external file or the like is compared with the feature quantity data of the search target object given from the outside as the search target. By presenting an object to be performed to the user, a function for searching for an object in the video is provided.

  FIG. 1 is a flowchart showing the configuration and processing procedure of the video search system according to this embodiment.

  First, original video data 100 reproduced from a medium such as a DVD is input (step 101), and a specific object in the video is detected from the original video data 101 by a method described in detail later (step 102). At this time, as will be described later, information regarding the background in the video is also detected. The detected object is combined with a figure such as an ellipse or rectangle generated so as to surround it, and is output as object detection result display data 104 (step 103).

  On the other hand, the feature amount data indicating the feature amount relating to the object such as the position, shape (including size) and movement of the object detected in step 102 and the feature amount data indicating the feature amount relating to the background such as the background motion are described. Feature data generation processing is performed (step 105), and the generated feature data 107 relating to the object and background is output to the outside and described as notation data (step 106).

  In steps 105 and 106, the feature quantity data relating to the object and the feature quantity data relating to the background may be described. However, the difference data between the features of the two may be generated and described. Difference data and feature amount data relating to the background, or difference data and feature amount data relating to the object may be generated and described.

  The description process in step 106 specifically means storing (recording) or displaying the feature data 107 in various recording media or memories. The recording medium for storing the feature amount data 107 may be a medium such as a DVD in which the original video data 100 is stored, or may be a recording medium different from this.

  Next, in order to search for an object, similarity determination is performed between the feature amount data related to the object generated in step 105 and the search target feature amount data 110 input in step 109 (step 108), and further similarities are determined. A composite display process for compositely displaying the degree determination result as the object search result display data 112 is performed (step 111). The search target feature amount data 110 is data representing feature amounts such as the position, shape (including size), and movement of an object to be searched.

  The series of processing can be realized by either software or hardware.

Next, the object detection process in step 102 in FIG. 1 will be described in detail with reference to FIG.
First, a motion vector is extracted from the input original video data 100 (step 201). When the original video data 100 is MPEG compressed data, a motion vector obtained from the P picture is used. In this case, a motion vector is given for each macroblock. If the original video data 100 is analog data or digital data that does not have a motion vector, it is digitized as necessary, and the motion vector is extracted using an optical flow or converted into MPEG compressed data and then extracted. To do.

  The motion vector extracted in this way may not necessarily reflect the actual motion of the object, and this is conspicuous in the peripheral part of the screen and the part where the texture is flat. Therefore, a process of removing a motion vector with low reliability is performed (step 202). This process is performed as follows.

  First, regarding the peripheral part of the screen, an area is determined in advance, and a motion vector included in the area is removed. On the other hand, for the portion having a flat texture, when the original video data 100 is MPEG compressed data, the DC component of the DCT (discrete cosine transform) coefficient of the I picture is used as shown in FIG. A collection of macroblocks whose variance of four DC components included in one macroblock is equal to or less than a threshold value is defined as a low reliability region, and a motion vector included in the macroblock in that region is removed as a motion vector having low reliability. .

  Since the motion vector data obtained in this way includes background motion due to camerawork etc. in the motion of the object, it is necessary to remove the background motion in order to obtain accurate object motion. is there. Therefore, in the present embodiment, a process for estimating the background motion is performed by using an affine transformation model as a transformation model that approximates the background motion due to camerawork or the like, and estimating the transformation coefficient using a motion vector (step) 203). There are several methods for estimating the affine transformation coefficient of the background motion, which will be described later.

  Next, the starting point of each motion vector is converted using the estimated affine transformation coefficient, and the background motion is removed by subtracting the amount of movement from the original motion vector (step 204).

  A process of dividing the motion vector data not including the background motion obtained in this way into regions composed of similar motion vectors is performed (step 205). Specifically, the cosines (directions) of two adjacent moving vectors are compared with the magnitude, and if the difference is equal to or less than a predetermined threshold, the process of dividing the same region into all adjacent moving vectors is performed. Do the combination.

  Since the area obtained in this way includes small areas that are not suitable for handling as objects, a determination process 306 is performed to remove these by threshold processing, and final object data 307 is output.

  Hereinafter, three methods of the affine transformation coefficient estimation processing in step 203 for approximating the background motion from the motion vector will be described.

<Method 1>
In Method 1, the affine transformation coefficient is estimated using all the motion vectors in the screen excluding the motion vectors with low reliability. The center of the i-th macroblock is y _i and the motion vector corresponding to the macroblock is v _i . At this time, error between the starting point x _{i =} y _i -v _i destination by affine transformation deformation model of, r _{i =} x _i α becomes when the affine transformation coefficient alpha, the actual destination y _i of the vector is the _{e i =} r i -y i. The sum of the estimated residuals is given by the following equation, and α that minimizes this may be obtained.

There is a least square method as a method for solving such a problem, and in that case, Ψ (z) = z ² may be used in the equation (1). However, when the least squares method is used, the background motion vector and the object motion vector are handled in the same column, so the affine transformation coefficient cannot be estimated from the background motion vector alone, and the affine transformation including the object motion is included. It becomes a coefficient.

Thus, assuming that the area occupied by the background region in the screen is 50% or more, the object's motion vector is regarded as a disturbance, and the affine transformation coefficient is estimated only from the motion vector of the background portion. A robust estimation method as disclosed in the literature: Toru Nakagawa and Yoshio Koyanagi, “Experimental Data Analysis by Least Squares Method”, Sakkyo University Press, is used as a robust technique for disturbance. Here, in particular, M estimation based on the Biweight method, which is one of robust estimation methods, is used. In the Biweight method, the weight of an element with a large error is lowered to make it less susceptible to disturbance. Specifically, the following equation (2) using the weight w is used in Ψ (z) of the previous equation (1). The constant c should be selected from 5 to 9.

<Method 2>
A processing procedure when Method 2 is used in the affine transformation coefficient estimation processing that approximates the motion of the background will be described with reference to the flowchart shown in FIG.

  First, for the motion vector data 500 after performing the process of removing the low-reliability vector, the process of dividing the adjacent motion vector into similar regions using the same process as in the previous method 1. (Step 501). However, unlike the method 1, the process of removing the background motion is not performed at this point.

  Next, affine transformation coefficient estimation processing is performed when the motion of the region is approximated by an affine transformation model from the motion vector included in each divided region (step 502). For the estimation process at this time, a robust estimation method similar to the method 1 is used.

  Next, clustering processing is performed on each divided area (step 503). For this purpose, a table comprising combinations of all areas is prepared, and the distance between the areas is obtained from the affine transformation coefficients. Here, the Euclidean distance of 6 coefficients of the affine transformation model is used, but other distances may be used. Next, the two regions having the smallest distance are integrated, a new affine transformation coefficient is obtained for the integrated region, the two integrated regions are deleted from the table, the integrated region is added, Update the table. This process is repeated until the distance between the areas becomes greater than a predetermined threshold or there is one area.

  Of the areas clustered in this way, a process of determining the area with the largest cluster as the background area is performed (step 504), and the affine transformation coefficient of the area is output as the affine transformation coefficient 505 of the background motion.

<Method 3>
A processing procedure when Method 3 is used in the affine transformation coefficient estimation processing for approximating the background motion will be described with reference to the flowchart shown in FIG.

  First, a plurality of frames are read at a time, and processing for dividing each frame into regions where adjacent motion vectors are similar is performed using the same processing as in method 2 (step 601).

  Next, a conversion coefficient estimation process is performed when the motion of each area is approximated by an affine transformation model (step 602). Further, based on the position of the area, motion vector data, and the conversion coefficient, a corresponding area between frames is determined. After performing the required processing (step 603), the regions in each frame are clustered by the same clustering processing as in method 2 (step 604).

  When the region associated in the inter-frame association processing is clustered in another cluster, the result clustered in the largest number of clusters is taken as the correct answer, and correction processing is performed to move the region clustered in another cluster (Step 605).

  Finally, a determination process is performed in which a region having the largest area among a plurality of frames is used as a background (step 606), and a conversion coefficient 507 for the background region in each frame is obtained. This method 3 has an advantage that it can be correctly estimated even when the background area temporarily becomes smaller than other areas in a specific frame.

  In the above example, the affine transformation is used as the transformation model used for the process of estimating the background motion, but other transformation models such as perspective transformation may be used.

  Next, data representation used in the description processing of the feature amount data relating to the object and the background in step 106 in FIG. 1 will be described with reference to FIG. Here, as shown in FIG. 7A, for example, the notation data 700 of three objects included in the video 705 in the 1000th frame is shown. This notation data 700 is composed of frame information 701 indicating the corresponding frame in the video stream 706 of the original video data, the feature amount 703 relating to the object, and the feature amount 704 relating to the background, and a pointer 702 to the next notation data is provided. It is managed by the list structure used.

  The feature quantity 703 relating to the object includes at least information on the position, shape (including size) and movement of the object, and specifically includes various feature quantities as exemplified in FIG. 7B, for example. Is done. In this example, the feature quantity 703 regarding the object is composed of “position”, “rough shape” as shape, “affine transformation coefficient” and “average and direction of motion vector” as motion information, and “color histogram”. Is done.

  Here, the outline of the object may be approximated by a simple figure such as an ellipse or a rectangle. The affine transformation coefficient is a coefficient estimated when the motion of the object is approximated by an affine transformation model as described above. The average of the motion vectors is an average value of the sizes of the motion vectors in the object. When the color information of the object can be acquired, the color histogram of the object area can be used as the feature amount. As for the movement of the object, either the background movement removed or not removed may be recorded.

  When there are a plurality of objects as in this example, it is desirable to assign an individual ID number to each object feature quantity 703 and manage it by a list structure as shown in FIG. By using such a list structure, it is easy to add or delete object feature amounts.

  As for the feature quantity 704 related to the background, various feature quantities as exemplified in FIG. 7C, for example, “affine transformation coefficient”, “average of moving vector, direction”, “camera work”, like the feature quantity 703 related to the object. It consists of “species”, “color histogram” and the like. The camera work type is a typical type of camera work used for shooting such as a van or zoom.

Next, the similarity determination process in step 108 of FIG. 1 will be described using the flowchart shown in FIG.
This similarity determination processing is performed by sequentially comparing the feature amount data 800 regarding each object included in the original video data with the feature amount data 804 input from the outside. The feature amount data 804 input from the outside may be directly given as numerical data or the like, or may be extracted from the video and given as feature amount data.

  When the object has a plurality of types of feature amounts, the similarity is obtained sequentially by the similarity determination process for each feature amount (step 803).

  Comparison between the feature data 800 included in the original video data 800 and the feature data 804 input from the outside uses an appropriate method based on the type of the feature. For example, if the feature quantity is a color histogram, it is conceivable to use differences between the elements of the histogram. When the objects to be compared have different types of feature amounts, only the matching feature amounts need be compared.

  If it is determined in steps 801 and 802 that the search has been completed for all the feature amount data for all objects, search result display processing is performed for the information of the corresponding object (step 805), and the processing ends.

  When comparing the movements of objects, it is possible to remove the movement of the background using the feature amount data related to the background and compare them. With reference to FIG. 9, the effect of the search with the background motion separated will be described.

  As shown in FIG. 9, the original video data 901 is taken while moving the camera so as to track the object moving to the left, but in the video, the object appears to be stationary and the background moves to the right. Looks like. In order to search for an object that moves to the left, when data of the object 905 is input from the outside, the object of the video data 901 is stationary, so the feature amounts do not match and cannot be searched.

  However, when the feature quantity related to the object and the feature quantity related to the background are described according to the present invention, the object with the original leftward movement is obtained by the process 902 for separating the background 904 moving by camerawork using the movement of the background 903 can be detected. That is, in the process 902, only the object 903 is detected by obtaining the difference between the feature quantity related to the object and the feature quantity related to the background.

  Therefore, by comparing the detected object 902 with an object 905 input from the outside, the same object 905 as the object input from the outside can be searched from the input video data 901, or the object 905 input from the outside can be searched. A video frame including the same object can be searched from the original video data 901. In this case, if the difference data is described as described above, the processing 902 becomes unnecessary.

  In addition, when the feature quantity related to the object and the feature quantity related to the background are described according to the present invention, it is possible to search for a camera work image that matches the camera work input from the outside as shown in FIG. That is, as shown in FIG. 10, only a background 1004 that moves due to camera work is detected by processing 1002 that separates the object 1003 from the original video data 1001. Then, by comparing the detected background 1004 with the background 1005 that is moved by the camera work input from the outside, a video frame using the same camera work as the camera work input from the outside is obtained from the original video data 1001. Search for.

  Also in this case, if the difference data is described as described above, the processing 1002 becomes unnecessary.

[Second Embodiment]
Next, a second embodiment of the present invention will be described with reference to the flowchart shown in FIG.
In this embodiment, instead of detecting and describing an object in the first embodiment, original video data 1100 to which feature amount data analyzed in advance is added is input (step 1101), and this original video data 1100 is input. The feature amount data relating to the object is separated and extracted from the object (step 1102).

  Subsequently, similar to the first embodiment, similarity determination processing is performed between the feature amount data related to the original video data 1100 extracted in step 1102 and the search target feature amount data 1110 input in step 1109 (step 1108). ), A combined display process for combining and displaying the result as object search result display data 1112 is performed (step 1111).

  The series of processing can be realized by either software or hardware.

[Third Embodiment]
Next, a third embodiment of the present invention will be described using the flowchart shown in FIG.
In this embodiment, in order to compare a series of movements input from the outside with notation data over a plurality of frames and to search for objects based on time-series movements, they are included in a plurality of continuous notation data 1201. A process of associating the same object among the objects is performed (step 1202). On the other hand, sampling processing for extracting motion data from the motion data 1203 input from the outside at the same interval as the notation data 1201 is performed (step 1204).

  Then, the notation data corresponding to each other and the sampled movement data of the external input are compared (step 1105), and the video including the matching object is displayed as the search result (step 1106).

With reference to FIG. 13, the association processing of the objects included in the continuous notation data 1201 in step 1202 of FIG. 12 will be described.
A predicted position 1302 of the object in the (N + 1) th notation data is obtained using the feature amount (position and motion) related to the object 1301 included in the Nth notation data. Then, an object 1303 included in the (N + 1) th notation data present at the position closest to the predicted position 1302 is set as an object corresponding to the object 1301.

The sampling process of the motion data 1203 input from the outside in step 1204 of FIG. 12 will be described using FIG.
Since motion data 1401 (same as 1203) input from the outside is continuous motion data, it cannot be compared with notation data which is discrete data added every few frames. Therefore, the motion data 1401 is sampled at the frame interval of the notation data, and the sampled motion data 1402 is compared with the notation data.

  The series of processing can be realized by either software or hardware.

The flowchart which shows the basic process sequence of the video search system which concerns on the 1st Embodiment of this invention. A flowchart showing a processing procedure of object detection in the embodiment The figure which shows the relationship between the I picture and P picture of the MPEG stream used for the object detection in the embodiment The figure for demonstrating removal of the low reliability spectrum in the object detection in the embodiment Flowchart for explaining a method for obtaining a conversion coefficient of a background region in the embodiment Flowchart for explaining another method for obtaining the conversion coefficient of the background region in the embodiment The figure which shows the structure of the feature-value data used by the object description process in the embodiment A flowchart showing a processing procedure of object search in the embodiment The figure shown about the removal of the camera work in the object search process in the embodiment The figure which shows about the search of the flame | frame in which the same camera work as the input camera work is used in the object search process in the embodiment The flowchart which shows the basic process sequence of the image | video search system which concerns on the 2nd Embodiment of this invention. The flowchart which shows the basic process sequence of the video search system which concerns on the 3rd Embodiment of this invention. The figure for demonstrating matching of the object between the continuous notation data in the embodiment The figure for demonstrating the sampling method of the motion data input from the outside in the embodiment

Explanation of symbols

700 ... notation data 703 ... feature quantity related to object 704 ... feature quantity related to background 706 ... video stream

Claims (8)

Describe the feature quantity related to a specific object in the video to be searched and the feature quantity related to the background in the video,
After subtracting the feature quantity relating to the background from the feature quantity relating to the object in the video, and comparing it with the feature quantity relating to the object input from outside, the same as the object input from the outside from the video to be searched Or at least one of the frames in the video including the same object as the object input from the outside or the object input from the outside. Describe the feature quantity related to a specific object in the video to be searched, the feature quantity related to the background in the video, and the difference between each feature quantity,
By comparing the difference and the feature quantity related to the object input from the outside, the same object as the object input from the outside or the same object as the object input from the outside from the video to be searched A video search method comprising searching for at least one of the frames in the video. Describe the feature quantity related to a specific object in the video to be searched and the feature quantity related to the background in the video,
By comparing the feature quantity related to the background with the feature quantity related to the background in the video inputted from the outside, it is almost the same as the camera work when the video inputted from the outside is obtained from the video to be searched. A video search method characterized by searching for a frame in which camera work is used. Describe a feature amount including at least information about movement of a specific object in a video to be searched and a feature amount related to a background in the video,
By comparing the motion information of the object in a plurality of consecutive frames in the video with a series of motion information of the object input from the outside, the same object as the object input from the outside from the video to be searched A video search method comprising searching for at least one of an object or a frame including the same object as the object input from the outside. Describe the feature quantity related to a specific object in the video to be searched and the feature quantity related to the background in the video,
After subtracting the feature quantity relating to the background from the feature quantity relating to the object in the video, and comparing it with the feature quantity relating to the object input from outside, the same as the object input from the outside from the video to be searched Or at least one of the frames in the video including the same object as the object input from the outside or the object input from the outside. Describe the feature quantity related to a specific object in the video to be searched, the feature quantity related to the background in the video, and the difference between each feature quantity,
By comparing the difference and the feature quantity related to the object input from the outside, the same object as the object input from the outside or the same object as the object input from the outside from the video to be searched A video search apparatus for searching for at least one of the frames in the video. Describe the feature quantity related to a specific object in the video to be searched and the feature quantity related to the background in the video,
By comparing the feature quantity related to the background with the feature quantity related to the background in the video inputted from the outside, it is almost the same as the camera work when the video inputted from the outside is obtained from the video to be searched. An image retrieval apparatus for retrieving a frame in which camera work is used. Describe a feature amount including at least information about movement of a specific object in a video to be searched and a feature amount related to a background in the video,
By comparing the motion information of the object in a plurality of consecutive frames in the video with a series of motion information of the object input from the outside, the same object as the object input from the outside from the video to be searched A video search apparatus for searching for at least one of an object or a frame including the same object as the object input from the outside.

Images