JP2006244074A - Moving object close-up frame detection method and program, storage medium storing program, moving object close-up shot detection method, moving object close-up frame or shot detection method and program, and storage medium storing program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a moving object close-up frame detection method that can accurately detect the video time of moving object close-up frames imaging a moving object in close-up, and moving object close-up shots, irrespective of the type of the subject. <P>SOLUTION: Whether motion vectors of input video conform to a predetermined camerawork model is determined. If they do not conform, the video is detected as a moving object close-up frame. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to an animal up-frame detection method, a program, and a storage medium storing the program, and an animal up-shot detection method, an animal up-frame or shot detection method, a program, and a storage medium storing the program. A storage medium storing a moving object up-frame detection method, a program, and a program for generating a representative image of a video in a structured technology, a moving object up-shot detection method, a moving object up-frame detection method, a program, and a program Related to the storage medium.

  As a first conventional technique, there is a video overview generation technique for detecting a video change point (cut point) caused by editing or turning on / off a camera and presenting the image to a user (for example, Non-Patent Document 1). reference).

As a second conventional technique, an object to be detected is learned in advance, the color feature amount of the object is stored, and the feature amount and the feature amount in the video are matched to thereby obtain a specific object. There is a technique for detecting an image area in which there is. This technique can calculate the size and position of an object since the area is known (see, for example, Patent Document 1). In addition to this, there are various techniques for finding similar features of color features and shape features stored in advance in a video and presenting them as objects of interest.
Taniguchi, Tonomura, Hamada “Video Shot Switching Detection Method and Its Application to Video Access Interface” IEICE Transactions, D-II Vol.J79-D-II No.4. Pp.538-546 Apr. 1996 Japanese Patent Laid-Open No. 10-247246

  However, the conventional technique has the following problems.

As issues related to the first prior art,
(A) In a video such as a television program, the cut point image is very large for one video and is difficult to see. Also, it is not effective for videos with few cut points, such as personal shot images.

  (B) In addition, there is a case where no significant image content is shown in the cut point image.

In addition, as a problem related to the second prior art,
(C) Only specific objects designated by the user or stored in advance can be detected.

  (D) The color feature amount and shape feature of an object are likely to change in appearance due to the movement of the object at the time of shooting, and may not be detected with those feature amounts. Moreover, in order to cope with the change in appearance, it is necessary to prepare a large number of templates, and the calculation cost for comparing them brute force is relatively high.

  The present invention has been made in view of the above points, and can accurately detect the video time of an up-frame of a moving object in which the moving object is large regardless of the type of subject, and can be used for providing a video overview and a video digest. Storage medium storing moving object up-frame detection method, program and program, moving object up-shot detection method, moving object up-frame or shot detection method, program and program capable of providing possible index The purpose is to provide a medium.

  FIG. 1 is a diagram for explaining the principle of the present invention.

The present invention (Claim 1) is directed to an animal in an up-frame detection apparatus for moving objects that detects a video frame image (moving body up-frame image) in which a moving object is relatively large from a video storage means in which video is stored. In the body up frame detection method,
A motion vector calculation procedure (step 1) for calculating a motion vector using a plurality of frame images read from the video storage means and storing the motion vector in a memory;
A camera work model conformity determination procedure (step 2) for acquiring a motion vector from a memory and determining whether the entire motion vector conforms to a predetermined camera work model;
When it is determined in the camera work model conformity determination procedure that the camera work model does not conform to the camera work model, a procedure (step 3) of detecting as a moving object up frame is performed.

  In claim 1 above, by calculating a motion vector and determining whether or not the motion vector conforms to the camera work model, whether or not the moving object that does not conform to the camera work model is greatly reflected Can be determined. That is, when the camera work model is, for example, a parallel movement model, the movement of the moving object on the video image is often not in accordance with the movement model. In particular, when a moving object is large, the movement that appears on the object is easily affected by the size of the object in the depth direction, and the movement does not conform to the camera work model. The present invention can detect an animal up-frame by detecting this based on the degree of motion vector variation.

  Further, the present invention (Claim 2) performs a process of detecting a cut point of the frame image read from the video storage means before the motion vector calculation procedure (Step 1), and the cut point is not detected. In this case, a cut point detection procedure for shifting to the motion vector calculation procedure is further performed.

  In the second aspect of the present invention, in order to solve the problem that a motion vector that does not conform to the camera work model is generated even at the cut point and the up-shot of the moving object is erroneously detected at the cut point, the cut point is detected by an existing method. By not detecting the moving object up frame before and after the cut point, it is possible to suppress erroneous detection and improve the detection accuracy of the moving object up frame.

Further, the present invention (Claim 3) is a camera work model conformity determination procedure (Step 2),
A camera work parameter calculation procedure for obtaining a motion vector from a memory, calculating a camera work parameter and storing it in a parameter storage means;
The camera work parameter is determined from the parameter storage means, and a camera work parameter abnormality determination procedure for determining whether the camera work parameter is abnormal is performed.

  According to the third aspect of the present invention, the determination as to whether or not the camera work model conforms is made by calculating the camera work parameter representing the camera work model and determining whether or not the camera work model does not comply with the camera work model. Thus, the moving object up frame can be detected.

In the present invention (Claim 4), when it is determined to be normal in the camera work model conformity determination procedure (Step 2),
Of the frame image units used when the camera work parameters are acquired from the parameter storage means and the motion vector is calculated in the motion vector calculation procedure, the unit of the frame image that does not conform to the camera work model is the motion of the image unit. And the camera parameter is calculated using the ratio of the number of image units or the weighted sum value to detect the animal up frame, and the video time of the animal up frame is stored in the up frame storage means. The moving body up frame determination procedure is performed.

  According to the fourth aspect of the present invention, in accordance with the camera work model, when the camera work parameter can be calculated by the above method, the motion of the image unit for calculating the motion vector is detected, and the calculated camera work parameter is calculated. The up-frame of the moving object can be detected by calculating whether the ratio of the number of image units having a large deviation is large. This is because an image unit with a large deviation is considered to be an image unit that is a part of a moving object, and it can be detected whether the image unit is up in the proportion of the screen. The proportion of the image on the screen may be given as a weight for each image unit, and may be measured by the sum or by the number.

The present invention (Claim 5) is a moving object up frame detection program for detecting a video frame image (moving body up frame image) in which a moving object is relatively large from a video storage means in which the video is stored. ,
A program for causing a computer having a plurality of memories, up-frame time storage means, and parameter storage means to execute processing for realizing the moving object up-frame detection method according to claim 1.

The present invention (Claim 6) stores a moving object up-frame detection program for detecting a video frame image (moving object up-frame image) in which a moving object is relatively large from a video storage means in which the video is stored. A storage medium,
A storage medium storing a program for causing a computer having a plurality of memories, an up-frame time storage unit, and a parameter storage unit to execute processing for realizing the moving-body up-frame detection method according to claim 1. .

The present invention (Claim 7) is a moving object up-shot in a moving object up-shot detection device for detecting a video section (moving object up-shot) in which a moving object is shown up from a video storage means in which video is stored. In the detection method,
5. An image for calculating a temporal unit by further accumulating the detected moving object up frame in the storage means in time series in the procedure of the moving object up frame detecting method according to claim 1. Add section calculation procedure.

  In the above-mentioned claim 7, by collecting the detected moving object up-frames in the video section, it is possible to create a video section in which the moving object is greatly reflected, and use an index for browsing the digest video section. It can also be provided to a person.

According to the present invention (Claim 8), a frame in which a moving object is shown up from a video storage means in which a video is stored, or a moving object up-frame or shot detection for detecting a video section (moving object up-shot) In the moving object up frame or shot detection method in the apparatus,
A plurality of frames used for a motion vector calculation procedure in the procedure of the moving object up-frame detection method according to claim 1 or the procedure of the moving object up-shot detection method according to claim 7. A video input update procedure for selecting using the calculated movement is added.

  According to the eighth aspect of the present invention, the processing frame interval for detecting the up frame is made variable by using an index such as the size of the moving object or the compatibility of the image, thereby improving the up frame detection accuracy and the processing speed. Can be expected.

The present invention (Claim 9) is a moving body upshot detection program for detecting a video section (moving body upshot) in which a moving body is captured from video storage means in which video is stored,
A program for causing a computer having a plurality of memories, an up frame time storage unit, and a parameter storage unit to execute a process for realizing the moving object up-shot detection method according to claim 7 or 8.

The present invention (Claim 10) is a storage medium that stores a moving object upshot detection program for detecting a video section (moving body upshot) in which a moving object is captured from a video storage means in which a video is stored. There,
A storage medium storing a program for causing a computer having a plurality of memories, an up-frame time storage unit, and a parameter storage unit to execute processing for realizing the moving object up-shot detection method according to claim 7 or 8.

  As described above, in the present invention, it is considered that a moving object (animal body) in an image is an important object that is viewed by a viewer, and an image frame (animated body up) in which the moving object is relatively large is shown. The video time (or video section) of the frame image) is automatically detected, the video is acquired based on the video time, and a list can be presented to the user.

  In the present invention, not all shots are presented, but only shots in which an animal up frame is detected (animal up shot), and about one animal up frame image is presented for each shot. Thus, the number of frame images to be presented can be reduced.

  In addition, even in the case of a video having almost no cut points, it is possible to combine them into sections according to the time interval of the detected animal up frame, etc., and present any one representative video for each section. Can do.

  As a result, the above-mentioned problem (a) is solved.

  In addition, since the presence of a moving object in the video is detected by movement, it is guaranteed that the moving object is present in the detection frame. Thereby, the above-mentioned problem (b) is solved.

  Moreover, the above-mentioned problem (c) is solved by performing the moving body up-frame detection which does not specify an object by motion information.

  Furthermore, since the moving body up-frame image is detected by using the motion information, the moving body up-frame can be detected robustly to the change in the shape and color of the appearance of the moving body. Solved.

  According to the present invention, it is possible to accurately detect the video time of a moving object up frame in which a moving object is greatly shown regardless of the type of subject.

  In addition, it is possible to collect moving object up frames as video sections and use them as an index. That is, it is possible to provide a video overview and an index that can be used when skipping video.

  In addition, as a pre-processing for detecting and recognizing a specific object with existing techniques such as object detection and recognition with relatively high calculation cost, the animal body up-frame detection of the present invention is performed, and the detected animal By detecting and recognizing only the body up frame, it is possible to use a method of reducing the calculation time for specifying the subject.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

  FIG. 2 is an apparatus configuration diagram according to an embodiment of the present invention.

  The detection apparatus shown in FIG. 1 includes a video storage unit 10, a video input unit 20, a cut point detection unit 30, a motion vector calculation unit 40, a camera work model conformity determination unit 50, a video section calculation unit 60, a parameter storage list 70, and an upload. It consists of a frame time storage list 80.

  It is assumed that the video storage unit 10, the parameter storage list 70, and the uplink time storage list 80 are stored in a storage medium such as a disk device.

  The camerawork model conformity determination unit 50 includes a camerawork parameter calculation unit 51, a camerawork parameter abnormality determination unit 52, and a moving object upframe determination unit 53.

  The video input unit 20 reads video data from the video storage unit 10 and outputs a frame image at a designated time. When time information (video time) is input from an operator via an input device (not shown), a frame image corresponding to the video time can be acquired from the video storage unit 10. Further, the video input unit 20 designates the current reference frame time as the next processing target frame time, designates the time of the next reference frame, and sends the video data from the video storage unit 10 as a processing target frame image (one frame). ), And read out as a reference frame image (multiple images are possible).

  When a frame image at a plurality of times and its time information are input from the video input unit 20, the cut point detection unit 30 detects a cut point included therein and stores the video time at which the cut point exists in the memory (see FIG. (Not shown). When there is no cut point, a time that does not actually exist is output, for example, a negative time is output. Details of the operation will be described in the second embodiment.

  When a frame image at a plurality of times is input from the video input unit 20, the motion vector calculation unit 40 calculates a motion vector between two predetermined frame images and uses it until the camerawork model conformity determination unit 50 uses it. The motion vector is stored in the memory. Details of the operation will be described in the first embodiment.

  The camera work model conformity determination unit 50 includes a camera work parameter calculation unit 51 that calculates camera work parameters, a camera work parameter abnormality determination unit 52 that determines whether the calculated camera work parameters are abnormal, and the camera work parameters are normal. In this case, it further includes a moving object up frame determination unit 53 that determines whether the moving object is an up frame.

  The camera work parameter calculation unit 51 calculates camera work parameters from the motion vectors calculated by the motion vector calculation unit 40 and stores them in the parameter storage list 70. Details of the operation will be described in the first embodiment.

  The camera work parameter abnormality determination unit 52 acquires the camera work parameter calculated by the camera work parameter calculation unit 51 and the camera work parameter calculated from the parameter storage unit 70, and follows a predetermined camera work model. It is determined whether or not. Details of the operation will be described in the first embodiment.

  The moving object up frame determination unit 53 determines whether the camera work parameter abnormality determination unit 52 determines that the camera work parameter conforms to the camera work model is a moving object up frame, and the moving object up frame is detected. If it is, the video time and the like are output. Details of the operation will be described in a third embodiment.

  When the up-frame time of the moving object is input from the moving object up frame determination unit 53, the video section calculation unit 60 outputs the time of the start point and end point of the section together in the video section. The output of the video section calculation unit 60 is the calculated moving object up frame time and section.

  Hereinafter, various operations in the above configuration will be described.

[First Embodiment]
FIG. 3 is a flowchart of the operation in the first embodiment of the present invention.

Step 101) Initial frame input procedure:
This step is a procedure executed in the video input unit 20, from a processing target image stored in the video storage unit 10 to a frame image (processing target frame image) at a processing start position from the video, A frame image (reference frame image) for referring to motion is read. At this time, the frame images are read in chronological order. The initial position of the processing target frame image can be arbitrarily determined such as a video initial frame. The reference frame image is selected as a frame image after a certain frame from the processing target frame.

Step 102) Motion vector calculation procedure:
This step is a procedure executed in the motion vector calculation unit 40. In the initial frame input procedure or the video input update procedure described later, the processing target frame image and the reference frame image input or updated are input, and the image Using the local dissimilarity, a motion vector between them is calculated and output to the memory. The motion vector reference method is described in block matching of existing technology or “Jianbo Shi, et al,“ Good Features to Track ”, IEEE CVPR94, pp.593-600, 1994” (hereinafter referred to as Reference 1). Various techniques such as a feature point tracking technique can be used.

  An image unit for calculating one motion vector is simply abbreviated as an image unit below. For example, an image unit in block matching indicates an image block, and an image unit in feature point tracking is a feature point. Here, as an example, it is assumed that a motion vector calculation procedure using the feature point tracking technique described in Document 1 above is used.

  Also, in each motion vector calculation technique, when the degree of dissimilarity between image units is greater than or equal to a certain level, no motion vector is calculated as a lost image unit.

Step 103) Camerawork model conformity determination procedure:
In the camera work model conformity determination procedure, the camera work model conformity determination unit 50 obtains the motion vector calculated in step 102 and stored in the memory, and whether the motion vector conforms to a predetermined camera work model. This is a procedure for determining. For example, when the camera work model is defined as parallel movement, it can be determined that the time when the variance value of the motion vector calculated in step 102 is large does not conform to the camera work.

  Also, the camerawork model may be a two-dimensional projective transformation model, etc., and camerawork parameters are calculated using motion vectors, and whether the values are abnormal values or not is determined according to the camerawork model. it can.

  Here, the latter example is adopted as an example and described in detail.

  Hereinafter, a camera work parameter calculation procedure for calculating a camera work parameter from a motion vector and a camera work parameter abnormality determination procedure for detecting an abnormality of the value from the calculated camera work parameter will be described in order.

  Step 103-1) The camera work parameter calculation procedure is a procedure executed in the camera work parameter calculation unit 51. The motion vector calculated in the motion vector calculation procedure in step 102 is input and applied to the camera work model. Calculate and output work parameters. As an example of the camera work model used here, a model such as Expression (1) is used. Of course, other matrices that can be used as a camera work model such as a two-dimensional projective transformation matrix may be used. The camera work parameters in this model are a set of four values (parameter set) of (a1, a2, a3, a4). As a camera work parameter calculation method, the method described in Document 1 can be used. For example, when block matching is used for motion calculation, camera work parameter calculation in feature point tracking can be used by regarding the center coordinates of an image block that is an image unit as a feature. In other words, if the point correspondence is calculated by adding the calculated motion vector and the representative point coordinates to calculate the corresponding point, then the feature point tracking technique “Jeda et al.” Feature points are applied to the model by the method using the trajectory between two frames of "Extraction of moving object from", Information Processing Society of Japan, Research Report, 2004-CVIM-143 Mar. 2004 (hereinafter referred to as Reference 2). By classifying the points into points (background points) and other points and using the movement of the background points, the camera work parameters in Expression (1) can be calculated. Further, as another method, it is possible to calculate each camera work parameter by any method for solving the equation (1) using the trajectory of the feature point.

（但し、ｘ，ｙの原点は画像中心座標とする。）
ステップ１０３−２） カメラワークパラメータ異常判定手順：
カメラワークパラメータ異常判定手順は、カメラワークパラメータ異常判定部５３において実行される手順であり、上記で算出されたカメラワークパラメータを入力してその値を検証し、そのカメラワークパラメータが異常かどうかを判定することでカメラワークモデルに則している動きベクトルであるかどうかを判定する手順である。ここで、カメラワークモデルに則していないと判定される場合は、以下の２つの場合である。

(However, the origin of x and y is the image center coordinates.)
Step 103-2) Camerawork parameter abnormality determination procedure:
The camera work parameter abnormality determination procedure is a procedure executed by the camera work parameter abnormality determination unit 53. The camera work parameter abnormality determination unit 53 inputs the camera work parameter calculated above, verifies the value, and determines whether the camera work parameter is abnormal. This is a procedure for determining whether or not a motion vector conforms to the camera work model. Here, the case where it is determined that the camera work model is not followed is the following two cases.

  (1) The camera work parameter cannot be calculated in the above camera work parameter calculation procedure.

  In this case, it can be determined that all of the calculated motion vectors do not conform to the camera work model, and the entire image does not conform to the camera work model.

  (2) When the time change amount of the calculated parameter set is equal to or greater than a threshold value.

  In other words, this is a case where any one of the absolute difference values of each component between the parameter set calculated immediately before and stored in the parameter storage list 70 is equal to or greater than the threshold value. The threshold value used here is set for each parameter component. However, if the previous camera work parameter has not been calculated and is not stored in the parameter storage list 70, it is determined whether the value of each component of the currently calculated parameter set is within a certain range, If any one is off, it is determined that the camerawork model is not followed. In this case, it is detected that the value of the calculated camera work parameter is abnormal, and it is detected that it does not conform to the camera work model.

  Step 104) If the camera work parameter abnormality determination unit 52 determines that the camera work model does not conform to the camera work model, the process proceeds to step 105. If not, the process proceeds to step 106.

  Step 105) When it is determined that the camera work model is not followed, the video time of the current process target frame is stored in the up frame time storage list 80, or the video time of the current process target frame (or reference frame). Is output to shift to the video end condition.

  Step 106) When determining the video end condition, it is determined whether the reference frame time to be read next exceeds the time of the last frame. If it exceeds, the process is terminated, and if not, the video input update procedure is performed. The process proceeds to (Step 107). The output at the end is a sequence of video times of the up frame of the moving object. The next reference frame image selection method is a frame image that is spaced from the processing target frame by the same manner as in the initial frame input procedure (step 101).

Step 107) Video input update procedure:
The video input update procedure is a procedure performed by the video input unit 20, specifies the current reference frame time as the next processing target frame time, outputs the time of the next reference frame to the video input unit 20, By inputting each image, the processing target frame image and the reference frame image are updated.

[Second Embodiment]
FIG. 4 is a flowchart of the operation in the second embodiment of the present invention.

  In the figure, the same parts as those in the process of FIG. 3 described above are denoted by the same step numbers and the description thereof is omitted.

  In the present embodiment, a cut point detection procedure and a case classification procedure based on the determination are added to the processing of the first embodiment described above, and other processing is the same as that of the first embodiment. It is a processing procedure.

Step 201) Cut point detection procedure:
The cut point detection procedure is a procedure executed in the cut point detection unit 30, and uses the existing cut point detection technique to cut the cut point between the processing target frame and the reference frame (video by editing or camera on / off). The discontinuity point) is detected, and the true / false value is output to storage means such as a memory.

  Cut point detection technologies include `` Taniguchi, Tonomura, Hamada `` Video shot switching detection method and its application to video access interface '', IEICE Transactions, D-II Vol.J79-D-II No.4. Various technologies such as “pp.538-546 Apr. 1996” can be used.

  As another example of the detection of the cut time, the cut point detection process is performed on the entire video (or a part of the video) in advance, and the time of the cut point is calculated and stored in a memory (not shown). , Input. This can be performed when cut detection in principle cannot be incorporated into this loop (the above-mentioned document 1) and cut point information input in advance by hand is available. In this case, when there is a cut time between the time of the processing target frame and the time of the reference frame, it is determined that there is a cut.

  In addition, when the cut point detection technology can detect a cut with a time effect such as a video crossfade as a section (cut section), or when an error is expected to some extent in the accuracy of the cut point detection, the above “ The condition that “the cut time exists between the processing target frame time and the reference frame time” means that there is an overlap in the section between the processing target frame time and the reference frame time and the cut section, or the error width. The condition processing is changed to the same condition.

  Step 202) If it is determined that there is a cut point, the process proceeds to Step 106, the end condition is determined, and otherwise, the process proceeds to the motion vector calculation procedure in Step 102.

[Third Embodiment]
FIG. 5 is a flowchart of the operation in the third embodiment of the present invention. In this figure, the same steps as those in FIGS. 3 and 4 are denoted by the same step numbers, and description thereof is omitted.

  The operation shown in the figure is obtained by adding a moving object up frame determination procedure after the camera work parameter abnormality determination procedure of the second embodiment, and changing the subsequent branch condition to whether or not a moving object up frame is detected. . Only the changed processing procedure will be described below.

Step 103-3) Moving object up frame determination procedure:
The moving object up frame determination procedure is a procedure executed in the moving object up frame determination unit 53. When the determination result in the camera work parameter abnormality determination unit 52 is input and the input indicates an abnormality in the parameter value, it is output that the moving object up frame is detected, and the process proceeds to the condition determination procedure. When the input is normal determination, the moving object up frame determination unit 53 performs the following determination.

  It is determined whether or not the ratio of the number of image units (blocks or feature points) having a large difference between the calculated motion vector and the motion vector that can be calculated from the parameter set is equal to or greater than a threshold value. This will be described in detail below.

Motion is calculated from the parameter set vectors (u _{p, v p)} it is, in the case of using the camera work model of Equation (1) can be calculated by the equation (2).

但し、式（２）内の（ｘ，ｙ）は、動きベクトルの算出単位（画像単位）の座標である。この座標は、例えばブロックマッチングの場合は画像ブロックの中心座標、また、特徴点追跡の場合は特徴点の座標である。

However, (x, y) in the equation (2) is a coordinate of a motion vector calculation unit (image unit). This coordinate is, for example, the center coordinate of the image block in the case of block matching, or the coordinate of the feature point in the case of feature point tracking.

Calculating a vector difference between the motion vector calculation procedure motion vector calculated at (step _{102) (u p, v p} ), calculates the number of coordinates is the square value of the vector length is more than a predetermined value It is assumed that the moving object up frame is detected when the value obtained by dividing the number of coordinates by the total number of coordinates is equal to or greater than the threshold value. In other cases, it is determined that the moving object up frame is not detected.

  As another detection method, when feature points are used for motion detection, as described above, the difference between the motion vector of each feature point and the motion vector calculated from the parameter set is divided into a feature point and a small feature point. For each feature point having a large difference, for example, there is a method of performing up-detection by using a distance to another feature point closest to the feature point as a weight of the feature point and performing threshold processing on the sum of the feature points. This takes into account the size of the area that each feature point bears.

  When the feature point tracking method is used as the motion vector calculation method, the coordinates between the coordinate values of the corresponding points calculated by the feature point tracking and the movement destination coordinates by the camera work of the representative points obtained from Expression (1). Similar processing can be performed using the vector length.

  This condition is an additional condition for not complying with the camera work model, and is a condition for determining whether a large object exists in the image when the camera work is normally determined.

  Step 301) When the moving object up frame is detected, the process goes to Step 105, where the video time is stored in the up frame time storage list 80. When it is not detected, the process goes to Step 106, and the video end condition is continued. Judgment is made.

[Fourth Embodiment]
FIG. 6 is a flowchart of the operation in the fourth embodiment of the present invention. In this figure, the same steps as those in FIGS. 3, 4, and 5 are denoted by the same step numbers, and description thereof is omitted.

  The operation shown in the figure is a processing procedure in which a video section calculation procedure is added to the operation in the third embodiment, and when there is a cut point, each subsequent process storing the video time of the moving object up frame is a video. This is the section calculation procedure. Here, only the added video section calculation procedure will be described.

Step 401) Video section calculation procedure:
The video section calculation procedure is a procedure performed in the video section sending unit 60, and is a process for collecting the detected moving object up-frame times stored in the up-frame time storage list 80 as a video section. The following two examples are given as this processing method.

  (1) When branching from “with cut” in step 202, a set of minimum and maximum values of moving object up-frame times stored in the up-frame storage list 80 is output as one video section. Are deleted from the up frame time storage list 80.

  By doing so, it is possible to collectively output the moving object up frame time in the video section in each cut.

  (2) When the video time is stored in the upframe time storage list 80, the start point and the end point are stored as the same video section. If there is a video section stored in the previous up-frame time storage list 80, it is referred to, and if the time difference from the end is within a threshold, the end of the video section is updated to the currently stored video time. If it is greater than or equal to the threshold, one video time is stored as a video section having the same start and end points. Further, when the processing procedure is reached after branching from “with cut” in step 202 (step 202, Yes), the video section currently stored in the upframe time storage list 80 is output, and the upframe time Erase from storage list 80. The time difference threshold processing may be frame image similarity threshold processing. In the case of a certain degree of similarity or more, it is assumed that the above-described terminal update is performed.

  By doing so, it is possible to detect a section in which the moving object is captured as a video section within a certain range and within the same cut.

  The video section in which the moving object calculated as described above is shown as an up shot is represented as a moving object up shot. When a cut is detected in the cut point detection procedure (step 201), it is checked whether or not a moving object upshot is newly detected. If it is detected, one frame image is arbitrarily selected in the video section. The time of the video section is output.

[Fifth Embodiment]
FIG. 7 is a flowchart of the operation in the fifth embodiment of the present invention.

  This figure shows a method for detecting the moving object up, and the differences from FIG. 6 are the initial frame input procedure, cut point detection procedure, motion vector calculation procedure, and video input update procedure.

  Each is changed to a process corresponding to a plurality of frames.

Step 501) Initial multiple frame input procedure:
In the initial multiple frame input procedure, the video input unit 20 acquires a fixed number (F) of reference frames at a predetermined frame interval.

Step 502) Procedure for detecting cut points between multiple frames:
In the inter-frame cut point detection procedure, the cut point detection unit 30 determines whether or not there is a cut point between each of the plurality of reference frames and the processing target frame image by the same method as described above. A result of the determination, f _a th in F reference frames _picture: When the first cut point is detected in the reference frame (f a F or less constant natural number), the selected reference frame to the frame There is a method of determining that a cut has been detected by setting the frame to be the next processing target frame. Also, the frame interval with the reference frame to be processed next is reset to the interval in the initial input means.

  Step 503) When there is a cut point, the routine proceeds to step 401, and when there is no cut point, the routine proceeds to step 504.

Step 504) Multi-frame motion vector calculation procedure:
In the inter-frame motion vector calculation procedure, the motion vector calculation unit 40 calculates a motion vector between two frames for each of a plurality of reference frames for one processing frame, and determines a reference frame using the calculation result. . The motion vector calculation procedure between two frames is the same as the motion vector calculation procedure (step 102). When this is performed for F reference frames, F motion vectors are calculated. Using these F motion vectors, one of the frame images is selected as a reference frame image to be adopted. In the selection method, for example, the number of image units (for example, lost feature points) that have a high degree of dissimilarity when calculating motion or cannot be calculated by other methods is minimized (or less than a certain value). For example, a method of setting a frame image having a maximum frame interval among the reference frames is considered. The frame interval adopted here is output to the video input unit 20 as f _sel and used in the video input multiple frame update procedure (step 505).

Step 505) Video input multiple frame update procedure:
In the video input multiple frame update procedure, the reference frame calculated above is set as the next processing frame. Further, a maximum of F next reference frames are read from the video storage unit 10 using the frame interval f _sel .

For example, when the maximum value of the frame interval between the processing target frame and the reference frame is set to F _Max (> F), the reference frame is determined at equal intervals in the range of 0 <f <f _sel {(f _sel −1) / 2, F / 2} (truncated after the decimal point), min {(F _Max −f _sel +1) / 2, F / 2} at equal intervals in the range of f _sel ≦ f ≦ F _Max ( It is assumed that each reference frame is selected and read. For example, when F _Max = 10, f _sel = 9, and F = 4, the selected reference frame intervals are 3 (8 × 1/3), 5 (8 × 2/3), 10 ((9 + 10) / 2 (rounded up), and when F _Max = 10, f _sel = 7, and F = 2, the selected reference frame interval is 3 (6/2), 9 (7 + 10) / 2) is there. In addition to the frame interval update method, a plurality of reference frames may be selected using an arbitrary arithmetic operation.

  By changing the frame interval as described above, it is possible to reduce the frame interval in a video section with a large motion and suppress erroneous calculation of camera parameters. Also, in a video section with little motion, the overall processing speed is improved by increasing the frame interval.

  Further, by expanding the frame interval in a video section with little movement, it is possible to detect the moving object up frame using the movement of the moving object at a wide time interval. That is, it is possible to detect the moving object up frame even when there is little movement.

  The operations of the flowcharts shown in FIG. 3 to FIG. 7 are constructed as a program, installed in a computer used as an animal up-frame detection apparatus and an animal up-shot detection apparatus, executed, or distributed via a network. It is also possible to make it.

  Further, the constructed program is stored in a portable storage medium such as a hard disk connected to a computer used as a moving object up-frame detection device or a moving object up-shot detection device, a flexible disk, a CD-ROM, It can be installed and run on a computer or distributed.

  The present invention is not limited to the above-described embodiment, and various modifications and applications can be made within the scope of the claims.

  The present invention is applicable to a technique for generating a representative image of a video in the video structuring technique.

It is a figure for demonstrating the principle of this invention. It is an apparatus block diagram in one embodiment of this invention. It is a flowchart of the operation | movement in the 1st Embodiment of this invention. It is a flowchart of the operation | movement in the 2nd Embodiment of this invention. It is a flowchart of the operation | movement in the 3rd Embodiment of this invention. It is a flowchart of the operation | movement in the 4th Embodiment of this invention. It is a flowchart of the operation | movement in the 5th Embodiment of this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Image | video memory | storage part 20 Image | video input part 30 Cut point detection part 40 Motion vector calculation part 50 Camera work model conformity determination part 51 Camera work parameter calculation part 52 Camera work parameter abnormality determination part 53 Moving object up-frame determination part 60 Image | video area calculation part 70 Parameter storage list 80 Upframe time storage list

Claims (10)

In a moving object up frame detection method in a moving object up frame detection device for detecting a video frame image (moving object up frame image) in which a moving object is relatively large from a video storage means in which a video is stored,
A motion vector calculation procedure for calculating a motion vector using a plurality of frame images read from the video storage means and storing the motion vector in a memory;
A camera work model conformity determination procedure for acquiring the motion vector from the memory and determining whether the entire motion vector conforms to a predetermined camera work model;
In the camerawork model conformity determination procedure, when it is determined that the camerawork model does not conform to the camerawork model, a procedure for detecting as an animal up-frame,
A method for detecting an up-frame of an animal body, comprising:   Before the motion vector calculation procedure, a process of detecting a cut point of the frame image read from the video storage means is performed. If the cut point is not detected, the process proceeds to the motion vector calculation procedure. The moving object frame detection method according to claim 1, wherein a cut point detection procedure is further performed. In the camera work model conformity determination procedure,
A camera work parameter calculation procedure for acquiring the motion vector from the memory, calculating a camera work parameter, and storing it in a parameter storage unit;
A camera work parameter abnormality determination procedure for obtaining the camera work parameter from the parameter storage means and determining whether or not the camera work parameter is abnormal;
The method for detecting an animal body up-frame according to claim 1 or 2, wherein: In the camera work model conformity determination procedure, when determined to be normal,
Of the frame image units used when the camera work parameters are obtained from the parameter storage means and the motion vectors are calculated in the motion vector calculation procedure, the frame image units that do not conform to the camera work model are Calculated using the difference between the unit motion and the camera parameter, detects the animal up frame based on the ratio of the number of image units or the weighted sum, and the video time of the animal up frame is up-framed. 4. A moving object up frame detection method according to claim 3, wherein a moving object frame determination procedure stored in the storage means is performed. An animal up-frame detection program for detecting a video frame image (animal up-frame image) in which a moving object is relatively large from a video storage means in which video is stored,
In a computer having a plurality of memories, up frame time storage means and parameter storage means,
5. A moving object frame detection program for executing the moving object frame detection method according to claim 1. A storage medium storing a moving object up frame detection program for detecting a video frame image (moving object up frame image) in which a moving object is shown relatively large from a video storage means storing a video,
In a computer having a plurality of memories, up frame time storage means and parameter storage means,
5. A storage medium storing a moving object up frame detection program, wherein a program for executing processing for realizing the moving object up frame detection method according to claim 1 is stored. In a moving body upshot detection method in a moving body upshot detection device that detects a video section (moving body upshot) in which a moving body is captured from a video storage means in which a video is stored,
5. The procedure of the moving object up frame detection method according to claim 1, further comprising: storing the detected moving object up frame in a storage means in time series to calculate a temporal unit. A method for detecting an upshot of a moving object, characterized in that a video section calculation procedure is added. The frame in which the moving object is shown up from the image storage means in which the image is stored, the moving object up frame for detecting the video section (moving object up shot), or the moving object up frame in the shot detection device In the shot detection method,
The motion vector calculation procedure is used for the procedure of the moving object up frame detection method according to claim 1 or the procedure of the moving object up shot detection method of claim 7. A moving object up-frame or shot detection method characterized in that a video input update procedure for selecting a plurality of frames to be performed using the calculated motion is added. A moving body upshot detection program for detecting a video section (moving body upshot) in which a moving body is captured from a video storage means in which the video is stored,
In a computer having a plurality of memories, up frame time storage means and parameter storage means,
9. A moving object upshot detection program for executing a process for realizing the moving object upshot detection method according to claim 7 or 8. A storage medium storing a moving object upshot detection program for detecting a video section (moving body upshot) in which a moving object is captured from a video storage means in which a video is stored,
In a computer having a plurality of memories, up frame time storage means and parameter storage means,
9. A storage medium storing an animal upshot detection program, wherein a program for executing a process for realizing the animal upshot detection method according to claim 7 or 8 is stored.

Images